Testing is the most valuable AI investment in DevOps, finds survey

Wed 7 Aug 2024

Continuous testing and quality engineering company, Tricentis, has found DevOps practitioners ranked testing as the most valuable (60%) area for artificial intelligence (AI) investment across the delivery cycle.

Tricentis released the findings in the AI report ‘AI-augmented DevOps: Trends Shaping the Future’, aiming to understand to what extent the anticipated benefits of AI in DevOps have been realised today, and how a lack of trust, skills, or other challenges could affect its adoption.

“DevOps teams looking to get started with AI should look no further than their testing processes. AI in testing helps to detect, auto-heal, and predict defects during development, as well as identify which tests need to be run based on high risk,” said Mav Turner, Chief Product and Strategy Officer at Tricentis.

Turner said when coupled with low-code/no-code technology, this means that, regardless of a team’s technical expertise, AI can significantly contribute to overall software quality.

The research found DevOps teams are realising the benefits of AI, with mature DevOps teams who have adopted AI significantly more likely (30%) to rate their teams as either extremely or very effective.

The biggest challenges DevOps teams are using AI to address are developer team efficiency (60%), reducing the skills gap (54%), cost reduction (47%), and software quality (42%). In fact, almost one-third (32%) of respondents estimate AI-augmented DevOps tools will save teams over 40 hours per month, equivalent to an entire workweek.

The 2024 results show teams use AI to augment a wide range of testing tasks, including test planning and deciding what to test (47.5%), test case generation (44%), and analysing test results (32%). Nearly half (42%) of respondents also expect AI to perform a risk analysis of code changes, helping QA teams focus on code areas with the greatest risk of errors to quality.

AI Innovation Concerns Uncovered

Other findings revealed regulation is expected to help build trust, but others worry it will stifle innovation and outcomes. Nearly two-thirds (63%) of those surveyed view increased regulation as a way to build confidence in AI across their organisation, while a smaller number of DevOps practitioners (16%) feel increased regulation will hinder or stifle the potential impact of AI in organisations.

Humans also remain vital to ensuring software quality, but there is a shortfall in AI skills. Data shows that humans are still very much ‘in the loop’ with over two-thirds (71%) of respondents checking outputs at least half of the time, and almost one in five (19%) claiming to check AI outputs all of the time. However, a lack of AI skills (28%) is seen as the greatest hurdle to AI adoption in DevOps.

Generative AI (GenAI) and AI copilots are key drivers of AI adoption, with GenAI (45%) now being the most widely adopted type of AI used by DevOps practitioners. In particular, AI copilots are also on the rise, with use cases in planning, code development, and software testing.

Tricentis’ Chief Product and Strategy Officer added as AI technology is further developed, training software development and quality engineering teams with the necessary skills to effectively work with AI will be absolutely critical.

“As DevOps teams continue to mature, testing will be pivotal to realising their investment in AI-augmented DevOps tools and practices,” said Turner.

The report surveyed over 500 DevOps practitioners, managers, and executives from small, mid-size, and enterprise organisations across the globe and in several industries, including financial services, healthcare, and manufacturing.