Artificial Intelligence in Quality Assurance: Impact & Role

Software testing efficiency and software testing effectiveness are two key metrics that determine the overall progress of a test strategy. Artificial Intelligence (AI) and Machine Learning (ML) focus on these two parameters in testing. AI & ML can optimize risk coverage, prevent redundancies, perform portfolio inspection, detect false positives, diagnose defects automatically, and analyze user experience. 

“It is estimated that more than 60% of enterprise test cases portfolio’s test cases are redundant. AI identifies such test cases that are physically and logically identical and eliminates duplicates, which do not add any business value and can be removed without decreasing the business risk coverage.” 

AI can maximize defect detection and risk coverage while minimizing costs, execution time, and the number of test cases by identifying the optimal test sets. It can uncover weak spots in test case portfolios by tracking flaky test cases, unused test cases, untested requirements, and those test cases that are not linked to the requirements. Additionally, AI has self-healing automation properties, which means it can heal the broken automated test cases and make test automation better resilient to changes. 

Overall, AI makes software testing smarter while promoting higher efficiency and effectiveness.  

Kevin Pyles, Director of QA (Quality Assurance) at Domo Inc., discussed the impact of AI for Quality Assurance on our recent podcast. This blog is an excerpt from their interview on the QATalks podcast.

The present state of AI-driven software testing 

AI has been buzzing around since the 1900s, and it still upholds the hype across the globe. Everyone keeps talking about the possibilities of the role of AI. However, there is still a wide gap between where AI reached today and where it must go.  

Kevin explains the present state of AI in software testing,  

“The vision, the hope for everybody, is that someday, AI can do the testing for us. We are not there yet. I am not promoting that. But what is here is AI-based tools and AI that help us with our jobs.  

So, we should not look at it as AI replacing testers yet. We should not look at it as AI replacing most of our processes yet. What AI does right now is it helps us be better testers, meaning it takes out some of that mundane work that we would not like to do anyway.  

Or, as we will hear a little bit later, AI can help us do things like prediction or analytics better than we have done in the past, which just allows us to do our jobs better.” 

AI resolving the ‘Test Automation trap’ 

Software testing is a time-consuming and cost-intensive activity. A challenge with regular test automation is that by the time the test code is completed, the requirements start changing, and applications start evolving with regard to business functionality and UI (user interfaces).  

This means that the whole effort put into developing the test code goes in vain, and you must adapt the test automation needs accordingly. Kevin calls it the ‘Test Automation trap”. 

He explains, “Test automation trap is when the test teams are not getting enough time to be able to do the failure triage from the previous test run before building the next test automation code.  

That is where AI can be used to solve this dilemma and accelerate manual testing. With some of our clients, we can apply AI in the context of prioritizing test cases and maintaining the test automation code in an automated manner, as opposed to manually investigating what needs to be changed. 

And I expect that over a period, we will see that it can play a significant role in the context of analyzing the test results and deciding on what needs to be tested and things like that, which can happen freely without human intervention.” 

Realizing the DevOps transformation with Intellectual Property 

In a way, the DevOps transformation is like the previous transformations, say, Agile or Waterfall. Even DevOps has been rebranded to DevSecOps or QASecDevOps so that everybody is involved in this transformation.  

Kevin advises, “What you really need to do is look at your business, what does your business need, and what best practices are good practices out there that can be applied to your company.  

And why are we always looking externally for solutions when we have many smart people internally who could help us develop our own techniques and business processes, dev processes, and techniques?  

We should do a better job there and we can improve what we have. And so, we need to look at what makes sense for our companies and apply whatever new terminology we want to apply.  

But more importantly, apply practices that will help us develop better products that suit the needs of our customers and our businesses.” 

AI and ML-driven dashboard for greater productivity 

There is no dearth of data in the present ecosystem. However, there is a concerning dearth of the abilities to collate the available data in one place, derive valuable insights from this information, and apply it to day-to-day operations to improve productivity. 

Intelligent dashboards, like the one Kevin developed at Domo, allow the stakeholders to pull and visualize the data from anywhere and share it across the company for real-time status updates.  

While elaborating on his perspective on the dashboard’s power, Kevin states, “They give a quick status update and I believe they answer just about any question that executives would ask – So, what’s the status of the project? Are we ready to ship? How many bugs are coming in? And then how do we feel about the release? And all those questions are answered using this dashboard.” 

As more and more organizations are moving into agile and DevOps, and there is a great need to get real–time insights and analytics, the teams at Cigniti are able to act decisively in terms of the changes that need to be made to the projects.  

And of course, something like this is also needed to be able to make release readiness decisions. The dashboard we built can analyze the data and provide it from a descriptive, diagnostic, predictive, and prescriptive point of view.  

So, not only can it tell you what happened, but with the AI/ML features, it can predict what could happen based on previous projects of comparable size and scope.  

The dashboard is built to drive better business outcomes, improve predictability, and accelerate the transformation within the organizations. 

To sum up 

There is certainly a lot of hype around AI in Quality Assurance, and constant efforts are being made to narrow the gap between this hype and reality. We might not be where we want to be in terms of AI-led testing yet. But we will be there soon. 

Cigniti’s BlueSwan™ is a next–generation Proprietary Testing platform. Aligning with the needs of the digital era, BlueSwan is Cigniti’s enabler in becoming the leader in independent software testing services.  

It also fortifies our IP-led software testing services approach along with the Cigniti 4.0 services and Domain-centric approach for offering world–class testing services to businesses across different verticals.  

To know how our innovative solutions and intellectual property can help your organization, schedule a discussion with us.