A Tale of Test Automation and Artificial Intelligence

Note: I originally published this article with BlogTrunk back in January 2020. The BlogTrunk website is no longer available and hence I am re-publishing in my blog. Credits goes to BlogTrunk for publishing it first.

History of Software Testing

In the 1980s, Software Testing was performed manually by all organizations. Then, from the beginning of the 1990s until around 2003, the first usage of test automation came into existence by using sophisticated tools called Test Automation Tools. Over the period from 2004 to 2009, multiple tools started evolving and open-source projects, which are developed by test enthusiasts, started becoming very popular and many people started to adopt new tools and strategies till the last decade and of course until today.

But, what’s next? The future of Test Automation is going to revolutionize the way test automation happens today. Artificial Intelligence will be at its peak and many organizations will start adapting AI-based test automation tools for their testing. There are many product-based companies that have started investing heavily to bring up the best AI-based solution for multiple challenges the users (testers) face.

In this article, we will see some of the major challenges that a test engineer face to do automation and various kinds of AI-based solutions available in the market for test automation.

Major Challenges with Today’s Test Automation –

Resources with the right skill sets
Adhering to best practices
Maintenance of Test Scripts
Flaky Test Results
Time-consuming RCA’s

Artificial Intelligence in a nutshell

In simple terms, Artificial intelligence (AI) is the ability of a computer program or a machine to think and learn.

Having said that, we need to understand other terminologies like Machine Learning (ML) and Deep Learning (DL).

“Machine learning (ML) is an application of artificial intelligence (AI) that provides systems with the ability to automatically learn and improve from experience without being explicitly programmed.”

“Deep Learning (DL) is a subset of machine learning in artificial intelligence (AI) that has networks capable of learning unsupervised from data that is unstructured or unlabelled.”

AI today and in future will get involved in many of our day-to-day gadgets which we use. Say, for example, Google Assistant and Alexa which we use every day are the best examples of AI-based applications. It can understand what we are looking for and replies to us with the correct and most accurate results. The application of ML is amazing. For example – Google Assistant can recognise your voice and replies with your calendar schedules. It uses ML to learn your voice tones every time you interact with it.

AI in Test Automation

Enough of real-time examples, let’s now see how AI can help us with our Test Automation which we do as a profession.

As I said earlier, many product-based companies are investing heavily in AI to develop a tool that can perform tasks that humans do or can’t do in no time.

Considering the above challenges which we face in Test Automation, companies like Applitools, Testim.io, and Functionize have done amazing research on bringing their AI products.

In fact, with the enhanced record and playback functionalities, many tools are helping testers to save a lot of time. There is a misunderstanding among testers that record and playback will never work as expected. Gone are those days when we think like that. Testim.io has an excellent record and playback functionality that captures the images and patterns and then executes the tests accordingly. Same with Selenium IDE, though it is not AI-based by nature, it can be enhanced by integrating tools like Applitools to perform Visual Testing.

Visual Testing is another dimension that the testing world is moving towards. It is a concept of image comparison between two screenshots of the applications and identifies the differences and provides us with the differences. Of course, there are many open-source and other tools that do the work of image comparison. But Applitools has taken a step back and thought of embedding AI in it to do tasks that normal image comparison tools fail to do. Basically, identifies the exact differences irrespective of pixel changes as other tools work in the principle of pixel-by-pixel comparisons. It is said that Visual Testing tools, when properly chosen and used, can reduce functional testing to a certain amount.

Another major issue that AI helps is with the dynamically changing locators of components of the applications. We all face the issue of dynamic locators, which always gives us the pain to modify the scripts more often. Though we use powerful XPath with RegEx, most times it fails. AI-based tools can identify multiple locators of the components on the go, which means one component will have more than one locator specified/identified. During the run time, when the tests fail because an element is not visible, the AI has the option to automatically allocate the other locators that can identify the element. In this way, the maintenance of the test scripts will drastically reduce.

Flaky test failures are one of the most common failures that we get while executing test cases. These failures can include the browser getting crashed, active login sessions, caching, the environment being unavailable. To address this kind of issue and save time, we need to use AI to analyse the failure and retry the tests. Say for example, if the test execution has 1000 tests to execute, and if the first test fails because of the environment is unavailable, the other 999 issues should not be executed. So, this feature will avoid flaky results and save a lot of time.

We as a tester, spend more time in analysing the failures of the test results. In a typical Agile-driven environment, there is a possibility that we run the regression suite every night against the build or in some cases every weekend. So, analysing the test failures give more pain when we run thousands of tests. The usage of AI is quite in this case. Tools like ReportPortal.io does a remarkable work helping us to analyse the failures and categorize it accordingly. With its ML capability, it does analyse the test results and learns itself with the pattern of the test failures which eventually reduces a lot of time.

So, should I be nervous about AI in Test Automation?

No!!! All these AI-based tools are developed to help the testers and the delivery team to make life easier for them. In the world of DevOps, AI will play a vital role along with test automation and help diagnose the issues much before and enabling the team to shift left with confidence.