Over the years I have delved into several areas of testing and through trial and error I have learned what works and what doesn't work in terms of testing and quality. As a result, I have been able to speak in the field of testing at several conventions and seminars of TestNET, Noordertest and the National Test Day, and have given many workshops at CIMSOLUTIONS customers.

The opposite is specialization as you now encounter with tools like test.ai. This tool focuses exclusively on webshops. The goal is to eventually offer a test service for webshops that can work completely independently. There are tools with AI that specifically target the app market. It is also important for apps to be able to run quickly and on as many platforms as possible.

With larger test frameworks, people are also starting to get interested in offering AI. Functionize, Appvance and Eggplant are examples of American test platforms that have been doing this for some time and each in their own way.

Testing becomes more creative
With the advent of tools such as Selenium, repeated testing has been removed from the tester and manual testing could focus more on creative test situations. This will be reinforced if AI testing tools take over even more standard testing from us.

In addition, AI will help to write and run many of the (technical) unit tests, so that the tester can focus more on functional and more complex non-functional tests. Manual, creative testing is shifting to verifying requirements and delivering value to the organization.

How does the role of the tester change?
The work of a tester will continue. However, the activities required to provide insight into quality and risks will change. This will also change the necessary knowledge and skills. We see the following developments;

Clear formulation of objectives
This white paper shows that an AI can support us in all kinds of aspects of the testing process. There's even a possibility of the AI ​​exploring itself. This means that we must clearly indicate what we actually expect from the application and what we would like to have tested. Either; more attention to requirements, acceptance criteria and test goals. Because in the most extreme scenario, the AI ​​​​could explore without any steering and we would spend months assessing the results. A good set of expectations in advance can prevent this.

Incidentally, this does not mean that we should stop exploring the application ourselves, for example through exploratory testing. It may become even more important; the human tester does a good exploration of the playing field, we leave the large-scale and repeated testing to the AI.

Managing uncertainty in the testing process
The application of self-learning AI (or machine learning) ensures that we do not get absolute 'right' or 'wrong' results. As a tester, we will have to make our own interpretation of a certainty percentage that the AI ​​returns. So if the AI ​​considers a test case to be 87% successful, how do we report this and what follow-up action do we take? This will be different per project; it is human work to determine this and sometimes to do additional manual testing where necessary.

The test manager will also experience this uncertainty. If most tests pass at least 90% and some tests pass 75%, are we done? This is an interpretation that the test manager must do and nicely returns to the previous theme; make sure there are clear expectations in advance. Moreover, it requires the skill to interpret large amounts of figures. And that brings us to the next point.

Knowledge of statistics
Interpreting large numbers of results and certainty percentages requires the tester and the test manager to have some statistical knowledge. Even if ready-made tools claim that it is happy to perform the interpretation on your behalf, it is important to have a well-founded opinion about all the figures that an AI returns. Should the tester and test manager then become a part-time data scientist or data engineer? That's still hard to say, but some basic knowledge will certainly help you on your way.

Better knowledge and understanding of testing tools
There is a very diverse amount of tools available. Regardless of whether they are 'truly AI', each has a different way of contributing value, in different environments. The trend that is already visible is that a large number of tools are being linked together to achieve the test goals. AI-assisted tools will be added to this.

It is expected that the automated part of the testing process will increase, compared to manual testing and test support activities. This makes knowledge of which tools you can use, in which combination and especially for what purpose, even more important.

So the test area is still expanding. At CIMSOLUTIONS we see that customers demand expertise in the field of testing that they do not have in-house. CIMSOLUTIONS facilitates this and works closely with its customers to arrive at a suitable solution for an often very specific problem. But there is also an increasing demand for test knowledge in the broad sense of the profession. I am currently setting up a general functional testing course that will be used specifically to involve the business with limited testing experience in a legacy or DevOps environment much closer to the development of software in order to achieve the required business value.

In other words: there will always be a role for someone who thinks about quality, who asks critical questions and who takes care of communication about goals and priorities. To quote Jeremias Rößler [ad.2]; the chance that we will ever build software automatically is greater than that we will test it fully automatically. Because precisely when an AI builds an application itself based on a model, it is important that it is not tested according to the same model.

Marek Loff
Project Manager / Technical Consultant

[ad.1]  Information about Working Group Testing and AI
[ad.2]  Dr. Jeremias Rößler – When Will AI Take My Job As a QA Manager