AI is vindicating the role of the Software Tester

Figure 1. Image generated by AI.

Traducción al español, aquí.

Over the last few weeks, I've read and heard all sorts of things about AI (in software development), both good and bad. It's becoming increasingly clear who's in favour of AI and who's against it (although it seems the former are gaining the upper hand). If someone were to ask me my stance on the matter, I would reply that I view the issue with a certain degree of scepticism, but also with a certain degree of optimism (I cannot afford to be left behind on something that affects my entire professional career and which I also use daily at work). I can discuss my personal views on AI in another post.

Today I want to focus on something I've been mulling over for a while. And that is that, with all the scaremongering surrounding AI-driven software development, I can see the following very clearly: AI is turning us all into software testers.

Yes, you read that right: AI is turning us all into software testers.

I hope I don't get too tangled up in this, but I'll start from the beginning. Ever since the first AI agents were released, people (both technical and non-technical) began experimenting with them and developing all sorts of applications. At the same time, it became very clear that many of these creations had critical bugs (as well as serious security vulnerabilities). Over time, software development practices using these agents became standardised, and slightly more robust applications began to be developed (though still not free from defects and security vulnerabilities). There is a wealth of literature online on how to create perfect prompts to ask AI agents to generate solutions to the technical or business problems we face in our day-to-day work. These help us make the agents' work more efficient and use fewer tokens.

There is a lot of talk now that, as developers and testers, we will spend most of our time (at least for now) orchestrating what AI agents do. And they are not wrong. There are many things involved in this orchestration: requirements analysis, planning, design, environment configuration, execution, monitoring and completion (reporting). Does all this sound familiar?

AI is gradually turning us all into software testers, although many (I'm not saying all, as there are exceptions) of our developer colleagues aren't keen on the idea of software testing. And this whole process is «manual» (though it would be more accurate to call it «exploratory»).

If we've jumped on the AI bandwagon, we now spend most of our time analyzing whether it's doing what we've asked, and correcting or adjusting where necessary. Of course, this doesn't mean that at some stage in the software development process we don't have to get our hands dirty and write code (or solve something that the AI can't).

But there's something else I'd like to mention. I've noticed there's a lot of fuss about AI taking away the jobs of all of us who work in software testing, as it performs the tasks we do. I'd take this claim with a pinch of salt, as I've seen quite the opposite.

I've worked with several developers (and was once one myself) and follow a number of them on social media. I've seen how most of them are fascinated when tools come onto the market that seem to magically perform the tests they don't want to do. I observed this phenomenon when Cypress was released and later when Playwright came out with all its characteristic features. And I see the same phenomenon when people say that AI performs the tests for them.

There is a significant bias amongst developers when it comes to understanding what a software tester does. Many think that automation is testing, and that is not the case. Many also think that by running automated e2e tests, they are covering what their code should do (sidelining unit tests and other types of testing they ought to be doing). Any sensible person with knowledge of software development knows that not testing from the earliest stages is more costly and counterproductive, and that at the beginning the process will always be exploratory, not automated.

Michael Bolton, a leading figure in the testing industry for decades, sums it up well when he says that testing is a process of analysis, intuition, reflection, criticism and feedback, and that the use of automation tools is something that helps us streamline our work to a certain extent.

There is a very significant risk involved in delegating software testing to AI. Agents learn from us (human beings). Developers are human beings (for now). If agents are learning from us, who can guarantee that AI will perform software testing properly? Many of us know that developers have their tricks for making tests pass «successfully» and ensuring everything on the dashboard or report appears green. Likewise, they have their own biases when conducting their tests. Several years ago, Blanca Moreno (a fellow tester from the QA Minds community) told us, half-jokingly: «Developers will treat their code as if it were their baby. They will always see it as beautiful. But the testers' job is to prove the exact opposite.»

AI can lead us to believe that everything it has generated is correct (when this is often not the case). We must carefully analyse what the agents are generating for us. Experience helps us to avoid as many errors as possible during development (and not to become complacent during the process).

I recently saw this on LinkedIn. This is something that was bound to happen.

Summarizing. AI is turning us all into software testers, and that's a good thing! We mustn't write off QA/QC (and we shouldn't. Now more than ever, the roles of software tester and cybersecurity engineer are more essential than ever!). Automation is not the same as testing. We must continue to educate the community about the importance of testing (and what testing actually entails). Experience helps us write good prompts that we can send to the agents, thereby obtaining higher-quality software. We must be more critical of what AI generates for us and not blindly trust it. And let's not forget to drink water (it's important to stay well hydrated).

Suggested readings.-

• Bolton, M. (2024, June 25). Sharper Terms for “Manual Testing.” Developsense.com. https://developsense.com/blog/2024/06/sharper-terms-for-manual-testing.
• Bolton, M. (2021, August 24). Alternatives to “Manual Testing”: Experiential, Interactive, Exploratory. Developsense.com. https://developsense.com/blog/2021/08/alternatives-to-manual-testing-experiential-attended-exploratory.
• ‌Bolton, M. (2013, February 24). “Manual” and “Automated” Testing. Developsense.com. https://developsense.com/blog/2013/02/manual-and-automated-testing.