It is generally accepted that a serious IT company cannot do without a testing department. By implementing the principle of division of labor, testers are more effective at catching bugs, relieving developers, and improving product quality and value.
We are used to the Q.A. Passed marking on various electronics and expect that control will work in the same way with IT products.
But development is not conveyor production. Global experience (and experience of an IT integrator kt.team including) says that in complicated environments, evaluation criteria are determined for each feature — this is clearly beyond the competence of ordinary testers. With their involvement, the value of the product is eroded, its cost and development time increase, and conflicts begin within the team due to incorrectly divided responsibilities. I'll take a closer look at why this happens.
Testing in a confusing environment
“Entangled environments” is a concept from the Kenevin framework that divides all environments into four types: simple, complex, confusing, and chaotic.
In linear (simple and complex) environments, testers need to strictly follow the regulations. Everything is clear there: just verify the plan and the fact, “ring the details” or plug the device into an outlet. The test result is unambiguous; the product either complies with the regulations or does not.
But that's not the case in a confusing environment. It does not have a final result and no fixed path by which this result is achieved. There is no standard for acceptable and excellent results — only a vector of development. The evaluation criteria typical for testing do not match the fundamental principles of Agile, which are most effective in a confusing environment. In a confusing environment, responding to changes and feedback comes to the fore, not regulations.
However, these are words. What does it look like in real life?
Once at an IT company...
Why doesn't this situation benefit either the developer, the tester, or the product?
Both of them are put in a situation where the quality of the result fades into the background and conflicts are inevitable. Here are the factors that trigger them.
Implicit product responsibility
It's bad if developers start thinking that their job is to deliver code, not value. In the case above, with developer Petya and tester Vitya, Yu thinks about value: the developer thinks about the number of features, and the tester thinks about the number of bugs. Business interests remain outside the process.
Inevitable bureaucracy
The result requirements change in every Agole cycle and in every feature. To avoid conflicts, you need to write evaluation criteria for each feature. And this leads to a huge increase in bureaucracy and time to develop regulations. Both the developer and the tester will have much less time to spend on the product.
Constant switching between tasks
Do the little experiment described in the book”Focus”. First, write “mom washed the frame” sequentially, and then try to write these words in parallel: “m... m... r...”, “ma. we. ra.”, etc. Which option will take more time and effort?
It's the same in development. When Petya got bugs on problem X (“ma.”), he was already immersed in problem Z (“ra.”). He had to remember his code, finish writing it (“mom.”), or challenge the fixes and get back to his current task (“frame”). He will spend more time on switching alone than working on the task.
Loss of communication with the user
When a developer doesn't think of a bug or a problem that a user might face, they can't foresee them. Therefore, developer Petya, who completely shifted his thoughts about how the user will interact to his tester. While Vitya tries to look at the feature through the user's eyes, Petya is losing touch with reality.
Why Google doesn't have testers
Global IT market leaders do not use testers. Using the example of MAANG: Meta* (banned in the Russian Federation), Apple, Amazon, Netflix, Google, we see that the idea of pipeline testing is viewed as an outdated concept. Headliners don't believe that hiring testers by default adds quality value to software. For example, in the book How Google Tests Software, the authors point out that Feature Developers has what it takes to deliver quality value on their own, and the so-called Software Engineers in Test are outdated. Google writes about IT product companies, but the same logic applies to the practice of service companies that work on Agile.
In Agile practices, testers do not exist as a class. The Kenevin framework recommends frequent value delivery and constant feedback from the product when working with non-linear tasks. Accordingly, we need a single center responsible both for delivering value and for receiving this feedback — a developer. Once he starts sharing responsibility with the tester, he inevitably loses focus.
To test or not to test?
Based on kt.team's experience, the best way to build a process is when a developer:
- Pair programming
- Test driven development (tests are written by developers before code)
- Continuous integration
- Refactoring and others.
This is a proven and effective way to avoid a large-scale failure.
How did we come to this? We tested various schemes for working with and without testers. More than a year ago, based on experiments, we removed the testing link from nonlinear problems. Instead, we are actively using extreme programming techniques:
- initially thinks about value, not about code;
- covers the code with tests and
- worries about the highest rate of feedback collection through logging and dashboards.
And one more practical observation: to improve product quality, you need to make the most of different types of feedback.
When testers are needed
I'm not going to say that testers in complicated environments are never needed at all. There are situations in which a team cannot do without Q.A..
The testing process is specific
For example, when creating a mobile app. There are many platforms and device options on which the app should work correctly. A developer can test its performance on several major platforms and devices, but it makes no sense to “run” it through everything by the developer alone.
It is better to involve Q.A. in this, especially if you use specific hardware that does not have software emulation, for example, some rare non-second-tier controllers. The involvement of testers is justified here: the developer remains responsible for the value of the product, and testers are responsible for catching bugs when replicating the system for rare variations of devices.
Don't push Q.A. testing for popular browsers: functional tests perform such testing tasks much more effectively.
Besides, it's not 2010 now, and there's no problem if a button pops up in some version of a rare browser. And automated pixel-by-pixel DOM analysis tools for any browser are easily accessible.
You need to set up testing processes in the company
In this case, the company does not need an ordinary “bug catcher”, but a manager who thinks about global tasks and builds testing processes based on them. It defines test zones, certifies teams for quality, develops and implements test design standards, and adjusts processes for collecting and responding to feedback from users.
A tester is required for safety or legal requirements
There are narrow areas of development that are strictly regulated by specific legislation or special safety requirements.
Even in the banking sector, this includes a small part of the work that is done only by high-tech mega-corporations. These are companies that have many systems that work as one on all continents and are subject to the same legal and other specifics.
Testers in such companies check compliance with regulations and compliance with common standards. It is not their responsibility to deliver value: their task is to regularly read the code and identify a specific vulnerability. When they discover a weak point, they incorporate part of the automated test into code that must then be supported by the team.
Such specialists are jewelers in their field. The company has an established culture and a standard for assessing the expertise of developers helps them a lot in their work.
