Repeated software-development tasks are becoming automated through the application of Continuous Delivery and DevOps. If developers are taking more and more testing responsibilities into their hands, I wonder what will be the role of traditional (manual **only**) testing and testers moving forward?
This question has been troubling me since I looked at the testing pyramid, a concept coined by Mike Kohn in his book Succeeding with Agile. Its essential point is that you should have 70-80% unit tests, followed by 10% integration tests, 5% system-level tests, and 5% GUI-level tests.
Although the percentages of preciseness may differ, the point is that you should have many more low-level unit tests than high-level end-to-end tests running through a GUI.
Testing is all about risk mitigation. Instead of mitigating risks using just manual testing or GUI-based testing, the test pyramid looks at risk mitigation from a holistic point-of-view. Testers in themselves are not the last line of defence; developers also have to play an important role by laying out a strong, quality foundation in the form of unit, integration, and workflow tests.
Having said that, risk mitigation remains the primary responsibility of the tester role. If a tester doesn’t know how to read unit / integration tests and understand how they can be used in mitigating part of the risk, he / she will never be confident in releasing code to production based on developer-oriented automated testing, plus automated functional testing.
There are two ways to resolve this challenge.
- If a tester has programming skills, he / she can make sense of automated unit and integration tests.
- Developers also write their tests in natural language (BDD) using Gherkin syntax (given, when, then) which then both developers and testers could understand. These tests should be separate from normal development xUnit and integration tests.
- In both above mentioned cases, collaboration between developer and tester is important in identifying and defining the test scenarios. That helps in defining if a risk (test-scenario) gets mitigated at unit/integration level or at GUI test level.
It will be important to mention here yet another interesting development lately. In some cases, people are removing queues or hand-offs in development as mentioned in LeSS queuing theory. In process they are moving away from the tester role completely. That’s getting surprisingly a bit common in many startups these days and also is used in Microsoft these days.
As part of the evolution of engineering roles, both developer and tester roles will soon have coding as a prerequisite. The only difference between the two roles will be that test engineers will put testing first, and vice versa. Apart from doing manual exploratory testing, testers will spend a good deal of their time writing code in the form of automation scripts and code that drives usage scenarios.
Just to clarify, manual testing in the forms of exploratory and UI testing will still be alive; it’s just that testers will also value programming as an important skill.
In my opinion, a tester who has following characteristics will soon be irrelevant:
- Accepts a regression suite run taking longer than 15 hours
- Does not collaborate with rest of the team members
- Has to be the “quality gate” (i.e., cannot explain risk and mitigation)
- Believes that automating all tests is a worthwhile endeavour
- Does not work towards improving quality by engaging with others
Conclusion
Gone are the days when manual testing was enough, and the testing cycle used to take months. Now almost the entire regression is automated. Out of that, only 5-10% is covered through end-to-end test cases. The boundaries between developers and tester are getting blurred day by day.
Testers need to also be good programmers moving forward. Just being familiar with programming will not be sufficient. Test engineers need to be product experts, quality advisers, and analyzers of risk. They should be able to learn and evolve latest automation frameworks and work with scripting technologies to stay relevant.
Hi Shrikant, Good thoughts. Another view is that more intense unit testing does not obviate the need for integration and other forms of testing. We have not even talked about security, performance and other forms of testing. While using techniques like ATDD may reduce part of pure testing efforts, the need for specialists in testing will remain.
Prabhaker Panditi
http://www.LeadingAgility.com
@Prabhakar I never mentioned that the testers or specialization will not be required. The big takeaway is – testers also need to have programming as their basic skills moving forward.
I do agree that everyone should know coding, the more you know the more you can break the system easlity
Everything can’t be automated with the tools available.
Programming is not testing. if I am writing a code to test a scenario its not testing but automation is something which will help me. So testers should be someone who will have more end to end knowledge about the system and will try to think about those scenarios which can be automated by others who are good into coding.
Automation is a way to help but it’s not testing. It complements and help to reduce task. Clicking buttons and running a script will not help people to know about system.
@Gaurav As part of testing pyramid, 90% risk mitigation needs to be done through unit, integration, workflow and service tests. Only 10% test risks need to be mitigated through acceptance and GUI tests.If tester just understands the end-to-end tests, he will not be able to appreciate the risk mitigation done by automated unit, integration and workflow tests as he doesn’t understand programming.
These days people don’t want multiple team-members for testing and for automation. Also having a team of 10 automation tester for a team of 100 testers doesn’t scale. So people prefer testers with automation skills and that will be the way to go moving forward.
Just to make sure we are on the same page, I also believe everything cannot be automated and there is an important role of exploratory testing.
I accept that automation ease our task, but its done on a stable thing. Its good to know tools. but we should agree that, developers tests the code and testers test the requirement.
It would be great to follow the idea by which 90% of the defects are found at unit level itself, but with the big size of the software products, developers sometimes don’t understand the end users. So it would be good to involve testers at unit level,
Collaboration is always good and involving testers to define unit level tests is a great idea. However still don’t agree that only tester can test the user-story. If you follow the discussion on LinkedIn (http://bit.ly/1Eb8Q5B) you’ll find that in Microsoft currently there are no testers anymore. Many organizations are moving in that direction.
Yes i followed that, there is a comment over there which says there is nothing called as traditional.
“Microsoft currently there are no testers anymore” – if we follow this statement, there will be companies where there will be no BA also.
Think why do we need BA. Developers know what to produce so they can document it themselves. So may be some companies can avoid such roles if single person is capable of doing multiple things.
Only intention was to say, yes we need people who can code and make work easy but every tester need to do coding is not necessary.
A person who can write 100 valid cases is better than a person write just 25 and busy in thinking how to automate it.
For “developers sometimes don’t understand the end users. So it would be good to involve testers at unit level”, you can read https://mysoftwarequality.wordpress.com/2013/08/10/should-developers-test/ for some explain. There is no need for second person to do unit testing for another person’s code. In my experiences with working with devs, all of them have quality in mind and they do want to validate their code works fine before check-in. We used to have a large test team which hurt devs to validate by themselves due to the separation of duty.
For example, I am not sure how many testers use Mock based testing technique. I never use it for testing in 10 years, but it is pretty natural for a dev to write mock based testing which is much simple and efficient to test the code.
What’s the point of being a tester then? If a tester should know programming and a programmer is better payed, then why remain a tester?
Testers bring a plus to the quality of the application. Most programmers don’t want to test too. That’s why there were 2 rolls: programmers and testers.
OK let’s let the customers find the bugs, maybe even the bugs that will bring to companies a lot of money loss 🙂 Good luck!
I would not consider Microsoft to be a good example, their products have many problems which I now understand why if they have gotten rid of QA. The idea behind having QA and test resources is that they have a different perspective to the developer. You do not know how many times I have been told by developers “I did not think of that”. Also although unit tests are very important, they use a lot of mocking and test at a granular level. Not to mention many developers either do not know how to unit test or refuse to do so, in an ideal world that would not happen but it does. Testing came out of a need to find issues before the customer does improving quality and costs of software releases, if you get rid of it you will see the consequences such as Microsoft products have.
Quick list of Windows 10 bugs: http://www.trustedreviews.com/opinions/windows-10-problems-bugs-help-fix
The need of dedicated testers are reducing day by day. As the responsibilities of developers are evolving towards testing also and the evolution of various automation tools the testers responsibility will be limited to manual/UI testing only.