This is a guest post by Nishi Grover Garg.
Continuous testing is the process of testing continuously throughout the software development lifecycle, at all stages of the process. The different stages can include different types of testing like unit testing, integration testing, system testing, and acceptance testing, spread across phases of the development process like requirement gathering, design, development, and delivery.
Continuous testing is a key piece of the Continuous Delivery puzzle. As companies are scrambling to release faster and run businesses with limited resources and time, getting their continuous testing right is more important now than ever! Let’s examine some common challenges in continuous testing and ways to overcome them-
Trying to Automate Every Test
In previous years, the challenge might have been not automating enough. Getting testers to board the automation train, and having them contribute and constantly run automated tests was the first step to this. But the challenge now is that we’re trying to automate everything!
This results in huge test suites that are unmanageable and take more time in maintenance than testing itself. Moreover, they cannot be run continuously and thus defeat the whole purpose of automation.
We know the benefits of automating your tests. But we also must realize that not every test is worth automating.
Teams need to have a checklist of criteria a test needs to fulfill to make it to the test automation suite. We must keep our tests nimble, our automation suites flexible, and layer them such that they can be executed on-demand, in full, or in parts.
Only Automating Tests
For continuous testing to succeed, the focus needs to shift to automating more than just functional tests. Testers need to continuously share test results, find problems, advise of risk areas, and create meaningful test reports. We must look at ways to automate more of these tasks and leverage the useful tools available today in the market.
Automation can help you achieve continuous testing by:
- Helping you create test suites that can be triggered at every build
- Customizing the results, creating useful dashboards
- Sending reports automatically to teammates via email or Slack
- Re-running tests on different platforms
- and so much more!
When automated, every one of these tasks is a value-add to the continuous testing goal of your team.
Choosing Popular Testing Tools over the Right Tools
Since we need to pick tools to make our own DevOps pipeline, we are faced with the choice of a multitude of tools available in the market. And with so many tools promising us the best features and quality, we most often find ourselves taking a safe bet with the most popular kids on the block.
While that might seem like a safe bet, in the beginning, it could also turn out to be an expensive mistake later!
We must not go just by the popularity of the tool or just pick open source tools. We must survey the market for the actual fit for our team as well as our project. The tools we select have to fit the context, meet our unique needs, and be something our team can work with and appreciate. So do your own POCs, give your teams a run at it, and have enough demos and pilots before you finalize any tool you add to your project.
Adding Processes During Development
Processes are good to maintain the sanctity of the project and deliver consistent quality. Having checklists for each task as well as clear definitions of done and exit criteria help the team know their goals and direction.
Whatever you do within the team must be defined, explained, and agreed upon at the beginning of the project. We can make improvements. But if we keep adding more and more processes during the ongoing iterations, they can cause more harm than good.
Introducing new ‘rules’ or mandates during the course of development feels like overhead and causes confusion and delays. Apart from very urgent or critical changes, it is better to reserve any new process changes for a retrospective meeting with the entire team before bringing them into action.
Only Focusing on Testing
It is 2022, and we still hear testers complain that quality is still thought to be ‘their job’. Development teams focus on writing code and tend to leave everything else on the testing team since it is called ‘continuous testing’. Although the term is continuous testing, it is not just about testing.
Continuous testing focuses on continuous quality and improvement and, thus, it looks deeply at areas of code quality, application correctness, and an overall good user experience. The entire team has to examine their own pieces and find areas for improvement.
Through code reviews, tackling unit testing debt, improving the documentation, or enhancing the UX – everything is a part of continuously delivering high value to users.