Surviving as a Tester in DevOps

This is a guest posting by Justin Rohrman.

DevOps threatens to cut testers out of the development process in two key ways: making the development flow and feedback loop so fast that there is no place for them to join in, and making the consequences of failure so cheap that the service most testers provide isn’t worth the money.

Developers strip down the monolith into small services so that parts of the product can be changed and tested in isolation of everything else, and technologies like containerization help us create new environments faster than ever. Once a product is in production, monitoring tools and feature flags help us to discover problems and turn off the source in minutes.

How can testers compete with that?

Get Familiar with Environment Management

The first threat testers feel is from the feedback loop. A normal development workflow in my experience includes handoffs and waiting — not just between developers and testers, but also with a third party that does server management, including product installation, database management, installation and upgrading of product dependencies, and configuration of the continuous integration tool.

Testers in this scenario are crucial because they can collect information from several different groups that wouldn’t otherwise be known. Testers are some of the few people who have a holistic understanding of the product being developed and all of the different people, processes, and tools that make it work.

DevOps removes most handoffs by merging roles. At one company I worked with, there were two ops people who were subject matter experts on server management and configuration, but developers were expected to do the majority of server and deployment management.

Rather than losing a role in the development process, the testers at this company became familiar with the CI tool and Docker usage. Each tester on the team had their own environment to which they could deploy new code or configure for testing as needed. We all had SSH access to get into the test server for normal product configuration, as well as for running bits of product code for part of a test.

Environment management isn’t testing, but it can inform testing. When someone on the test team found a software problem, we could log into our server and look at the logs. When a deploy went weird, we could SSH into the Docker container and database to try to figure out where the problem was happening.

Technical competence makes testers more useful in most environments.

Join in the Development Process

Another key focus of DevOps is improving the feedback loop on code changes. If something that a developer did causes a new problem, we want to know in minutes or seconds, not hours. We get that information through architecture, testing, and process.

Rather than building entire features in one go, we might update an API endpoint, then update the model and controller, and then build the user interface. Each of these changes can be tested in layers and pushed to production independently. This reduces the risk of introducing new problems because we are focusing on building and testing one small thing and learning about it quickly, rather than investing in weeks of code at a time.

So, thanks to DevOps, we have much faster development and much higher code quality. How do testers fit?

I made myself useful by joining the party. Most of the software development at my last company was done in pairs, so when it was possible, I would join a pair or a developer flying solo. Testers who did this were able to add an additional layer to the testing effort.

When the production code was being written, we could talk about the best way to test it, or even just ask how we could know that line of code works. Throughout the code-writing process, we could review test coverage. As soon as we were able to build a testable container, we could explore the change in the product as a pair and bounce test ideas off each other.

Testers can easily avoid being removed from the process by joining in.

Start Off on the DevOps Journey

There are lots of bits and pieces involved in a DevOps implementation — staffing, process, architecture, tooling and, the hardest of them all, skill — and most people aren’t quite there.

One company I worked with had a very modern development practice. Developers paired on code changes from start to production. We used containers, so there was no question whether the test environments were identical to production in terms of the software and dependencies that are installed. And all test runs and container orchestration were performed through one tool that everyone on the technical team had access to. Even with all of that, we had a testing group; it was small, but it was there.

Squeezing all of the benefits out of DevOps can take years, not to mention significant changes in architecture, process and staffing — and a significant monetary investment. All these changes can make it tricky to see how testers fit into this new norm. Testing skills will still be valuable and necessary, but it’s a good idea for testers to develop the skills needed to thrive in a DevOps environment.

This is a guest posting by Justin Rohrman. Justin has been a professional software tester in various capacities since 2005. In his current role, Justin is a consulting software tester and writer working with Excelon Development. Outside of work, he is currently serving on the Association For Software Testing Board of Directors as President helping to facilitate and develop various projects.