When I look up the definition of DevOps on Google, I see references to collaboration. Developers, product people, and operations all working together to make the best software they can. That sounds a lot like every other “modern development method”. When I dig deeper and look at the practice, I see a heavy focus on process automation. Developers write code and tests and check them into CI. Tools like puppet and chef turn builds that were previously handled by one person into something anyone can do at any time. And things like containers mean that any developer or tester on the team can have a brand new environment with the latest code in a matter of minutes.
Good automation, and the fast and frequent release cadence that make DevOps interesting, doesn’t exist without good testing. So, while there is no time for the common agile pace of writing code for a week and a half before pushing to an independent test group and starting the find, fix, build, test cycle, testing still happens. In DevOps, exploration and automation happen in tandem.
Get TestRail FREE for 30 days!
Exploration Via Automation
A developer, or maybe two (more on this in a bit), take a look at a well groomed list of small tasks. Their current task is to create a new API endpoint to pull a list of movies watched during the last six months by whoever is currently logged in. Digging deeper, the task is to retrieve a few pieces of data from a database based on time and authentication. The old way of doing this would be to write the code, maybe add a unit test or two if the developer was feeling fancy, merge code into the build, and then push the change off to whoever in the testing group isn’t currently busy.
In DevOps, this flow would probably start with a developer making the code change using Test Driven Development (TDD). They begin with a test, and then write some feature code to make the test pass. Back and forth like that until they have something that looks like a complete feature change. These code changes are made on a separate branch so that they are isolated from everything else going on in the development group. Build and deploy configuration tools such as ansible or puppet are often used to facilitate developers getting new code to the right place.
Automated unit tests run in Continuous Integration, but the changes have to be explored too. Technical teams do this quickly by using containerization technology, such as Docker. A developer can create a new complete test environment in minutes through code branching and Docker containers. Moving from code, to Continuous Integration, to a new test environment in minutes is the key here. In waterfall, testers might wait months to look at new code changes. In agile, it is a few days or a week. DevOps makes testing in new, clean environment possible in minutes.
The next layer of testing, some people might call this integration testing, makes calls against the API and looks at the response. I could make a GET call against the API logged in as a test user against a test database and look at all of the data returned. I might glance at that response and say “yup, that is a lot of JSON with information about movies”, but I could also ask some other questions. How exactly are we defining 6 months, and should data right at the boundary be included or excluded. What happens if I am not authenticated when the call is made? Are the data types we are expecting correct? Are field lengths reasonable?
Each of these ideas might be good for a one time test to see what happens, or they might lead to new or different automated tests. The process of performing those tests, making decisions about what should be turned into code that can be run again, and maybe more importantly making judgement about what you learn from those tests is pure exploration.
In an ideal world, by the time that code change is ready to move from the developer branch to somewhere else, that somewhere else should be the release branch. The emphasis DevOps has on flow, automation, and tooling position developers to get new software to customers as soon as they see a green bar on the Continuous Integration dashboard. When DevOps works well, there is no last minute pause to have people review a release candidate build. Layering of automation, exploration, and the testing required to make automation all lead to release ready software.
I’m sure you noticed that there was no mention of a tester in this article on testing. How do DevOps groups mitigate that risk?
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
Coaching and Pairing
You could have people working independently, pushing code every few minutes like it’s the wild west, or you could add some collaborative themes to DevOps.
ExtremeProgramming and pairing aren’t exactly part of DevOps, but they mesh well. Rather than one person at the keyboard, there are two. While one is writing code, the other might be asking questions, helping refine ideas, or stubbing test code. They might switch from driver to observer when one person gets stuck, or at some interval that they agree on before starting. At the end of a pairing session, they have a finished, or something closed to finished, piece of code just like when working independently. But, there is a special difference. Pairing is a built-in code review process. This is one more layering of test strategy that leads to better code quality, and probably better software in general.
The Evolving Role of Test
When I was starting out in software, automation was an oddity. The teams I was working with then were getting used to Continuous Integration and build pipelines were only an idea. Test automation, especially in the User Interface, was something managers thought was magic, and were often proven wrong about. The tooling available was difficult to use, not fully-featured, and we didn’t have slick architectural patterns like the page object.
When I talk with testers or look at the available jobs today, there is an expectation that testers have some technical understanding and an ability to become fluent in tools that used to be the domain of developers and release managers. This technical skill set ranges from being able to open developer tools in your browser of choice and actually understand how to apply the various data capture tools there, all the way through writing code.
Software testing appears to be riding a pendulum. At one point, there were only programmers. They wrote code, did the testing, and delivered (hopefully) working software. Some time later, the world thought it was a good idea to separate concerns. Development got their cube farm, testers got their corner of the building, and business people were moved away from the riff raff so they could keep the company running. Today, I see software swinging back to a more united team. In the most radical cases, people with testing experience and skills are being made into developers. At a minimum, it seems that all of the roles are at least sitting and working together now.
I don’t think there is any risk of testing disappearing in the future. Or even the testing role. But, people that want to stay relevant and useful should probably lose their fear of technology and tools.
To discover testing in DevOps, we have to think small. Shrink each task in the development backlog into something manageable and get it all ordered by priority. Create small tests that return information on aspects of the product people care about at each level when it makes sense. And, deliver small pieces of code to production as often as possible. Working on smaller pieces of code, over shorter periods of time makes the testing problem a little more solvable.
This is a guest posting by Justin Rohrman. Justin has been a professional software tester in various capacities since 2005. In his current role, Justin is a consulting software tester and writer working with Excelon Development. Outside of work, he is currently serving on the Association For Software Testing Board of Directors as President helping to facilitate and develop various projects.
