Today’s technical advances make it possible for new code to go to production automatically. Collaborative philosophies like Extreme Programming (XP) help teams deliver code with higher quality right out of the production line. Layers of testing from the unit level on up make future changes less risky. Continuous Integration and deployment systems work in tandem to merge the latest code, run tests, and get everything into production. Instead of a week of development and a week of testing, we have layers of tooling, very small changes, and a constant stream of releases.
Testers that have been paying attention are worried about their future. Do they have a place in modern software development? There could be a new opportunity for continuous human testing in the future, rather than a closing door.
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
Building Automation
Continuous human testing requires the very thing many testers are afraid of; infrastructure and automation.
Even in the most agile teams, software projects are riddled with hand-offs. The developers take user stories and turn them into code. Testers take new code and try to discover what might be wrong with it, and then send reports back to the developers. There is one last hand-off to an ops or implementation team once a build is ready for delivery. Each hand-off is a point in the delivery process where a tool could be used more efficiently, a process could be changed, or a deliverable change could be smaller.
Extreme Programming (XP) and high functioning agile teams set the standard for removing hand-offs between people. Imagine this scenario; there are usually two people working together for any change such as code, configuration, building tests or changing the build system. Instead of thinking about feature development in terms of feature lists and release dates, these teams work on tasks that can be shipped today. Developers making code changes start and end with new tests. Whether at the unit or the service level, the code is built with the question of ‘how can I test this?’ in mind.
When a developer commits new code to their source code repository, a CI server jumps into action. Continuous Integration merges the latest code into a build branch, and runs a test suite. A green test suite tells the development team that their code works as expected, and provides evidence that they didn’t break anything major while making changes. By the time there is a green bar on the CI dashboard, the team should be ready to deploy to production. This could occur many times every day.
The best way to start is by focusing on one piece at a time. Have your developers start writing unit tests as a sprint deliverable for every feature. Embed testers on small development teams to coach on test design and build tests at the API and UI level during the sprint. Get a Continuous Integration system up and running so that tests are running with every build. The final step is giving your team the ability to create a new environment with the latest build any day, any time. Build and deploy systems so each step builds on the previous.
A development set test automation infrastructure, and collaborative process explore new builds at any time.
Build at a Moment’s Notice
Automation is a useful tool for developers. It helps them design better code, and can sometimes alert people when a new change broke something that was previously working. Automation is not great at finding surprises, though. For example, I might build a test against an API to assert that every new user profile must have a birth date assigned. If someone clicks the save button on that user profile with no birth date, they should see a validation error. We can run those tests with every build and find new problems quickly. Most development teams want to know more about their product than what a series of assertions can find. This is where human testing comes into play.
Imagine it is the third day of a sprint. Developers have been writing and committing new code but there hasn’t been any sort of official test build. It doesn’t matter, on day one or day fourteen, a tester can run a script that creates a new test environment. This script creates a new clean Virtual Machine and installs the latest build and every needed dependency like databases. In a matter of minutes, a new test environment with the latest product is ready for review.
I could create a new test environment to perform testing that compliments what developer pairs are already doing when they build test automation. Let’s say a developer pair is creating a new reporting API ending point. That report returns data about a customer base, based on where they are located and what age group they are in. That developer pair created some service tests when they were developing the feature. The service tests check that data from a limited data set returns correctly, and examines whether the right data types are in the response. Another set of developer pairs would usually have just finished the user interface for this feature. In this scenario, all the tests are green, and the feature is released to production. However, there are some open questions.
With that new fresh environment, a dedicated tester can start exploring the software as a customer might see and use it. These testers try to find the answers to questions on how much data the report can reasonably return, the types of data that will cause problems, and what happens if the report is called with large amounts of data from several computers at the same time. These are all open-ended questions and they cannot be answered from an assertion in an automated test. These questions can usually be easily answered by a person investigating the product, and with continuous human testing that can happen at any point in the development process.
Continuous human testing is a perfect fit for challenging support calls. Earlier in my career I was working on a product that was installed directly on customer hardware. To manage a support call, we had to figure out what version of the product they were running, what patches they had if any at all, and what their environment looked like. If we were lucky, and our Ops department had time, we could recreate the environment in a couple of days and begin investigating the issue. Development teams with good delivery infrastructure can build this in minutes, if not seconds. Companies that make products designed to run in the cloud have an even easier time.
Science Fiction or Reality?
It’s a fantastic idea that I could spin up a new virtual machine and testing environment with any build and database in just a matter of minutes. The concept is far-fetched, but it would be incredibly useful. Near instant deployments, and continuous human testing aren’t science-fiction, there are a handful of companies doing this today. For the testers out there wondering what this will do to their careers; develop your skill set so that you can test anywhere in the life of a code change and you will always have a job.
This is a guest posting by Justin Rohrman. Justin has been a professional software tester in various capacities since 2005. In his current role, Justin is a consulting software tester and writer working with Excelon Development. Outside of work, he is currently serving on the Association For Software Testing Board of Directors as President helping to facilitate and develop various projects.
