There’s something dubiously iconic about the software defect. Companies spend months, or even years, building a piece of software. Then they put it into production, and at least something goes horribly wrong at some point. It’s probably some kind of immutable law of the universe.
If you’re lucky, whatever goes wrong will be something fleeting or minor. After all, it’s not like every software release is a flop. But if you’re unlucky, you might wind up the poster child for failed software releases, like the initial rollout of Healthcare.gov.
Problems with and defects in software that you release is the scourge of the software industry. It’s also the raison d’etre of testing and quality assurance groups. But it’s not the only way that bugs rear their ugly heads.
They also come in the form of regression defects.
Get TestRail FREE for 30 days!
What are Regression Defects?
Regression defects can be as devastating or embarrassing as a buggy launch. These defects occur when a feature of your software that has always worked without issue suddenly breaks because of unrelated changes. In a sense, the software regresses. Hence the name regression defect.
When this happens, it can be an annoyance, but it can also lead to incredulous or angry reactions from users.
- “All you did was change the font on the login button — how could that possibly break the ability to log in? What is wrong with you people!?”
- “I thought you said you fixed this! I’m seeing the bug again.”
To understand how your software looks to users, imagine taking your car to a mechanic where every fix breaks something else. You bring it in for an oil change and the air conditioning stops working. When they fix the air conditioning, your brakes stop working. How long before you stop bringing the car there?
Regression defects are a serious problem in software since software systems are complex and heavily interrelated. And so the idea of regression testing becomes critical.
What is Regression Testing?
If regression defects are features that used to work suddenly breaking, regression testing is the activity of detecting, and thus preventing, regression defects. In practice, this means putting the software through paces to make sure it continues to work as expected.
Everyone does this to at least some degree, even shops with the most rudimentary testing processes imaginable. Think of it this way. Let’s say that you have a web app and you add a text box to the “edit your profile” screen. When testing that functionality, you have to login and perform some basic navigation even to get to where you can test it. And, when you do that, you’re performing a de facto regression test.
Of course, pretty much any professional software organization will be a little more deliberate in their approach. They’ll have more formalism around testing, both regression testing and testing of new features.
Let’s take a look at some of the different approaches that organizations employ to help them with regression testing. And, it’s worth noting that a lot of companies actually perform regression testing, even if they’re not familiar with that exact term.
Manual Regression Testing
First up, you have manual regression testing. Technically, my example of navigating to the new feature you want to test would qualify. But hopefully you’re a little more organized than this.
Most commonly, manual regression testing means putting the software through some standard paces that you do regularly, perhaps as a sanity check. This might mean that you do some end to end work, like creating a new user, making some modifications and then deleting that user.
Whatever the specifics, manual regression testing involves kicking the tires of the software to make sure everything still looks good.
Manual Regression Testing with a Test Plan
As you might imagine, having a tribal knowledge regression test process doesn’t scale or perform especially well. Pretty quickly, you’ll need to get more organized and formal.
This is where a test plan comes in for regression testing. Here, you put together sequences of steps that you perform with each release.
The purpose here is still the same, but having a better test management approach will ensure both better coverage and fewer false positives.
Automated/Scripted Regression Tests
Again, there’s a limit to how well manual regression testing will scale, even with a well thought out test plan. The binder (or, hopefully by 2018, test case management tool) full of test cases will grow thicker and thicker to the point where, no matter how many testers you employ, you struggle to get through it all.
That’s where automation enters the picture. Your organization’s testers are smart folks, and it’s good to devote them to exploratory testing, if you can, rather than following a script by rote. When you have a script that a human can follow by rote, it’s pretty likely that a computer can do the same, given the right tool.
So to scale your regression testing efforts, it’s important that you do exactly that. Make use of tools that can launch your software, supply it with inputs and verify its outputs. You can then run these automatically with each release (or even each build) to make sure things that used to work still work.
Automated Unit Tests
Regression testing isn’t purely a pursuit for the testers in the organization. Software developers should also be pitching in to the effort, albeit in more granular fashion.
What I’m talking about here is the unit test. Unit tests are relatively small pieces of test code that software developers write to test relatively small pieces of production code in isolation. If you imagine the codebase as a complex machine, unit tests are the equivalent of deconstructing the machine into its smallest parts and verifying that those parts behave as expected.
When the developers do this, they’re actually creating regression tests at a very granular level. As they work on the software, they constantly run these tests to make sure they all pass. Should any one of them “go red” the developer knows within seconds that a regression exists and can course correct.
As you can imagine, this works powerfully in tandem with automated end to end tests.
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
Your Overall Regression Testing Strategy
Having a multi-pronged strategy for regression testing is critically important. The developers work on a granular one for source code, while developers and testers build an automated suite that tests the application end to end. This frees the testers to perform exploratory testing as well, going through the application they know so well, looking for anything suspicious.
As I mentioned earlier, software is insanely complex, and the occasional regression defect is, frankly, inevitable. But with a sophisticated, multi-part regression test strategy, you can keep regression defects to a minimum. And, in so doing, hopefully you avoid ever being used as a famous example of what can go wrong with a software release.
Written by Erik Dietrich, founder of DaedTech LLC, programmer, architect, IT management consultant, author, and technologist.
Test Automation – Anywhere, Anytime
