This is a guest post by Erik Dietrich, founder of DaedTech LLC.
How do you feel about your group’s software testing strategy? Do you think that it is bulletproof? Planning a nice relaxing holiday to start the day after your next roll to production?
If you’re anything like the majority of organizations out there, I’m betting that the answer to these last couple of question is somewhere between “no” and a cynical, barked laugh. And that’s okay. I mean, it’s not ideal, but it’s perfectly understandable. Even if you had the most sophisticated testing practice in the world, you’d probably not feel entirely comfortable.
This is largely because testing is like studying for a midterm in college. No matter how much you’ve already done, you could always do more. And you can never be totally, exhaustively, and completely prepared for what may come.
Still, by combining test preparation with a good testing strategy, a strategic approach can leave you much better situated. Let’s take a look at the holes you might have in your current testing strategy, so that you can improve your preparation.
Get TestRail FREE for 30 days!
1. Load Testing
First on the list is load testing.
This falls under the general heading of performance testing.
The idea is pretty straightforward. You put a burden (i.e. a “load”) on your software and measure how it responds.
Why do this? Well, a lot of testing in lower environments is about functionality. Does the software meet its acceptance criteria and deliver the expected behaviors? To find that out, you might deploy your web app and then execute tests against it… as the only user. But that hardly approximates the real world, where perhaps thousands of users will use the app at once.
Enter load testing. You want to do your best to recreate production conditions and observe how the system behaves. This lets you decide what acceptable performance is, and it also lets you baseline performance so that you can know later if the system is becoming sluggish.
2. Exploratory Testing
Next up, a different kind of testing. Exploratory testing is a more open-ended style of testing that relies on the expertise of the tester. The tester explores the software, relying on a knowledge of how it should work and interact with the users to look for issues that might not arise in any other form of testing.
For example, imagine a piece of software where a lot of the users were transplants from an old green screen system that relied heavily on keyboard shortcuts. Knowing this, a good exploratory tester might think to try keying in a shortcut for “save” from the old application on this application’s customer maintenance screen. It’s really unlikely that this would make it into a standard test script or automated suite, but it’s a valuable thing to know and to explore.
Way too often testing groups focus so much on scripted tests that they don’t give testers the freedom and time for exploratory testing.
3. Stress Testing
Stress testing is the cousin of load testing. But where load testing is designed to examine how your system behaves in normal situations, stress testing examines it when pushed to its limits.
Why would you want to do this? After all, isn’t the idea of good planning that operations should prevent the software from being pushed to its limits?
Well, sure, ideally. But in reality, nobody can truly control this or predict when an outage or a traffic spike no one could foresee will destroy the best-laid plans. So it’s important to understand what the system’s limits actually are, what it looks like when you start to get there, and whether it handles overages with reasonable grace.
Stress testing provides all of this knowledge to you ahead of time, before you have to learn the hard way.
4. Regression Testing
Regression testing is a much different concept than stress testing. It falls into the more “traditional” realm of verifying the application’s behavior. But, unlike standard functional testing of the stuff you’re building, regression testing is when you test the stuff that isn’t supposed to have changed.
For instance, let’s say that you add a feature where new users can import their data automatically from one of your competitor’s services. Obviously you’re going to test this functionality before rolling it out. But do you also test the normal, old fashioned way for users to sign up?
If you don’t, you should. And when you do, you’ll be performing regression tests.
Software is complex, and, even under the best circumstances, it’s heavily interrelated. This means that touching it always creates at least some danger of breaking something that previously works. Make sure you guard yourself against this.
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
5. Threat Modeling
Switching gears yet again, let’s talk security. In the broadest sense, threat modeling is an exercise that lets you reason about and test your system’s defenses.
Threat modeling involves first identifying people that interact with your systems, the systems’ boundaries, the access levels, and the flow of information. You then take this and start to identify potential sources of risk. What happens when an administrator leaves the company? Could someone with a thumb drive save sensitive data? You ask yourself all sorts of questions like this.
Test execution comes in once you’ve identified the threats and decided on how to mitigate them. Referencing the questions I just mentioned, you might devise tests for deprovisioning users and disabling a thumb drive, respectively. This sort of risk mitigation becomes part of your testing.
6. Usability Testing
Security isn’t the only angle from which you should examine your system. What about your users and their experience with the system? This is known as usability testing.
Usability testing involves putting actual users in front of your software and observing how easy and intuitive it is for them to complete tasks. Do not confuse this with functional testing of the software. It is completely possible for the software to do what it is technically supposed to, but still be a nightmare to use.
As a memorable example, imagine a white save button with white text and white background on a white page. When clicked, the button properly executes a save. But good luck finding it. Usability testing will show users consistently (and understandably) failing to find the button.
Your test strategy should include usability testing to ensure that people actually want to use it.
7. Unit Testing
I’ll close with a developer-centric concern, which is unit testing. Unit testing is the practice of writing small, automated tests that exercise your code at a very fine level of granularity. Think of passing two variables to an Add() function and verifying that the function returns the correct results.
Developers write, maintain and execute unit tests in their development environments. Assuming you have a CI setup, they’ll also execute on the build machine. And in both cases, they serve to help developers catch regression defects early and to ensure that they understand the system’s behavior.
Unit tests are yet another example of how testing and quality control are really everyone’s responsibility. A development group that doesn’t unit test puts undue strain on everyone else to pick up the slack. So make sure you’re catching as many issues as you can as you build the software by having a solid unit testing practice.
This is a guest post by Erik Dietrich, founder of DaedTech LLC, programmer, architect, IT management consultant, author, and technologist.
Test Automation – Anywhere, Anytime
