This is a guest posting by Justin Rohrman. Justin has been a professional software tester in various capacities since 2005. In his current role, Justin is a consulting software tester and writer working with Excelon Development. Outside of work, he is currently serving on the Association For Software Testing Board of Directors as President helping to facilitate and develop various projects.
I first learned about quick attacks from Elisabeth Hendrickson’s Test Heuristics Cheat Sheet. The product we were working on was used by sales people to figure out the best way to price a product, such as a gallon of oil. Our clients would enter what they wanted to sell, figure out the different fees, taxes, shipping costs, storage fees and then calculate options for discounts. There were lots of places and ways to enter data that affected the final price. The Test Heuristics Cheat Sheet was helpful for testing the product, as it gave me a quick way to attack the software and find problems.
Quick attacks might be necessary as part of a test effort, but they are not sufficient in depth. Let’s cover the whole testing process, from first steps and beyond.
Get TestRail FREE for 30 days!
Getting Started with a Better Test Design
New testers sometimes like to begin testing with the ‘happy path’. The happy path is how we imagine the software should work, it’s what happens when everything runs smoothly. Open your specification or user story and search words like ‘should’, ‘must’, ‘will’ and ‘can.’ Each of these statements is an expectation; the user should be able to create a new discount, search for a book, and edit their address details. The happy path would be to create one discount, search for one book, and then edit one user email address. If the discount is created, the book found, and the email address edited, then the path is happy.
Quick attacks are tests that you can run right away with little setup. Quick attacks push the happy path one step further. Let’s use the discount as an example:
- Say you have a field that allows you to create a discount for a product you are selling. The user wants to enter a dollar amount discount.
- A tester could start by entering and saving a $10.00 discount. To explore further, the tester might start by entering a small number such as 10. If that works they might move on to a decimal value such as 10.99.
- A good tester may want to know more and might get there by attacking specific parts of the field such as data type, data length, and data formatting. What about things that aren’t numbers? Can we submit a discount that is a character string, a special character, or unicode? This helps us to discover what kind of data the UI will accept, what kind of data the API will accept, and what the database will store.
- Moving on to data length, we start to examine how small or big of a number string entered into the discount field can be. Can you create a single digit discount of $0, or should you be able to create a $0 discount at all? What about a discount of $1,000,000,00? How many digits will the field accept? How many can the database store? If you are usability minded, you might ask whether the UI should limit entries to a certain range.
- You could test data formatting by entering “10”, “10.000,99” (if your product is international), “10,000.00” and so on.
All these tests are quick to perform. You type a few characters into a field, click submit, and then make observations about what happens next. No complicated setup is required.
The happy path and quick tests give you a feel for how the discount field works. This is key to understanding how the software is working, and what is expected to create experiments and find more information.
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
Planning
I don’t mean a ‘test plan’ that takes days or weeks to create and is never read. This plan will act as a guide for the changes you might want to test for, and ideally suggestions of how to get there. You can do this in your head, in a notebook, or in XMind. Whichever method you choose, the main point of planning is to put thought into your approach, not to produce an artefact.
I like to start by categorizing aspects of the feature that I want to test, or that I have questions about. Some categories that come to mind for this discount code example are data, usage, platform and access. We talked about data: string length, data length, and data type. We also want to think how the discount will be used, what platform it might be used on, and who has access to the data. That’s just to start.
You may have noticed, we ran the happy path, quick attacks, then started to talk about a plan later. Often, a little bit of quick testing, for as little as five or ten minutes, can result in test ideas that are drastically more powerful than reading a spec and making a list. Of course, you can also plan up front, then improve the list after a quick testing session. Other times, I’ve been looking at a whiteboard with pictures of the user interface, along with the designers and product owners, and asked questions to simulate testing before planning.
Let’s get back to our example.
For usage, we want to apply a discount. Can we enter the discount code during checkout? Does it reduce the price in the way we expect? Can I apply a discount greater than my total, and make the store owe money to the customer? Can you create and apply discounts on different platforms such as Chrome or Internet Explorer? What about mobile devices? A better question might be; which platforms even matter?
The last category, access, concerns who should be able to use the discount and when. What happens when a person has a discount applied to their shopping cart, but someone deletes the discount from under them? Who should be able to create discounts, and who should be able to use them? Should discounts have start and end dates to control when they can be used?
How I started my test plan for this discount code feature.
Planning isn’t a stone tablet, it is a way to help testers figure out what is important.
Going Beyond
You have probably heard the saying “the map is not the territory” or seen the Ceci n’est pas une pipe painting by Rene Magritte. That is exactly how our plan works. The plan is what testers think they might do before actually getting started, it isn’t necessarily how the work will play out. I like to think of each node in the mind map as a mission for exploratory testing. For example, if I started testing discount creation with quick attacks, that immediately creates some questions in my mind that are completely off the plan.
After creating a few successful discounts and applying them, I would notice that the display of discounts was not in the original plan, so I begin exploring that. I might write a quick throwaway WebDriver script to create 500 discounts, just to see what happens. When the script is done, we have a very long webpage displaying a very long list of discounts and discount codes. I’m not sure whether the product manager was expecting this behaviour. I question whether this is OK. Should we be able to page discounts so people can see just a few at a time? Should people be able to search for discounts to find the one they want quickly? Should discounts disappear from the list when they expire?
Exploring a piece of software tends to expand the testing plan, and create lists of questions. Our testing started with the happy path and quick attacks. These first steps help to get a feel for the software, this continued process of exploration is then developed into a list of components to investigate while we are working on the feature. Exploration naturally leads to more questions, and the possibility of adding more nodes to our test plan. Moving beyond quick attacks is partially a game of awareness. What did we learn from the initial quick attacks, and how can we use that information to learn more about our software?
From Soup to Nuts – The Whole Testing Meal
Quick attacks and the happy path are a good place to start testing. They can give the tester a feel for how a feature works (or doesn’t work), and give almost instant feedback. Eventually, you will want to go deeper. A combination of planning and guided exploration can help testers move beyond quick attacks, to find ways a product might not make customers happy. How far you explore beyond quick attacks is up to you, but at least you can make an educated choice. The next big question is when to stop!
