This is a guest posting by Simon Knight. Simon Knight works with teams of all shapes and sizes as a test lead, manager & facilitator, helping to deliver great software.
Most of you will have heard at some point or another about startups or much bigger organisations that have decided they have no need for dedicated testers. Facebook is an often cited example of a large company who has decided that by using various mechanisms including dogfooding and production monitoring, they can eliminate testers. Choosing instead to release lower quality products directly into the public domain.
Such stories beg the question that, if it’s okay for Facebook and other companies not to have dedicated testers, can my organisation save time and money by eliminating the testing function also? Instead of having a tester do it, can’t I just get the developer responsible for building the software in the first place to check his or her own code?
Life Without Testers
Even without dedicated testers and even if the developers get it wrong, you can have other safeguards in place, right? Just like Facebook do. You can do things like:
- Have the developers write lots of automated tests so that you don’t need to spend any time manually testing our software.
- Have your own people dogfood for problems with a pre-release version of the software.
- Build some more logging and traceability into the software so that you can easily pinpoint where problems are occurring if you see any.
- Monitor current activities with previous versions of the software to see whether there’s some indication of different user behaviours and whether those behaviours are desired or not.
Well, maybe. And maybe not. Let’s take a look at some of those ideas in some more detail and see whether they hold up to scrutiny.
Automated Testing
Apparently, automated testing gets all the focus now. It seems like more or less every team in the world is varying degrees of Agile and is now writing code using some kind of model driven development – typically test, behaviour or acceptance test driven (TDD, BDD and ATDD respectively).
Normally in a model-driven-development kind of a scenario, the developer is attempting to write their code using their model of choice, mocking out any interfaces or integration points so that the code they’re working on can be tested in isolation and when any resulting automated testing is run as part of the build process, the test is run isolated from any other code dependencies. There are a few problems here though.
For a kick-off, the very notion of automated testing is problematic. You can’t really automate testing. Testing is a cognitive, analytical, human activity – not something that can be performed by a machine. A machine can only do what you tell it to. If you tell it to assert against a boundary or edge-case, that’s what it will do. If you don’t, it won’t. So that means that for an automated test to run, you have to know in advance exactly what it is you are going to test for. It doesn’t take into account the things that you might find out about your code along the way.
When humans test, they generate information about your program. When a machine tests, they generate bits of data. 1’s and 0’s might give you some information about your software, but it’s a black and white kind of a situation. If you want the full spectrum of greys, you need to pull in some human intelligence.
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
Dogfooding
Having your own people test pre-release code is actually a pretty decent strategy. But it largely depends on what the nature of your software is and how much people can use it without disrupting their normal, business as usual activities.
Microsoft have had some success with this approach over the years, but their bread and butter is productivity software. Microsoft can have their people dogfood an email or word-processing application very easily since a good proportion of their people are going to be using that stuff all the time anyway! For more niche applications, this approach might prove more difficult.
Dogfooding works well for web applications. It can help to reduce the cost of up front testing where the complexities of the production environment are prohibitively difficult to simulate. The nature of web applications also makes it possible to carry out controlled experiments like A/B-split testing or controlled test flights where potentially dangerous code is floated out to a small percentage of the user base. Combined with monitoring, dogfooding can be a very powerful tool indeed.
Logging and Production Monitoring
Adding logging to the software seems like a no-brainer. Why wouldn’t you want logging? Anytime you get an unhandled exception in the code you can trace the activity through the stack and see what the user was doing when things went wrong, and on what kind of an environment or browser they were working in.
Likewise with monitoring. If you have historical data that we can compare the current situation with, then changes in user behaviour can be monitored over time and you can see whether or not new features are having the desired impact. If you find that users are changing their behaviour in ways either you or they don’t like, you can take action accordingly.
Monitoring also helps you to see whether or not there are any non-functional concerns. You can see for example how long it takes a user to complete a specific task like searching for and landing on a specific item, or making a purchase. You can identify how real users are carrying out their various activities and look for improvements in both flow and response times.
You can also look for potential security issues. Some of which may relate to business logic, as well as the more specific vulnerabilities that inevitably rise to the surface over time.
The Opportunity
None of the activities listed above require a dedicated tester to carry them out. In many organisations, they can and are being carried out by developers, operations and support staff. With that in mind, it may seem like testers shouldn’t be much in demand. But that’s actually not the case. Though it’s fair to say that the market for testers has changed considerably over the last 5 years or so.
There’s very little call for Manual Functional Testers as they used to be called, for example. The market’s all about Agile Testers and Developers in Test these days. There are obviously some specific competencies associated with those job titles. Whether you have those vogue skills will have some bearing on your career trajectory, without a doubt. But there’s some skills you can work on that never go out of fashion.
If you can prove your worth on an agile development team by employing critical thinking skills throughout the backlog design and grooming phases of the project, exploring requirements with domain knowledge and specific examples, employing the art of humble enquiry along the way, you’ll make many developer friends and help them deliver a top quality product as a result.
Then, using your mad experiment design skillz, while they’re cutting code you can help them to design effective automated tests/checks that help constrain development to the functionality that needs to be delivered and prove it works the way the business intended. Having that layer of automation will ultimately make life easier as you carry out some manual testing, since you’ll be able to use it to ensure you get a reliable build and potentially as a taxi-service to get the build exactly where it needs to be for you to test it.
The Bright Future
Do you think your services as a tester are no longer in demand? Do you think the job market has no place for testing professionals any longer? You couldn’t be more wrong. But you need to make sure you have the right skills. By working on your test design and critical thinking skills:
- You can be responsible for the dogfooding – working directly with the business users to help them test the pre-release product, capturing any issues as they arrive and reporting them back to the development team.
- You can help define the areas that, based on your exploratory testing, you think need additional logging in case they do go wrong and help the team with analysing faults if they do occur.
- You can work with the production monitoring team or system to carry out various kinds of testing in production, providing valuable results and feedback to your team to help them in the next iteration.
If you can add value to your team by helping provide information about what users really experience in the production or pre-production environments, by designing dogfood and production experiments – you’ll always be in demand.
PS: Have you found this article useful? We publish a new relevant testing & QA related article every week, including topics such as building a great testing team, improving your testing career, leveling up your testing skills & boosting your team’s testing efforts. Make sure to subscribe below via email and follow-us on Twitter! You might also enjoy the following articles:
