This is a guest posting by Simon Knight. Simon Knight works with teams of all shapes and sizes as a test lead, manager & facilitator, helping to deliver great software.
If you’ve been working as a tester for any length of time, you can’t by now have failed to notice the shift towards Continuous Delivery in many projects and organisations. Businesses, project and operations teams all want to try and take advantage of at least some of the perceived benefits of being able to quickly and consistently release new builds to production, ostensibly at the push of a button.
And why wouldn’t they? After all, realisation of the Continuous Delivery dream means faster feedback, vastly improved time to market, increased quality and a better customer experience. Though not necessarily in that order.
In the meantime, testers will likely have found that the Continuous Delivery model has a big impact on how they need to approach testing. Development teams have little patience for testing approaches that don’t keep up with the pace of delivery, and there’s no hunger from the business for lengthy testing phases either.
Join 34,000 subscribers and receive carefully researched and popular testing & QA articles. Top resources on becoming a better tester, learning new tools and building a team.
What’s All The Fuss About Continuous Delivery?
Developers love the idea of Continuous Delivery mainly because they get much more rapid feedback on their code. When a feature or story is built and checked-in, they will find out very quickly whether they’ve introduced some kind of issue because of the various layers of testing that will likely have been implemented as part of the build pipeline. Even without automated tests, if a feature is delivered to production quickly, they can get feedback from the live users, and find out what does and doesn’t work directly from the customer base.
With faster feedback for developers also comes improved productivity. There’s no more waiting around for integration and release testing. That all gets done in the build pipeline. In theory, code can be written, tested locally, reviewed, merged, integration and acceptance tested before it even gets into a testers hands.
The holy-grail is that releases are ultimately made boring by completely automating the build and deploy process from start to finish. Releasing an executable to an environment, be it local, test or production, should be achievable at the click of a button and also be completely hands-off.
Rapid, reliable and high quality releases mean happier customers, which most likely equates to increased revenue for the business. Development and operations teams are justifiably excited by the idea of moving faster and, with support from the business, tend to be the driving forces behind a transition to Continuous Delivery. But with most of the focus on development and operations, it’s easy for testers to feel left out in the cold.
Challenges Of Working In A Continuous Delivery Environment
It’s not unusual for testers to struggle with the transition to Continuous Delivery. In this brave new world, there’s likely to be a strong focus on:
- Automating as much of the testing as possible
- Delivering features in small chunks, many of them without a user interface to test against
- Features being developed in a matter of hours or days, not weeks or months, with little time for planning and preparation
- Time to market over perfect software.
These changes in perspective require testers not only to optimise their existing skillset, but to learn some new tricks too. When delivering software continuously, testing needs to be continuous also. It needs to be infused into every stage of the development process, from identifying the story as a business requirement, to figuring out whether or not customers like and are using it once the feature is delivered to the production environment. How do you make your testing continuous? Here are some ideas for you.
Critical Thinking
There is certainly a requirement for strong technical skills when working in a Continuous Delivery environment, but that doesn’t mean you won’t need some soft skills too. The testers ability to question things and apply critical thinking to requirements, products, practices and processes are all valuable skills.
In a Continuous Delivery world it’s all too easy for a tester to fall into the trap of focusing exclusively on the technical considerations (automating all the things!) and forget that what they’re actually on the team to do is to think. So when you’re working on a Continuous Delivery project, do just that. Think! Think about:
- Whether the team is building the right thing in the first place
- Whether the business requirements can be captured as executable specifications
- Whether you can add value by working with the team to identify and express acceptance criteria, scenarios and examples
- Whether processes are working or can be improved
- Where the waste is and how it can be removed
- How to approach non-functional testing, performance, security, accessibility etc.
- What devices, browsers, operating systems and environments the product needs to support and how you can test those
- What data and configuration are required across all of the environments in your build pipeline
If you need them, use some tools to help build some creative and problem solving muscles in your brain. De Bono or Weinberg’s books are great places to start!
Effective Communication
And while we’re talking about soft skills, we should remember that much of the testers job (and indeed everybody’s job) relies upon effective communication. As a tester you should be confident discussing bugs and issues directly with either individuals or the team, since logging bugs in a tracking system is basically a form of waste. Bugs are either things that should get fixed immediately or stories that should be added to the backlog so that they can unlock additional value for the customer in the future.
But your team doesn’t just want to know about bugs. You should be able to tell the story of your testing to them during planning, grooming and stand-up meetings. Tell your team:
- What you want to do
- How you want to do it
- What problems you think you might face
If you can express your plan for the testing as a narrative, you’ll find your team engages with both you and it better (people love stories!). Side effects of a better engaged team include more support both technically in the form of more testable products, and swarming behaviour on the testing tasks when you simply need more bodies on the ground.
A Layered Test Strategy
In addition to talking about bugs and the test plan, you need to discuss how you’re going to build quality into the entire development process with your team. In the Continuous Delivery world you should be thinking about layering your testing so that it’s focused on different levels of architecture and functionality. Consider discussing a model like the one below with your team:
- Exploratory testing
- Automated user acceptance testing
- Automated integration testing
- Unit testing
Now think about how you’re going to support your team in accomplishing this model. Do you need to work on your programming skills in order to learn how to write the necessary code to implement all of the automated tests? Or do you just need to focus on what the test infrastructure will look like and the various abstractions it will need to contain? Perhaps rather than diving into the code you can support your team by helping to identify tests that will provide the kind of coverage you’re looking for.
And while we’re on the subject – coverage is an important topic! Do you need code and branch coverage? Feature coverage? End-to-end test coverage? Integration coverage? Unit and component coverage? Regression coverage? What about security and performance coverage? And cross browser and device coverage? As the testing expert you should be thinking about the tests your team will need, in order to deliver the kind of quality your customer expects.
Keeping the big picture in mind of the complete product solution will prove valuable over time as your developers get focused on the day-to-day business of rapidly delivering features and stories.
Exploratory Testing
Moving away from the technology facing automated tests, much further up the testing pyramid we have the business facing exploratory tests. Now probably I could write an entire book about what those might entail, but for now let’s keep it simple. When working in a Continuous Delivery environment we want to try and keep our tests as lean as possible. Exploratory Testing is ideal for this, but still we need some way of managing the process so that, if necessary, other members of the team can pick the tests up.
A way that we can achieve this is by using a Session Based approach (PDF by Elisabeth Hendrickson) to managing the exploratory tests. Defining the scope of the testing we want to do up-front as a kind of mission statement, or *charter* in Session Based Testing terminology. That charter might consist of a set of ideas, or heuristics, or it might take the form of a tour with an itinerary or set of landmarks to visit with some associated resources that may be helpful for the journey. Or it might take the form of a persona based test that focuses on the needs and objectives of a specific user.
Ultimately how you define your exploratory tests is up to you so long as you bear in mind the following key principles that will help you in a Continuous Delivery context:
- Keep the tests lean
- Timebox them
- Provide enough information in the mission statement or charter for someone else to pick them up if necessary
- Document the outcome of your testing
Test Coaching & Testing Tools
If you are technically inclined, there’s still no need to dive right into the code. You can pair with your developers and help them to investigate the code they’re writing as it’s being created. Coach them towards taking a Test Driven approach to developing their code. Help them to identify edge cases and scenarios. Steer them towards implementing a Behaviour or Acceptance Test Driven Development (BDD/ATDD) approach by working closely or pairing directly with them.
When you’re not coaching you can be researching and implementing tools to make exploratory testing faster, easier and more effective. Or you can research and experiment with agile testing tools such as TestRail to help you manage testing across the entire team or organisation. Tools to help you interrogate and exercise your system. Automated scanners to identify flaws and vulnerabilities in your solution. Tools to help you investigate usability, accessibility, portability, deployability and any other ility’s you can think of! Become an expert in supporting your team with testing solutions the developers didn’t think of yet.
If you think you’ve got the chops for it, then by all means start writing some code. You can help the team with automating the tests at any level once you’ve got some coding skills. Not only that but you can add some value to the review process as well. Assuming there is one. And if there isn’t – well, what’s to stop you from reading the code anyway?
Focus On The Benefits
As I mentioned earlier, it’s been my experience that testers struggle when trying to adopt the Continuous Delivery worldview. Particularly when coming from a more waterfall style background. Rather than focus on the specific skills that the Continuous Delivery model demands, I find it helpful to focus on the benefits Continuous Delivery is intended to deliver:
- Accelerated time to market
- Build the right product
- Improved productivity and efficiency
- Reliable builds
- Improved quality
- Improved customer satisfaction
I use the benefits as a set of heuristics to help guide my thinking and decision making. As a team, if we’re falling over in one of those areas, what can I do to help? Normally, some combination of the techniques I’ve discussed above goes a long way towards identifying a solution.
PS: Have you found this article useful? We publish a new relevant testing & QA related article every few weeks, including on topics such as building a great testing team, improving your testing career, leveling up your testing skills & boosting your team’s testing efforts. Make sure to subscribe below via email and follow-us on Twitter! You might also enjoy the following articles:
