This is a guest posting by Justin Rohrman. Justin has been a professional software tester in various capacities since 2005. In his current role, Justin is a consulting software tester and writer working with Excelon Development. Outside of work, he is currently serving on the Association For Software Testing Board of Directors as President helping to facilitate and develop various projects.
The metaphor most people think about for software projects, is one of driving results with pressure. If things are going poorly, get everyone in a room, yell at them a bit, and tell them to fix this schedule problem.
I’m pretty sure that’s not the most helpful way of thinking about it.
Personally, I prefer to think in terms of trade-offs. Every executive has three levers; time, quality, and scope. Pushing one lever tends to move the others, so at best you get to pick two. For example, teams that want to deliver faster, tend to reign back on either how much software they deliver, or how good that software will be. Teams that want to increase scope, or deliver more software, will have to sacrifice time, quality, or both. These are almost like laws of physics, simply how the software development universe works. Over time, investing in quality can lead to improvements in speed. That is an investment without an immediate payoff, so some companies try to hack these rules. They hire teams of software testers that live in another country, ideally for a much cheaper rate, to handle work that can’t be staffed in house.
Outsourcing isn’t magic, and all hacks have side effects. Here are a few lessons from my experience with outsourcing software testers.
The Problem Domain
Around 2006 I started working on a pricing science product. This tool was used by sales people to help them price things like airline seats, or deliveries of chemicals to manufacturing plant. Sales people would select the product they wanted to sell, and then the tool would calculate the extra charges such as shipping fees and tax. The tool would then figure out what discounts were appropriate. Ideally, the price the customer gets is at the intersection of profit for the seller, and good deal for the customer.
We had delivery problems.
Developers started new feature work before they understood what the customer wanted. As you can imagine, this created more than a few questions and bugs. Product Managers would add new features to the delivery at the last minute before each release (scope), further extending the amount of time we needed.
Our managers thought the test group was the bottleneck, and tried to solve the problem by hiring a team of testers in Bolivia. This outsource group was supposed to run test cases that were created in house, and build automation. That combination was going to get us back on schedule. That is what my management thought, at least.
Our product was complicated, and the business domain wasn’t any easier. Most tests that we performed required data, setup, and time. We tried to document this the best way we could in test cases. Inevitably, something that was second nature to the testers would be forgotten. Each day, we would get bug reports from the outsource team reporting tests that had failed because they didn’t understand the product, or were missing crucial information.
“Brooks Law”, named after the professor, Fred Brooks, and documented in his book The Mythical Man Month, states that adding people to a late project will make it later. The new people need to get up to speed, and that takes away from working on software. In the case of outsourcing, you also need to add time zones and culture differences to that equation.
Our business domain was the problem. It was complicated for people that worked in the office. Most new employees took around six months of living the product to get a real grasp of what we were doing. Confidence and the ability to train someone else might take a year or longer. Our managers were expecting a remote team to get the same proficiency of the software faster, and using documentation rather than conversation. A strategy that was supposed to save us time, money, and get us delivering software on schedule, ended up being expensive and slowing us down more.
An outsourcing strategy might work well if you have a known problem domain that can be explained quickly, and a fast feedback loop between the development team and remote outsourced team. It will be a struggle for teams with a complicated problem domain and slower feedback loops.
Hands On
We tried to send someone to Bolivia once or twice a year to spend some time with the outsource team. I had my turn in 2009 and spent a week in Cochabamba, Bolivia. I spent time pair testing on active projects with some of their testers, so they could get a feel for parts of the product that were hard to describe through text. We did some pair programming sessions on the User Interface automation framework they were building. There were some sessions for skill development, and lots of meetings in general.
The goal of these trips were like the goals companies have when they send employees to conferences; part skill development, part engaging and energizing teams, and part planting seeds of culture.
During the trip, and for a couple of months after, everything was great. Our remote team was excited to be on the project, and happy to get some much-needed attention. I enjoyed getting to work in person with people that I had been talking to over conference calls and instant messenger tools. The quality of work from our outsourced team improved significantly after each trip. Over the next 6 to 10 months, until the next time someone from the company I was working for would send someone out, the excitement and improvements would slowly disappear. The product would change, and teams and employees would change in ways that are hard to share when you are almost four thousand miles away.
The best partnerships, in my experience, happen when people from your company and the outsourcing team can be together several times a year. People tend to be much more understanding of problems when they are explained in person. Also, the tacit parts of testing, the components that cannot be written down, can be shared when two people are sitting together and working side by side.
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
Culture
There are different cultures across the US. The East coast is known for being fast paced, the West coast is known for a more a more technical, if laid back style. The South is casual and friendly. Those are broad generalizations of course; we should be careful not to allow them to become prejudices. Still, they are all distinct cultures. Companies distributed across the country need to navigate this phenomenon. This is magnified across countries and language boundaries.
It took a couple of weeks for us to realize that people in Bolivia tend to get to work a little later in the morning, and stay later in the evening than what you see in professional North America. They also tend to go home for a long lunch in the afternoon.
I got to the office around 7:00 to 7:30 am each morning during my visit. That is an early morning for most tech companies, but I like the early morning quiet to focus and get work done. Most people at this company were in to the office closer to 10am. Lunch was around 1 or 2 in the afternoon and lasted an hour or more. People would leave the office around 7 in the evening and either go out with their friends for a long dinner or go home to their families. Schedules were more relaxed and loose than I was used to, and events felt like they took longer in general. None of this is bad, nor it is good; it was just a learning experience for us on how to work with people in South America.
Schedules are the easy part to figure out. Each culture also has different communication styles, which affects communication up and down the organizational structure, in a way that can realistically only be understood by spending time together, or perhaps, with the help of a business coach.
Outsourcing testers can be a good way to extend your team enough to get back on schedule, if testing is the actual bottleneck (as opposed, to, say fixing). In addition to this it is important that management understand the real dynamics of the project, are realistic about trade-offs, and handle ramp-up well. Teams working a business domain that is easy to understand, and with teams that are not too far in distance, time, or culture will probably have the best luck. The more you stretch that combination of difficult software products with different people, the harder it will be to take advantage of the cost savings.
