This is a guest posting by Carol Brands. Carol is a Software Tester at DNV GL Software. Originally from New Orleans, she is now based in Oregon and has lived there for about 13 years. Carol is also a volunteer at the Association for Software Testing.
It seems like I’ve become known as ‘the one who knows how to create bug reports’ around the Dev room. Today is the third time I’ve been asked how to create a bug report by a developer. However, this time the developer got a little more than they bargained for.
Typically, when a developer asks, “How do I create a bug?” they mean “What buttons do I click?”. I show them the easiest way from whatever screen they’re currently focused on. I figure if they want more help, they’ll ask for it. I can’t really explain what made today different. Maybe it was the wording of the question, like “Can you show me how to write-up this bug?”. Maybe it was that I already have a better line of communication open with this particular developer.
But today, in addition to answering “What buttons should I click?”, I answered “How can I write-up this bug effectively?”
Documenting the Bug
The developer explained the problem, and started typing the technical details, describing what he thought needed to be done to fix the problem he was seeing. A division by zero error wasn’t being handled correctly due to a problem with the translation logic. This might have been a fine description for another programmer, but the product owners that review and prioritize bugs might not understand.
I suggested that he may want to add a ‘problem statement’ as the first line of the bug report. This statement needs to be something that can easily be read and understood by everyone who participates in addressing the bug. The product owner, who is less technical, needs to be able to read this to determine its priority. The developer needs to know the problem that needs to be solved. The tester needs to be able to read it to determine exactly what the problem is, so they can assess whether the developers fix has resolved the problem. The developer and I worked together to find a wording that met all of our needs; “Messages with ‘weird’ decimal values like NaN aren’t being caught as errors after calculation. Instead, they fail later in the process and become invisible failures.”
Follow up Testing
As we thought about the wording of our bug report, I noticed the developer had written a very specific aspect of the problem we were dealing with. I suggested that we may want to take a step back, to determine whether there were other important aspects of the bug we could include in our problem statement. In my experience, the trouble with a bug report that is too specific, is that the solution also ends up being very specific.
This means that if there are other related bugs, or if the bug occurs for more than one type of data, the solution may not be sufficient to address the real problem. The related problems may not be discovered until the bug fix is being tested, or worse, until the product is in the customer’s hands. We took a hard look at the problem and we did a little testing, and decided that we could make the bug description more general. NaN weren’t the only values that failed; Infinity and -Infinity were also problems. Furthermore, the problem wasn’t just that the messages errored out, they also weren’t being caught further upstream. We added the new information to the product description. This way, when the developer who fixes the bug looks for a resolution, they’ll look for one that fixes the whole problem, not just part of it.
Helping the Decision Makers
To further assist in prioritizing the bug, I suggested that the second sentence should indicate any important risks there may be in leaving this bug unresolved. It’s important that everything the product owner needs to know, to decide how to prioritize the bug is upfront and easy to find. Our product owner is not likely to dig through the full text of a bug to decide how to prioritize it. We want to make access to that information as easy as possible.
In this case, the bug was found by test data rather than user data, and could possibly be considered an ‘edge case,’ unless we could identify the situation where a user would run into this problem. We added a second sentence indicating that the ‘weird’ values could easily come from a division by zero error, since the messages are generated using customer-written formulas and customer data.
Join 34,000 subscribers and receive carefully researched and popular article on software testing and QA. Top resources on becoming a better tester, learning new tools and building a team.
We left the technical details as he had written them, now as the third line in the report. Finally, we reviewed the title of the report. It was succinct to a developer, but it was focused on the technical details laid out in the description. I suggested that the title is the first thing the product owner uses to evaluate a bug report, so it should be easy to read, easy to find later, and should make the problem obvious. We took the problem statement we had created, and shortened it down to its essence: “Weird decimal values aren’t being treated as failures.”
In all, we probably spent about 30 minutes writing the bug report together. I’m not sure the developer expected as much help as they got, but they seemed to appreciate it. Sometimes in the team room it feels like everyone has a skill to show off, except me. Getting the chance to show off some of the skills I’ve developed as a tester was pretty fulfilling, but not as fulfilling as knowing that I could help a team member.
