Bug vs. Defect

Are you drowning in work items? This could be the cause.

Riddle me This

I will describe three scenarios, and I want you to think about how your team responds to each.

Scenario #1

A user reports behavior in the production system that does not function as it should.

Scenario #2

When testing a new feature that has yet to be released, a team member discovers some unmet acceptance criteria.

Scenario #3

When testing a new feature that has yet to be released, a team member discovers an unexpected behavior that already exists in production and is unrelated to the functionality the team is implementing.

A Bug by any other Name

In his book, Scrum Shortcuts Without Cutting Corners, Ilan Goldstein produces the two following critical definitions.

An issue is a problem that occurs during the sprint and is tightly coupled to a user story that has not yet met its definition of done.

A bug is a bug only if it is identified after a user story has been completed and accepted by the product owner.

Now, I must admit that I’m not a fan of the word “Issue.”

I agree with the sentiment that it should only be a bug once the story has met the definition of done and therefore presumably exists in production. Suppose the situation doesn’t meet this definition. In that case, it is much more likely that the team can resolve the problem more quickly within the sprint without the additional overhead (i.e., work item creation).

However, the word “issue” in my mind is too closely related to “impediment.” There is a chance this is due to my experience using Azure DevOps. Depending on which project type you use, impediments are sometimes called “issues.”

I prefer the word “defect,” but word choice isn’t what’s important. Use whatever terminology resonates with your team. The critical element is that we reduce waste within the sprint and focus on getting product backlog items done.

Why You Should Care

In my experience, most teams respond to all three scenarios in the same manner. First, they create a product backlog item to track the unexpected behavior, and then the product owner prioritizes those work items along with other work items in the backlog.

If your team spawns bugs from user stories that do not yet meet the definition of done, I implore you to pause for a second and ask what value you’re getting from that activity.

“Judge us on the quality of our products, and not the quantity of our paperwork.”

-Michel Van Mellaerts

I propose that the waste created by the extraneous work items far outweigh any benefit. Worse yet, processes like these typically indicate a more profound cultural issue that you might want to resolve. To understand those concerns, let’s review some typical justifications for following this process.

Waste and Root Causes

If it wasn’t specified in the acceptance criteria, then it needs to be a different story.

Why? Who made up that rule? Who does this rule benefit?

Let’s look at a couple of principals from the Agile Manifesto:

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

And let us not forget about this value:

Individuals and interactions over processes and tools

If we have a problem with a user story that has not yet met its definition of done, why would we create yet another work item to track just this specific problem? Especially if the original work item cannot meet the definition of done without resolving the issue. By creating the additional work item, we will introduce a dependency in our requirements.

I don’t know about you but, I’ve never seen a tracking system that handles dependencies well. I’ve been in daily scrums where we’ve gotten whiplash from ping ponging between tickets. Oh, this ticket can’t move forward until someone squashes this bug. That bug is also dependent on this other bug that we found. In the end, we need a separate map just to get back to the beginning of the chain.

If we missed acceptance criteria, why don’t we have a quick conversation with the product owner to make sure it makes sense to cover this particular edge case at this time. If so, we can simply update the acceptance criteria and pass the story back to the developer while the codebase is still fresh in their mind. Don’t overcomplicate things with unnecessary product backlog items.

If we continue to add scope to a user story, that story will never be finished

This point could be valid in some cases. However, we need to be sure this is one of those cases before putting forth the effort to create a work item.

I recommend that the team should confirm three things before spawning tickets for a defect:

1. The defect is a separate and distinct work item that will not prevent the release of the original work item
2. The effort to resolve the defect is larger than the effort to create the work item
3. The team does not have to fix the defect in the current sprint

If these three statements aren’t fact, the team should update the existing work item, include the defect details, and resolve the defect before allowing the story to meet the definition of done.

If you’re updating the acceptance criteria, Ilan Goldstein suggests including the date and initials of the person who added the new entry. I believe this is a fantastic idea as it can be quickly done and increases the traceability of the change.

Blame Avoidance

These reasons might take several different shapes. Here are a couple of examples:

1. If the product owner missed the requirement, they should have to create a new ticket. (As if the process is some sort of punishment for not covering all edge cases.)
2. I don’t want management to question why I’m taking so long with this ticket. (As if avoiding the wrath of missing a deadline is more important than delivering value to the customer.)

Blame culture can be a toxic impediment that should be dealt with swiftly. If your team takes either of the stances noted above, you may want to consider some root cause analysis. Consider running a retrospective focused on cultural concerns to help identify potential improvements.

We want to keep our cycle time low

This one is a bit like sweeping dirt under a rug. The purpose of tracking metrics like cycle time is to get an idea of how we’re trending, faster or slower. Presumably, if your cycle time is trending up, we’d want to determine the root cause and find solutions.

Splitting off issues as a separate ticket to keep our cycle time lower may be artificially reducing our cycle time and thus cheating us out of an opportunity to recognize potential improvements.

If the defect is critical to the initial ticket meeting the definition of done, let the original ticket stand for both the original work and the additional work to solve the defect. In this way, the cycle time for this ticket should be longer than that of an average ticket.

With the complete history of what happened tracked in a single work item, the team can evaluate root causes and institute process changes that will have an overall positive effect on all future tickers, actually reducing cycle time overall.

It might be that developers and QA should pair for a short design session before writing the first line of code. Maybe the product owner should be working more collaboratively with the team when developing user stories. Perhaps the team isn’t asking the right questions in backlog refinement and planning meetings. Whatever the reason, we’ll never know the real issue if we game the metrics to make it appear as if there is no issue.

Works Consulted

• Scrum Shortcuts Without Cutting Corners: Agile Tactics, Tools & Tips

TLDR

Treating defects like bugs will have you drowning in tickets. Depend on your definition of done, face-to-face communication, and ruthless improvement to prevent the flood.