Articles / Providing Good Feedback for…

Providing Good Feedback for Bug Reporters

A comment on a bug I submitted recently spurred me to provide some feedback from an application user's perspective on bug reports. There are ways of responding to a bug report that encourage the types of responses that are helpful to developers, and there are ways of responding that only produce anger and frustration, without getting anything fixed. My hope is to encourage good communication between bug reporters and developers to enable better, quicker bugfixes.

My Background

I have been reporting bugs for a few years, on various pieces of software. I've had both positive and negative experiences with the responses I've received from developers. I come from a professional background of using computers, supporting computers, change control, and basic programming, so I've seen problem reporting from a few angles.

To begin, I'll outline responses I have received or seen that were unhelpful. Following that will be responses I have seen that were helpful. Finally, I'll outline some general guidelines which I feel promote good communication between developers and reporters.

Unhelpful Responses

It's not our product, it's some other software causing the problem (with no debugging or troubleshooting).

Trying to shift blame with no investigation does nobody and nothing any good. It only makes the developers look like they don't want to take responsibility for a possible bug or don't want to do any troubleshooting. Some reported bugs do wind up really being a problem with other software. If properly investigated, you've got the data to back the position that other software caused the bug. Posting the exact reason it's a problem with other software to the report is great feedback for the bug reporters, who can then report it to the correct place. They also know you cared enough about their bug to help them get it to the right place, making them more likely to report future bugs.

I can't reproduce it/works for me (with no other qualifications).

That doesn't help anyone, and it sounds like another "I don't wanna do work" response. The fact that person B cannot reproduce a bug does not negate the fact that person A can always reproduce it. Helpful information to add to a comment like this would be: Did you use the same version on the same OS, following exactly the same steps (using menus as opposed to an icon), etc.? The programmers may have done things exactly the same way, but maybe they didn't. The bug reporters don't know unless they are told. Also helpful are comments about what you're trying to do to reproduce the problem, such as "I'm continuing to try this on another machine" or "I'm working with a few people to see if we can reproduce this bug". One failed attempt to reproduce the bug with no further work isn't working the bug, it's akin to lip service. Tell the reporters instead what other troubleshooting information they need to give you to track the bug down.

The two responses above are unfortunately so common in the IT world, and so well-known by users (who aren't fooled), that a company made an IT-style Magic 8-Ball. The answers include:

  • It's not a problem with my code.
  • I can't reproduce the problem.
  • Reboot your computer.
  • Etc.

&!*#*&... We've said a million times not to report x because blah blah...

(And other not-so-subtle "You're stupid for having contacted us" or abusive types of responses).

A polite pointer to the FAQ or appropriate documentation is more appropriate. Telling people they shouldn't have written about one bug will discourage them from writing about any bug.

No response (when assigned).

Hello? Is anyone watching this thing? Even "Bug in queue, will get to this" would be helpful. (I like the note in Mozilla's Bugzilla which says "This means the bug is awaiting triage..."). Getting no response may lead reporters to feel that their time is being wasted.

Helpful Responses

Thanks for your submission.

It's always appropriate to thank someone for taking the time to report a perceived problem, even if it turns out not to be a problem with the software they thought was the problem. Even a badly-written bug report can be helpful with a bit of work (in most cases). The reporter can always be given a link to a "How to report bugs well" FAQ or your project's bug guidelines. Politeness is good. Happy bug reporters help us troubleshoot our software. They are a valuable resource to the Open Source community. Pissing them off means that you don't get as much testing from them. A simple "thank you" and a kind attitude go a long way to make someone feel respected and happy.

I can't reproduce the problem, but here's something you can do to provide me with more information....

Letting people know what else they can do to clarify a bug or provide more troubleshooting information lets them know that their response is not wasted, that the developer cares enough about the problem to want more information and actually wants to get the bug fixed.

Bug confirmed, working on this...

You may not yet know the cause of the problem, and don't have time to write much. A simple note to say you've duplicated the issue and you're looking into it lets the users know their bug report has been received and something is being done. This improves their confidence that their reports mean something.

Suggestions

Better communication and more questions rather than snarky answers make for happier bug reporters and better bug troubleshooting. Here are a few other, general suggestions that go toward improving communication:

Have a written, easily findable bug submission procedure for your project.

People who report bugs may not know what information is helpful for the developers. Your bug submission guidelines will tell them what that information is. You can't guarantee that everyone will read it or follow it. You can guarantee that you will get less of what you need in terms of helpful bug reports if you don't have one.

Try to speak in terms the bug reporter can understand.

Perhaps the meaning of "just run lsmod" is obvious to you, but it may not be obvious to the bug reporters. This doesn't make the reporters bad, nor does it justify getting angry at them. No one is born knowing all computer commands. Not everyone has the same level of expertise in the same areas as a developer. Communicating with the reporters in ways they understand means you will get more helpful answers from them. Sometimes, they can run a command they didn't know before if only you tell them how to do it or point them at the appropriate FAQ.

Don't respond in anger, respond when you are calm.

People are responding to your request to submit bug reports, and they deserve to be treated with a basic amount of respect. Responding in anger will increase the likelihood that you will say something inappropriate or something that you may regret. Wait until you have a cool head to respond to comments.

Respond with common courtesy, don't put people down.

Again, if you're asking people to submit bugs, putting them down when they do is counter-productive. It will only make you and your project look bad. Treating people respectfully increases their willingness to help you get your bugs solved and your programs running more stably. This shouldn't have to be said, but there it is.

There are many things we can do to improve communication in bug reporting. This article provides a few which can be used in most situations. By following practices of courtesy and good feedback, we can improve the rate at which bugs get fixed and encourage the volunteers waiting to help us in that process. The Open Source community as a whole can benefit.


This article is licensed under the Creative Commons Attribution-ShareAlike License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/1.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.

Recent comments

08 May 2004 01:40 Avatar knifemakers

Re: Well said...

> Every developer should read this and
> keep this article in mind.


I double that, i could not have said it better my self. I would like to add another comment, i think the problem with todays programmers is more of an issue where they are working on the same darn thing for so long they just get sick of fixing the bugs. They want to see something new and start programming some more bugs :)

05 May 2004 17:12 Avatar insajt

Both sides
This sheds some light over a few problems which affects both sides. From a developers point of view, I want clear, direct and well worked bug reports. A hasty and unresearched report is as good as no report at all. Then there's the bug reporters angle, which is (as previously pointed out) almost the same as the developers. They take time and sometimes effort to improve something that I, the developer, has messed up. Not always to gain something for themselves. Another rather important issue here is, how far does "common people" go in order to get their bug reported. I mean, some projects actually requires that the user registers on some bug database (with all the mail, approval mail, confirmation mail etc. it generates), find the suitable category, put the bug there, unsubscripe from the list (back to e-mail hell). Of course I understand that larger projects needs this in order to maintain sanity. Now, going through that alone is worth a kiss imho.

Personally, I accept all bug reports myself, through common e-mail, and I try to be as polite as possible in my responses, regardless of how the initial contact was. But sometimes, deep down, I rage upon those who sends things such as "This doesn't work.. bad bad bad Erik!".

So, to sum it up, this is a bigger issue than it at first seems. All bug reporters should follow the steps earlier described in this thread, and all developers should act the way the initial poster means (or better yet, stop producing bugs :-)

10 Apr 2004 00:09 Avatar BuyGifts

Re: Code sollutions

> You might also consider not talking crap
> about someone's patch submision in front
> of them. When someone takes the time to
> actually attempt fixing a problem and
> track you down to give you the patch,
> don't kick them in the teeth. If the
> patch is mistaken there are better ways
> to tell them then saying, "This is
> crap and I am not going to use it."
> There are certain projects I won't be
> helping because this is the responce I
> got when trying to be a helpful user.
>
> This would be under "Common
> Courtesy" in the article, but I
> think it deserves special attention.
> When someone goes out of their way to
> attempt a sollution to the bug be nice
> to them.
>
> NR


I totally agree with you... Your right on the money!

23 Feb 2004 10:47 Avatar nroberts

Code sollutions
You might also consider not talking crap about someone's patch submision in front of them. When someone takes the time to actually attempt fixing a problem and track you down to give you the patch, don't kick them in the teeth. If the patch is mistaken there are better ways to tell them then saying, "This is crap and I am not going to use it." There are certain projects I won't be helping because this is the responce I got when trying to be a helpful user.

This would be under "Common Courtesy" in the article, but I think it deserves special attention. When someone goes out of their way to attempt a sollution to the bug be nice to them.

NR

06 Feb 2004 07:37 Avatar kvandivo

Re: Reality Check

>
> KotH,
>
> Mr. Clark is simply reminding us that
> the bug reporter is not paid for his
> effort.


On the contrary, a bug reporter typically _is_ expecting to be paid for the effort, in the form of a bug fix that fixes the problem that they are experiencing at that particular moment.

Screenshot

Project Spotlight

Kigo Video Converter Ultimate for Mac

A tool for converting and editing videos.

Screenshot

Project Spotlight

Kid3

An efficient tagger for MP3, Ogg/Vorbis, and FLAC files.