Establishing priority is the most important step when reporting issues. You need to determine what has to get fixed before shipping. You don’t need to fix every bug — you need to fix the right bugs.
The bug reports’ priority determines the order that they should be addressed. When deciding on priority you should consider both severity and frequency.
Severity
Severity is a characterization of the seriousness of the problem. Some typical levels of severity are:
- Showstopper - e.g. blockers; development/testing cannot continue
- Critical - e.g. missing functionality; data loss; frequent crash
- Major - e.g. missing functionality; incorrect behavior; bad UI
- Minor - e.g. incorrect behavior; appearance problems
- Trivial - e.g. refactoring; pixel-level adjustments
Frequency
Frequency tells us how often the bug has been observed and provides important context to the severity.
The general nature of a problem is not enough to characterize it. You also need to know the frequency. Is a crash always a “critical” bug? Not if it was only seen once in 6 months of development. Is a pixel alignment issue always “minor”? Not when it appears at an obvious point in every launch of the app. Frequency can make a big difference to the overall priority of a problem.
Useful terms for frequency are: Always, Often, Sometimes, Once
Guidelines
The choice of wording here is significant. These are guidelines, so you’ll have to use your own judgement to apply these principles to the specific situation at hand. And, if you’re unsure, take a guess. Make a note in the bug report that says you’ve guessed on the priority, and others will know to change the priority if they feel strongly that it should be different.
Blocker: development or testing cannot continue
The highest priority is given to bugs that block others from working. Bugs that keep others from testing or developing other features should be fixed immediately, so that the whole team can keep working.
Critical: product cannot be shipped
Second place goes to really bad problems: missing functionality, major features that don’t work, frequent crashes, data loss, and changes that were specifically requested by the client as Priority One.
Major: product should not be shipped
Third are problems like missing functionality or problems in minor features, incorrect behavior, obvious performance problems, bad UI, crashes that happen occasionally.
Minor: product could be shipped
Fourth we have issues and problems of marginally incorrect behavior or appearance issues and subtle performance issues in minor features. These are often subjective issues — things that weren’t carefully specified or things on which people have different preferences.
Trivial: would be nice to fix, but…
Finally, we deal with issues that are not visible to the user or client (e.g. refactoring code), pixel-alignment issues that not everyone notices, upgrades to a new API that have no visible change to app behavior, or changes to the wording of dialogs that don’t affect meaning.
Finally
No doubt you’ll notice some overlap between the meaning and descriptions used in severity and priority. Some bug reporting systems don’t differentiate between the two, or simply give you one field. The label is unimportant, what is important is that you establish the order in which the bugs are addressed and then follow through.
