Defect Management
Customer Facing and WIP (work in progress) Defects are best prevented by writing a failing test and immediately resolving the failure in the application. Recording a defect is an anti-pattern usually when problems are discovered downstream from when introduced. Continually work towards testing as the application work is conducted to keep test coverage and accuracy current, and test runs passing.
- A Defect or Bug found during or after the related work in process completes that will survive past the work, is a regression found while testing other areas, or was found in the field. This Defect or Bug should gate a feature release if found before release. If found later in the field, it into the backlog with stories, tasks, and other technical debt. This is considered "customer facing" because it exists in or will be introduced to the field.
Select a Priority and Severity that reflects the issue recorded. Be careful to not group this issue in Business Critical if it doesn't warrant it. If the VOC Volume features are enabled, then the web panel will appear in the sidebar for the issue with relevant guidance. If that guidance doesn't look right then adjust the Priority, Severity, or both accordingly.
Adding Validators for the Create transition in Bug (Defect) workflows will block the creation without selecting a value other than the default "None" value.
The Defect workflow provides a minimum set of statuses that match up to workflow statuses for other issue types. These statuses appear as steps on the Kanban board (Task Board) along with other work items. You progress through the workflow by either buttons on the top of the page when viewing a defect, or by dragging and dropping the defect on the nest step in the Task Board.
💡 Each of the Defect Resolutions are represented by separate transitions from "In Development" to "Ready for Test" that update the Resolution field with the transition value in a post transition function. The Resolution value is cleared for both the "Return to Development" and "Reopen" transitions with a post transition function. There is a "Change Resolution" transition in the Closed status to update Defects that are "Unresolved" (no value selected), to select a Resolution.
The Resolution field is a Jira system field. The Resolved field is a Date/Time field that is updated with the last change to the Resolution field, or cleared when the Resolution field is cleared.
- Fixed: A fix for this issue is checked into the tree and tested.
- Fixed Indirectly: The problem no longer exists and may have been resolved by fixing another problem.
- NPTF: (No Plan To Fix) The problem described is an issue which will never be fixed.
- Duplicate: The problem is a duplicate of an existing issue.
- Incomplete: The problem is not completely described and requires more clarity.
- Cannot Reproduce: All attempts at reproducing this issue failed, or not enough information was available to reproduce the issue. Reading the code produces no clues as to why this behavior would occur. If more information appears later, please reopen the issue.
- Enhancement Request: This problem is not a Defect, and a new feature request is required.
- FAD: (Functions As Designed)
- External Party: Depends upon 3rd party
- Not Yet Implemented: The feature is not yet ready for testing.
- DBA: (Data Base Administrator) fixed the database in the environment.
Impact to the business. Priority reflects the voice of quality based on these descriptions, orthogonal to the backlog ranking given by the Product Owner. Conflicts between Priority and Rank order should prompt conversation in the team about the true impact to the business, with Rank or Priority adjusted accordingly.
- Blocker: Must be the first thing we do at all costs. For bugs this usually means the issue blocks development and/or testing work, production could not run.
- Critical: Key item without which the release will be deemed unsuccessful. For bugs, this is a user-facing problem likely to affect lots of customers.
- High: Top item for release or bugs causing loss of function for some customer or high priority internal users.
- Medium: Nice to have feature or bug with an obvious workaround.
- Low: Not a big deal. Assume won't get done!
- None: (default) Not Specified
Impact (or anticipated impact) to the Customer. Please review and adjust this Defect Severity Classification as appropriate during defect triage, such that the business criticality is accurate. Unusable: The defect affects critical functionality or critical data. It does not have a workaround. Example: Unsuccessful login, complete failure of a feature.
- Painful: The defect affects major functionality or major data. It has a workaround but is not obvious and is difficult. Example: A feature is not functional from one module but the task is doable if 10 complicated indirect steps are followed in another module/s.
- Annoying: The defect affects minor functionality or noncritical data. It has an easy workaround. Example: A minor feature that is not functional in one module but the same task is easily doable from another module.
- Polish: The defect does not affect functionality or data. It does not even need a workaround. It does not impact productivity or efficiency. It is merely an inconvenience. Example: Petty layout discrepancies, spelling/grammatical errors, making an error message more friendly.
- Enhancement: Customer is requesting
- None: (default) Not specified
Combining Priority and Severity results in balancing the priority risk to the team and the impact to the customer. The combinations within the pink box are "Business Critical" weighted priority.
The "Defect Counts and Trends" JIRA Dashboard Gadget visualizes the customer facing, customer found, and escaping defects, both total and business critical. The scope of months can be set to relative (time of rendering looking back n months) or absolute (start and end dates).
These charts illustrate the escaping defects monthly. This example shows a 2 year trend of continuous improvement.
This pair of charts focuses on the Customer Found (bottom two colors combined) represented in the black line chart on top and its trend line. Business Critical (pink bar charts) include all defects with Priority in (Blocker, Critical, High) and Severity in (Unusable, Painful, Annoying). Each data value is clickable with a URL to run the matching JQL query in the Issue Navigator in a new tab. This allows drilling into and further analyzing the resulting data. The trend line legend indicates the increase (red) or decrease (green) in net defect count month over month, and provides the r2 value as a % of confidence the trend will continue (0 is no confidence, 1 is 100% confidence, and .50 is 50% confident. This is the measure of variability from the trend line slope.
-
The darkest colors are Defects with linked Support Request (SRQ) created before the defect, indicating that the customer found this problem before Engineering. -
The middle shade in each chart are Defects with Support Request (SRQ) associated with these Defects where the Defect was created before the Support Request, indicating that Engineering found and reported the problem prior to the customer. -
The lightest shades are Defects with no linked Support Request (SRQ), indicating that Engineering found the issue and the Customer has not (at this point).
These charts are intended for use by team leadership in assessing defect densities, while focusing more on customer found trends as a subset of customer facing defects. These charts are also intended for use by the senior leadership in establishing targets and gap analysis for the Product Delivery organization. By focusing on customer found defects, one can interpret a very low customer found count with a high customer facing count that risk is being managed well by providing value to the customer with little support tracks traffic. However, one can also interpret a high customer facing count as a craftsmanship and SDLC concern to explore further.
✅ Drilling into the data on this chart will return the JQL for that data in the Issue Navigator in a new browser tab. This allows you to use the JQL in other queries, such that the data in that context will match the context where the project trends are assessed. The "Customer Found" values on the line chart represent the JQL query for the combination of the two most dense color regions of the bar chart below.
- Defect rate of resolution trend illustrates how long issues remain active once customer facing. The net add frequency illustrates the churn of fixing and finding new problems not resolved when discovered, and is an indicator of craftsmanship and SDLC concerns. Aging is the defect density in brackets of time span between when created and when resolved (e.g. < 1 week, < 1 month, < 1 quarter, > 1 quarter). New charting and measures underway...
This gadget visualizes month over month net additional defects, created vs. resolved counts, exploded view of created vs. resolved per 3 event horizons, and the average age of defects resolved in this month. This assumes compliance with workflows above, adding an appropriate resolution to the defects upon corrective action for each defect. This chart provides drill-in to each total, opening a new tab with a JQL query in the JIRA Issue Navigator. The Months are drill-in to a data table page with an explanation of each value in that month. The scope of months can be set to relative (time of rendering looking back n months) or absolute (start and end dates). For relative dates, the last month fills in only the % of data for that month.
This chart illustrates the created v.s. Resolved monthly for each of (current months, 6 month horizon, and over 6 months). This example shows a 2 year trend of continuous improvement.
ℹ️ Note that the >1-6 month chart is inverted from the same origin as the 1st month chart, visualizing the area of unresolved defects for all of 1-6 months together. If defects are not resolved in the month discovered, they accumulate in the next month defect counts in the next horizon. Following this same pattern, >6 months falls to the last horizon as lost defects (should be closed as NPTF or converted to another VOC type at this point).
Imagine this CTCT Created vs. Resolved aging chart as looking at an iceberg in the ocean, where the current month reported is above the surface, the 1 to 6 months horizon below the surface, and >6 month on the floor of the ocean. As defects age they move downward in the iceberg and then fall to the floor of the ocean. VOC (Voice of the Customer) tracking is managed by Tier 3 Customer Operations and captured in JIRA Support Requests issue types linked to Salesforce support records. These Support Requests are also linked to Defects in projects. These Support Requests are linked to customer engagements in Salesforce.com through a custom integration called NOVO, which capture counts per type of customer engagement, and feed this information back to JIRA within 24 hours of accuracy. The Support Request issues contain the aggregate numbers for instances of customer engagement and sentiment of how much an issue this is for the customer. This VOC Volume guidance is provided to the project team in several forms, including a banner with a Defect report and in the VOC Volume gadget. Refer to the document on VOC Volume Guidance and VOC Volume Reporting for more information about this data and integration.
ℹ️ TIP: Several chart gadgets that come with JIRA Agile can help visualize these trends. Use a filter for your project defects in each chart gadget.
- The "Average Age Chart" gadget displays the average number of days issues have been unresolved.
- The "Created vs. Resolved Chart" gadget shows created and resolved lines over time, colorizing the lines and areas between the lines, and also provide a net sum of adds (+/). This chart also includes an optional line plot of unresolved trend an a separate area.
- The "Resolution Time" chart gadget illustrates elapsed time between created and resolved.
- The "Time to First Response" chart gadget illustrates the number of hours taken to respond to issues.
When "VOC Volume" is enabled, there is a linked relationship between a Support Request issue and an issue in scope, the sentiment banner with populated count fields and their labels appear in the side panel. Here is an example:
VOC (Voice of the Customer) tracking are captured in JIRA Support Requests issue types and linked to Bugs (Defects) in product delivery projects. These Support Requests capture support engagement counts per type of customer engagement. The Support Request issues contain the aggregate numbers for instances of customer engagement and sentiment of how much an issue this is for the customer. This VOC Volume guidance is provided to the project team in several forms, including a sidebar banner within a Bug (Defect) report and in the VOC Volume dashboard gadget.