Defect Management
Customer Facing and WIP (work in progress) Defects are best prevented by writing a failing test and immediately resolving the failure in the application. Recording a defect is an anti-pattern usually when problems are discovered downstream from when introduced. Continually work towards testing as the application work is conducted to keep test coverage and accuracy current, and test runs passing.
- A Defect or Bug found during or after the related work in process completes that will survive past the work, is a regression found while testing other areas, or was found in the field. This Defect or Bug should gate a feature release if found before release. If found later in the field, it into the backlog with stories, tasks, and other technical debt. This is considered "customer facing" because it exists in or will be introduced to the field.
The "Defect Counts and Trends" JIRA Dashboard Gadget visualizes the customer facing, customer found, and escaping defects, both total and business critical. The scope of months can be set to relative (time of rendering looking back n months) or absolute (start and end dates).
These charts illustrate the escaping defects monthly. This example shows a 2 year trend of continuous improvement.
This pair of charts focuses on the Customer Found (bottom two colors combined) represented in the black line chart on top and its trend line. Business Critical (pink bar charts) include all defects with Priority in (Blocker, Critical, High) and Severity in (Unusable, Painful, Annoying). Each data value is clickable with a URL to run the matching JQL query in the Issue Navigator in a new tab. This allows drilling into and further analyzing the resulting data. The trend line legend indicates the increase (red) or decrease (green) in net defect count month over month, and provides the r2 value as a % of confidence the trend will continue (0 is no confidence, 1 is 100% confidence, and .50 is 50% confident. This is the measure of variability from the trend line slope.
-
The darkest colors are Defects with linked Support Request (SRQ) created before the defect, indicating that the customer found this problem before Engineering. -
The middle shade in each chart are Defects with Support Request (SRQ) associated with these Defects where the Defect was created before the Support Request, indicating that Engineering found and reported the problem prior to the customer. -
The lightest shades are Defects with no linked Support Request (SRQ), indicating that Engineering found the issue and the Customer has not (at this point).
These charts are intended for use by team leadership in assessing defect densities, while focusing more on customer found trends as a subset of customer facing defects. These charts are also intended for use by the senior leadership in establishing targets and gap analysis for the Product Delivery organization. By focusing on customer found defects, one can interpret a very low customer found count with a high customer facing count that risk is being managed well by providing value to the customer with little support tracks traffic. However, one can also interpret a high customer facing count as a craftsmanship and SDLC concern to explore further.
Drilling into the data on this chart will return the JQL for that data in the Issue Navigator in a new browser tab. This allows you to use the JQL in other queries, such that the data in that context will match the context where the project trends are assessed. The "Customer Found" values on the line chart represent the JQL query for the combination of the two most dense color regions of the bar chart below.
- Defect rate of resolution trend illustrates how long issues remain active once customer facing. The net add frequency illustrates the churn of fixing and finding new problems not resolved when discovered, and is an indicator of craftsmanship and SDLC concerns. Aging is the defect density in brackets of time span between when created and when resolved (e.g. < 1 week, < 1 month, < 1 quarter, > 1 quarter). New charting and measures underway...
This gadget visualizes month over month net additional defects, created vs. resolved counts, exploded view of created vs. resolved per 3 event horizons, and the average age of defects resolved in this month. This assumes compliance with workflows above, adding an appropriate resolution to the defects upon corrective action for each defect. This chart provides drill-in to each total, opening a new tab with a JQL query in the JIRA Issue Navigator. The Months are drill-in to a data table page with an explanation of each value in that month. The scope of months can be set to relative (time of rendering looking back n months) or absolute (start and end dates). For relative dates, the last month fills in only the % of data for that month.
This chart illustrates the created v.s. Resolved monthly for each of (current months, 6 month horizon, and over 6 months). This example shows a 2 year trend of continuous improvement.
- Note that the >1-6 month chart is inverted from the same origin as the 1st month chart, visualizing the area of unresolved defects for all of 1-6 months together. If defects are not resolved in the month discovered, they accumulate in the next month defect counts in the next horizon. Following this same pattern, >6 months falls to the last horizon as lost defects (should be closed as NPTF or converted to another VOC type at this point).
- Reading Aging Chart
Imagine this CTCT Created vs. Resolved aging chart as looking at an iceberg in the ocean, where the current month reported is above the surface, the 1 to 6 months horizon below the surface, and >6 month on the floor of the ocean. As defects age they move downward in the iceberg and then fall to the floor of the ocean. VOC (Voice of the Customer) tracking is managed by Tier 3 Customer Operations and captured in JIRA Support Requests issue types linked to Salesforce support records. These Support Requests are also linked to Defects in projects. These Support Requests are linked to customer engagements in Salesforce.com through a custom integration called NOVO, which capture counts per type of customer engagement, and feed this information back to JIRA within 24 hours of accuracy. The Support Request issues contain the aggregate numbers for instances of customer engagement and sentiment of how much an issue this is for the customer. This VOC Volume guidance is provided to the project team in several forms, including a banner with a Defect report and in the VOC Volume gadget. Refer to the document on VOC Volume Guidance and VOC Volume Reporting for more information about this data and integration.
Several chart gadgets that come with JIRA Agile can help visualize these trends. Use a filter for your project defects in each chart gadget.
- The "Average Age Chart" gadget displays the average number of days issues have been unresolved.
- The "Created vs. Resolved Chart" gadget shows created and resolved lines over time, colorizing the lines and areas between the lines, and also provide a net sum of adds (+/). This chart also includes an optional line plot of unresolved trend an a separate area.
- The "Resolution Time" chart gadget illustrates elapsed time between created and resolved.
- The "Time to First Response" chart gadget illustrates the number of hours taken to respond to issues.