ALM Analytics

Metrics & Reporting

Subject Matter Areas
Sample Reports
Sample Dashboards
Source Metric Overview Business Driver
Requirements (CaliberRM) Requirements Growth Tracks the number of active requirements as a snap shot over time. It is important to the success of any project that requirements be identified and documented, but continuing to add and change requirements adds risk to the success of the project. This metric allows you to monitor requirements growth over time in order to both monitor the process and spot trends.
Percentage of Requirements by Status Provides the percentage of Requirements by Status. As requirements are gathered and documented, they pass through a series of statuses from draft to approved to signed-off. By monitoring the overall percentage of requirements by status project managers can get an accurate objective view of the process, assess how close to completion they are, and identify and address bottlenecks.
Requirements Activity Tracks the cumulative increase in the total number of changes over time. Similar to Requirements Growth, Requirements Activity provides a cumulative count over time in order to monitor the process and spot trends.
Requirements Volatility After Baseline Tracks cumulative changes to requirements as a percent of the total requirements over time. This metric can also be displayed with a status indicator such as 0 - 25% green, 26% to 50 % yellow, and over 50% red. While requirements growth and activity look at absolute values volatility analyzes the percentage of overall requirements that have changed since the last approved baseline. Tracking the percentage cumulatively over time allows users to monitor requirements volatility over the entire project life cycle and if necessary take corrective action before it negatively impacts the project.
Untraced Requirements by Type Tracks the percentage of each type of requirement that is missing appropriate traces. Allows for a comprehensive analysis of all requirements and provides management with an assessment of how well requirements are traced, either up or down stream.
Reviews Conducted Tracks the percentage of requirements reviewed and passed by type (peer, team, etc.) Allows management to monitor both compliance and overall status of the project by providing a snapshot of the percentage of requirements that have been reviewed and subsequently passed review.
Requirements by Status Tracks the number of requirements by status (can filter by type). Efficiently gathering, documenting, and approving requirements is critical to the overall success of a project. This metric provides a snapshot of the number of requirements by status with the ability to filter by type to assist in monitoring status and project scope.
Requirements by Type Tracks the number of requirements by type (can filter by status). Another view to help manage the requirements process, this metric provides a snap shot of the number of requirements by type with the ability to filter by status to assist in monitoring status and project scope.
Average Throughput time (Draft - Accepted) Tracks the average time it takes for requirements to move from Draft to Accepted by Type. Regardless of how well requirements are gathered and documented, bottlenecks in the approval process can cause delays and often involve resources external to the development team, this metric helps monitor the time it takes to have requirements accepted.
Average Throughput time (Draft - Signoff) Tracks the average time it takes for requirements to move from Draft to Signoff by Type. Regardless of how well requirements are gathered and documented, bottlenecks in the approval process can cause delays and often involve resources external to the development team, this metric helps monitor the time it takes to have requirements approved.
Change Management (StarTeam) Change Requests By Type Count of active Change Requests by Type such as defect, enhancement, issue, risk, and requirement. Change is inevitable in software development and what is important is how that change is managed. This metric provides a snapshot of the active Change Requests at a point in time for a given project or portfolio.
Change Requests By Status Count of all Change Requests by Status such as new, open, deferred, closed, in progress, and fixed. Typically displayed in lifecycle order this metric also helps manage change by providing a snap shot of all Change Requests at a point in time to provide a high level view of how they are moving through the process. The ability to filter for a specific Type allows more focused management of specific types of changes such as enhancements, issues, risks, and requirements.
Active Change Requests By Week Count of all active Change Requests over time. By definition Change Requests are unplanned and tight management and monitoring is important in order to limit the impact to project schedules, this metric allows the number of active Change Requests to be viewed over time to identify trends and track progress. More detail and analysis is available by filtering for a specific Type.
Average Time To Resolve Tracks the average to resolve a Change Request over time. This metric also helps address the need to effectively manage Change Requests so as not to impact overall project timelines. By maintaining the Average Time To Resolve Change Requests over time, you can review frames where things slowed down, or spot trends. Additional detail and analysis is available by filtering for a specific Type.
Change Requests by Severity Counts open Change Requests by Severity. Provides the ability to further manage active Change Requests of all or a selected type by providing a count of open Change Requests by Severity at a point in time.
Change Requests by Priority Counts open Change Requests by Priority. Provides the ability to further manage active Change Requests of all or a selected type by providing a count of open Change Requests by Priority at a point in time.
Defects (Quality Center) Phase Detected by Cause Type A cross tabulation of closed defects counts by phase detected (when in the project life cycle was the defect identified) and cause type. Since reviewing and testing are ongoing throughout the project, identifying defects sooner in the project life cycle (especially certain types of defects) is an important factor in implementing a successful project. Reviewing defects by both type and the phase detected it allows management to better monitor the quality assurance process both during and after project completion and provide feedback and process improvements to help ensure that certain types of defects are detected early on in the process.
Defects by Status Provides a count of all Defects by Status. While a successful quality assurance effort identifies defects, management must then adapt the project plan to address and correct these defects. A snapshot of all the identified defects by type can help plan and manage the defect resolution process.
Open Defects Over Time Tracks a count of "Active" Defects (Status not equal to Closed or Withdrawn). Early on the quality assurance process defects are identified at a faster rate then they are being resolved. While this is healthy early on, the project can not be completed until this trend is reversed. By monitoring open Defects overtime management can bet an accurate object assessment of project status and quickly identify potential delays.
Open Defects by Severity Over Time Tracks counts of "Active" Defects (Status not equal to Closed or Withdrawn) over time by Severity. An effective measure of how close a project is to completion, as well as the overall health of the projects, is the rate of resolved defects to new defects. This metric tracks all open defects by Severity over time. Early in a project this would trend up, and trend down late in the project – a good indication that the project is being completed.
Open Defects By Priority Over Time Tracks counts of "Active" Defects (Status not equal to Closed or Withdrawn) over time by Priority. An effective measure of how close a project is to completion as well as the overall health of the projects is rate resolved defects to new defects. This metric tracks all open defects by Priority over time. Early on in a project one would expect this to trend up, however trending down is a good indication that the project is being completed.
Average Time to Fix by Severity Tracks the average time it takes to resolve a Defect over time by Severity. While monitoring the number of open defects provides an assessment of overall progress, monitoring the average time it takes to correct defects provides insight into the process itself. In turn, this can be used to identify staffing and other issues affecting the team. By breaking it down by Severity management can monitor how the team is responding to different types of defects.
Average Time to Fix by Priority Tracks the average time it takes to resolve a Defect over time by Priority. Monitoring the number of open defects provides an assessment of overall progress, monitoring the average time it takes to correct defects provides insight into the process itself. In turn, this can be used to identify staffing and other issues affecting the team. By breaking it down by Priority, management can monitor how the team is responding to higher priority issues.
Defects by Cause Type Provides the percentage of closed Defects by Type (typically formatted as a Pareto Chart). Data from the quality assurance process can not only be used to help manage the project, but can also provide valuable insight into potential areas for post-completion process improvement. Reviewing the percentage distribution of Defect Types, it is easy to identify areas that cause more then their share of problems. By utilizing a Pareto Chart, the 80/20 rule can be applied to identify the types of defects that make up 80% of the reported problems.
Defects by Phase Detected Counts the number of closed Defects by the phase they were detected in (when in the project life cycle was the defect identified). Another metric that can be utilized to review results both during and post project is the count of defects by the life cycle phase in which they were detected. By reviewing this metric, management can assess and identify areas in the project life cycle that can be improved to identify defects sooner and reduce overall risk.
Test Cases by Status Counts the number of test cases by status. Just as important as identifying defects is the process by which they are corrected. This metric allows an assessment of number of test cases by status.
Percent of Test Case by Status The percentage of test cases for each status. Similar to Test Cases by Status, this metric looks at the percent of test cases by status to provide a more relative assessment.
Project Management Cost Performance Index (CPI) One of the PMI defined Earned Value Management (EVM) metrics, the CPI looks at the ratio of budgeted cost of the work performed to the actual cost of the work performed. Budgets and estimates are only valuable when they can be relied on as a true predictor of future results. Analyzing the ratio between budget and actual costs provides both insights into ongoing projects as well as valuable feedback into improving estimation on future projects.
Schedule Performance Index (SPI) One of the PMI defined Earned Value Management (EVM) metrics, the SPI looks at the ratio of the budgeted cost of work performed to the budgeted cost of scheduled. A valuable tool in ongoing project management for both a static point and time as well as over the course of the project, this metric provides an assessment of project completion based on project budget and the remainder of budgeted work to complete.
Cross Tool (Caliber & Quality Center) Requirements Volatility to Number of Defects Compares the percentage of requirement changes after an approved baseline to the number of defects. While a certain level of defects is part of the development process, many defects are more process driven and can be reduced or eliminated with process improvements. One common cause of process-driven defects is requirements volatility, and improving the requirement identification and approval process, as well as the process for changing requirements, can reduce defects, cost and cycle time.