6. POA Techniques
6.12 Metrics and Key Performance Indicators
Guide to Product Ownership Analysis
Purpose
Metrics and key performance indicators (KPIs) measure the performance of the product or other attributes that are relevant to developing the product.
See section 10.28 of BABOK® Guide v3 for details.
See section 10.28 of BABOK® Guide v3 for details.
To create a valuable product that resonates with customers, many characteristics need to be considered. Also, competing goals need to be resolved, including:
From a POA perspective, metrics can be categorized based on the level of impact for the product:
To successfully use the insights provided by these metrics, a framework is needed that aligns the signals from these metrics to assess the product goals. Practitioners can use the HEART framework for this purpose.
HEART Framework
HEART is an acronym that stands for:
For example:
Frameworks allow POA Practitioners to organize and plan how the product needs to evolve by providing a structure to think about the key metrics.
- Business objectives vs. customer needs,
- Short-term benefits vs. long-term benefits,
- Strategic considerations vs. tactical considerations,
- Existing market segments vs. new target segments, and
- Customer experience vs. additional features.
From a POA perspective, metrics can be categorized based on the level of impact for the product:
- Strategic Measures: Determine the effectiveness of a product vision or strategy and are directly tied to business objectives.
- Product measures: Outline customer acceptance and the popularity of the product for the intended customer base.
- Delivery measures: Indicate team performance and outline the team's ability to develop a cohesive product.
| Strategic | Product | Delivery |
| Return on Investment (ROI): Ratio of total revenue to the cost of investment. | Net Promoter Score (NPS): A 10-point scale used to gauge customers' likelihood to recommend the product. | Iteration Goal Success: An agreed set of objectives for an iteration. |
| Monthly or Annual Recurring Revenue per user (MRR/ARR): Ratio of product revenue generated per month/ annum to a number of total users. | Customer Effort Score (CES): Evaluates the level of customer effort required to achieve their objectives, usually on a 5-point scale. | Escaped Defects: A count or a ratio of defects that are uncovered by the customer per release. |
| Customer Acquisition Cost (CAC): Total cost of acquiring a new customer for the product. | Adoption Rates: Measures customer adoption of the product over a period. | Defect Density: Number of defects per line of code (or story point or per sprint). |
| Net Present Value (NPV): The discounted cash flow generated from the product minus cost incurred over the product lifetime. | Feature Usage Rate: A measure of which features within the product are used most often. | Scope Change Rate: Ratio of additional effort included in the iteration scope vs. the original effort estimated. |
| Total Cost of Ownership (TCO): Total cost incurred throughout the product lifetime, including direct and indirect cost. | Retention/Churn: Number of product users retained or churned during a specific period. Either metric can be used, but not necessarily both. | Burndown Chart: Burndown chart indicates the difference between the actual and planned effort over iterations or releases. |
| Internal Rate of Return (IRR): Rate of return or the discount rate when applied equates the net present value to be zero. |
Team Velocity: A measure that evaluates the total number of story points delivered per iteration. |
|
| Time-to-Market (TTM): Time taken for the product, set of features, or single feature to be available and usable by customers. |
To successfully use the insights provided by these metrics, a framework is needed that aligns the signals from these metrics to assess the product goals. Practitioners can use the HEART framework for this purpose.
HEART Framework
HEART is an acronym that stands for:
- Happiness
- Engagement
- Adoption
- Retention
- Task Success
For example:
| Framework | Goal | Signal | Metric |
| Happiness | Improve customer satisfaction | Reduction in the effort spent on completing a user goal |
Customer Effort Score (CES). |
Frameworks allow POA Practitioners to organize and plan how the product needs to evolve by providing a structure to think about the key metrics.
| POA Domain | |
| Applying Foundational Concepts |
|
| Cultivate Customer Intimacy |
|
| Engage the Whole Team |
|
| Make an Impact |
|
| Deliver Often |
|
| Learn Fast |
|
| Obsess About Value |
|