Building an Instinct for Metrics

Metrics are the lifeblood of informed decision-making in any organization. Over the last ~30 years, I’ve built an instinct for metrics and learned to trust data while remaining flexible and adaptive. This proves invaluable when setting OKRs for the first time.  Understanding how to leverage metrics effectively involves recognizing the importance of leading and lagging indicators, identifying proxies for unknown numbers, and balancing qualitative and quantitative research. Here’s how I think about metrics and how you can develop a strong sense for them in your work.

Predicting Results and Learning from Outcomes

The journey to mastering metrics begins with predictions. Asking students to generate a prediction before presenting the correct answer is a popular and effective instructional strategy. Prediction helps learning by providing a benchmark to compare actual results against, enabling you to identify patterns, refine your understanding, and improve future decisions.

Before launching any project or initiative, it’s crucial to set clear expectations about the results you anticipate. These predictions are your hypotheses. As the project progresses, you’ll gather data to compare against your predictions. This comparison is where the learning happens. By consistently predicting outcomes and reviewing the actual results, you start to understand patterns and nuances in your data, helping you refine your instincts over time.

Finding Proxies for Unknown Numbers

In an ideal world, we’d have all the data we need to make perfect decisions. However, we often deal with unknowns and incomplete information. This is where proxies come in. Proxies are indirect measures that stand in for the data we can’t directly obtain. For example, if you can’t measure customer satisfaction directly, you might look at repeat purchase rates or net promoter scores as proxies.

Developing the skill to identify and use proxies requires creativity and a deep understanding of your business. Over time, as you test and validate these proxies, your ability to make informed decisions in the face of uncertainty improves.

Leading vs. Lagging Indicators

A key aspect of effective metrics work is knowing the difference between leading and lagging indicators. Lagging indicators, such as sales revenue or customer satisfaction scores, reflect past performance. They’re important for understanding the outcomes of your efforts but don’t offer insights into future performance.

Leading indicators, on the other hand, are metrics that predict future success. For instance, the number of leads generated or website traffic can be leading indicators of future sales. Identifying and monitoring leading indicators allows you to make proactive adjustments to your strategy before problems manifest in your lagging indicators.

Finding the right leading indicators requires experimentation and analysis. Start by identifying what activities or behaviors typically precede successful outcomes in your business. Test different metrics to see which ones most reliably predict your key results, and focus on these as your leading indicators.

Balancing Qualitative and Quantitative Research

Metrics aren’t just about numbers. Qualitative research plays a vital role in understanding the context behind the data. While quantitative metrics tell you what is happening, qualitative insights explain why it’s happening. Balancing both types of research ensures a more holistic understanding of your performance.

Quantitative research involves numerical data, statistical analysis, and measurable outcomes. It’s essential for tracking performance over time and identifying trends. However, relying solely on quantitative data can lead to missing the nuances of customer behavior and experiences.

Qualitative research, such as customer interviews, site visits, and open-ended survey questions, provides rich, detailed insights. These methods help you understand the motivations, emotions, and pain points of your customers. Use qualitative research to inform your quantitative metrics and vice versa. For example, if your quantitative data shows a drop in user engagement, qualitative research can help you uncover the reasons behind it.

Practical Steps to Develop Your Instinct for Metrics

1. Start with Clear Objectives: Before diving into metrics, define what success looks like. Establish clear, measurable objectives that align with your business goals.
2. Make Predictions: For every initiative, predict the outcomes you expect. Document these predictions and use them as a benchmark for evaluating actual results.
3. Identify Proxies: When you can’t measure something directly, think creatively about what other metrics might serve as useful proxies. Test these proxies to ensure they correlate with your desired outcomes.
4. Differentiate Indicators: Learn to distinguish between leading and lagging indicators. Focus on finding leading indicators that can help you make proactive adjustments.
5. Integrate Research Methods: Use both quantitative and qualitative research to gain a comprehensive understanding of your performance. Allow insights from one method to inform the other.
6. Review and Adjust: Regularly review your metrics and compare them against your predictions. Adjust your strategies based on what you learn, and continuously refine your understanding of what drives success.

Conclusion

Building an instinct for metrics is an ongoing process of prediction, measurement, and learning. By understanding the role of proxies, distinguishing between leading and lagging indicators, and balancing qualitative and quantitative research, you can develop a more nuanced and effective approach to metrics. This skill not only enhances decision-making but also drives continuous improvement and long-term success. Embrace the journey of learning from your data, and over time, you’ll cultivate a powerful intuition for what metrics matter most.