The Art of Measuring Success
Context
During my first few internships, I was struggling to understand the bigger picture of projects. I tended to work on assigned tasks and not questioning my manager's requests or gain deeper understanding. One of my learnings was that when working at a startup, you should understand that it is a race against death. The runway of your startup dictates when you are going to die. It is really important to understand how long your runway is and ways to extend it.
It was not until I decided to work full-time on my venture that I started thinking about impact, results, and outcomes. I took the habit of setting goals by defining how success would look like for specific tasks. Setting key performance indicators (KPIs) to the different features I was working on helped me in understanding the importance of my work. Through prioritizing my different tasks and setting clear timelines, my work became much more organized. When setting KPIs, I had to ensure that they would align with the organization’s goals as well as the different team capacities and constraints of the company.
If you are interested in starting a venture, look up this post:
Here are questions I usually ask myself:
- What makes my work valuable?
- What am I actually trying to achieve? And why?
- How does success look like?
- When do I know I am done?
- What would be the best use of my time at the moment?
- How do I make it easier for other people to do their work?
- Why should I do X task over Y task at this specific moment?
- How do I align my personal incentives with my business incentives?
Success Metric Definition Process
1. Envisioning Desirable and Undesirable Futures
I usually start this process really qualitatively. My first step is to envision future scenarios of desirable and undesirable outcomes. It is important to execute this exercise creatively, but have a precise vision about what you are trying to accomplish. Envisioning success is important because it helps in understanding the high-level goals. Furthermore, understanding undesirable outcomes help in considering factors that would lead to failures.
Here are some questions to ask yourself:
- Which behavior do I want to trigger from my target user?
- How do I want the future to look like for the user?
- How will my solution improve people’s lives?
- How should the product or service impact users’ life?
2. Identify Leading Variables and Constraints
Once you have the desirable and undesirable states defined, I usually ideate on possible factors that would lead to these different states to happen. I make assumptions on these factors by consulting my team in order to make informed decisions. During this step, it’s important to consider elements such as internal capacity, product constraints, and business factors.
Here are some questions to ask yourself:
- What would lead to a desirable scenario to happen?
- What do I need to achieve to get to this desirable state?
- What are the main possible obstacles to success?
- How do I overcome obstacles and avoid the undesirable state?
3. Find a North Star Metric (NSM)
Once I’ve qualitatively defined the desired future state, I identify one specific metric that best aligns with the desired future state. This metric should best capture the core value your product delivers to customers. This metric is going to be the most important metric to evaluate your product success. NSM is also referred to as a lagging indicator. More about North Star Metric
Based on this metric, I usually identify 3 other trackable metrics that have a strong likelihood to impact the north star metric. The metrics are called leading indicators. Leading indicators help me get more tactical when I am thinking about the feature I am working on. It acts as an anchor point to assess the specific product/design decision and helps in task prioritization.
Example:
Your goal is to increase sales of a product on an e-commerce site. According to previous research, you learned that 95% of your sales occur 7 days after people subscribe to newsletters. In order to make the desirable state happen, you choose “number of subscriptions” as lagging indicators. Based on this lagging indicator, you would identify 3 metrics that are strongly likely to affect the lagging indicator. You could use metrics such as bounce rate, click-through rate, and/or average time spent. These metrics are called leading indicators.
4. Monitor, Experiment, and Iterate
Once you have these KPIs are defined, I design for them in order to maximize the likelihood of success and minimize failure. In order to monitor your path towards success, you can also define the steps you are going to take to reach that defined successful outcome. Since a lot of these metrics are based on assumptions, I make sure to experiment with them and use the data to iterate on them. I am usually not scared to change them when I learn that the assumptions are wrong. The same method can be used for problems such as employee engagement, increase sales or any internal process improvement. Lastly, defining the success of a project and finding metrics is highly contextual to what you are trying to achieve, my biggest advice is to keep an evidence-driven approach and be strategic about how you set goals.
