To move from “data” to “insights” you have to look at more than one metric. However moving from an “insight” to telling a “story” requires understanding the synergies between metrics — just like a plot has multiple characters in play and the story comes to life through the interwoven relationships between them. A data story answers the following questions: what, why, so what, who, which.
The biggest challenge with data analysis is not knowing where to begin. I grew up in the analytics world and this thought often plagued me as well. Where do I start? And more often than not, I started searching for outliers and took that as a starting point. For any given analysis, my story could change based on where I started.
Until finally one day it occurred to me – the framework to help guide data analysis. As I use this framework more and more, I may tweak and optimize it to make it more efficient, but it is a pretty good first step to achieve efficiency in your analysis process.
The framework contains three steps: KPIs, Catalysts, Benchmarks
Using all three types of metrics from framework will empower your insights.
Let’s look at them individually:
- Are whole, absolute numbers
- Clearly indicate an increase or decrease in volume
- They are either more or fewer
We often use a Cost Per Conversion as a KPI, but does it really tell us the health of the program? If the CPConv. is low, does it give us the full picture? No. But if we look at driving higher conversions, we at least know if the program is helping with the growth. Then we can investigate if this growth is being driven at a higher or lower cost.
KPI (Key Performance Indicator) tells us the “What” for a campaign or program or initiative.
- Are fractions, percentages, rates
- Clearly indicate an increase or decrease in trend
- They do not provide an indication for volume
The increase or decrease in volume can be pinned down to a catalyst. Perhaps we had a lower click-through-rate on our campaign, or a higher conversion rate.
Catalysts help us answer the “why?” for the KPI.
- Are usually averages
- Clearly indicate behavior or point of reference
- They do not provide an indication for volume
While “catalysts” help explain the whys behind a “KPI”, the benchmark is a measure of the impact on the barometer. Look at the two examples below:
1. The increase in revenue (KPI) was due to an increase in CTR (Catalyst) but the Average Order Value (Benchmark) remained unchanged.
- So what does this mean? We are simply driving more traffic of the same quality
2. There was an increase in revenue (KPI) while the CTR and Conversion rates (Catalysts) remained constant. However, the Average Order Value (Benchmark) increased
Thus, Benchmarks help explain the “So what”.
- So what does this mean? We are now driving more qualified traffic
As you can see, the combination of the KPI, Catalyst and Benchmark helps us gain an insight.
You may have noticed that I always use KPI in the singular form. That is because you should always have such good clarity about your goals that you know the ONE most important measure of success. Any more than one and you don’t really know what you want to achieve. Be focused. Be clear.
Now that we have an insight, how do we get to the story?
We investigate the “Catalysts” or the “Benchmark” to understand the “which” for our story. For a paid search campaign, we can investigate an increase in CTR to “which keywords or ad copy drove a higher CTR”. Or for any digital program, “which campaigns or ad placements drove the higher AOV”.
Finally, as a bonus, we try to find out “who” drove this change. It could be a marketplace change or a political announcement, impact of an offline campaign like a TV ad etc.
At the end of this process you have your story. For example:
Last week we saw a 20% increase in revenue. This was due to an increase in click-through-rate on “xyz” related keywords. Our average position on the keyword remained constant at #1. however, our TV sponsorship for the abc event led to this spike in traffic.
Now that you have a complete understanding of the story, it is so much easier to provide a recommendation:
We recommend aligning our campaigns to these events to ensure coverage for such spikes in future.
I have been following this process for two weeks now and I am more efficient because now I know which metric to focus my attention on at a time instead of looking at all of them simultaneously.
I know the examples I used are over simplified, but that is just to get the point across. This has helped me with more complicated analyses as well. And I hope this helps you too!
Finally, I’ll leave you with a list of commonly used metrics that fall in the three buckets described in the above framework. Go find your story!