We have never lived in a more data-rich environment, with numbers all around us and data influencing decision making like never before. This has clear benefits; business leaders have empirical evidence to guide thinking and this can be presented to support instinct. However, what is often glossed over are the two fundamental problems which undermine this state of affairs:

1. Not all data analysis is created equal and data provenance is often poorly, if ever, articulated.
2. The level of data literacy across all levels of business is typically not as high as it needs to be.

How can we expect good decisions to be made by people that may not understand what the data is telling them, nor how the conclusions were arrived at in the first place. This is not a recipe for a high performing, data-driven organisation – it’s a recipe for disaster.

So what’s the solution to this problem?
By asking the right questions, anyone from the most seasoned analyst to a data novice can equip themselves with the tools to succeed. Over the next couple of months I’ll share my thoughts on the fundamental questions to ask to give you confidence in the data in front of you.

What does that number mean?
The first thing to ask yourself is ‘what does that number mean?’. A good place to start is how it was calculated, where the original data was taken from and what has been done to it since. Say, for example, that you’ve been presented with some figures saying revenue has increased by 25%. Great news, bonuses all round! But taking a step back, at this point it would be sensible to ask ‘what does that number mean?’. We all know that a percentage is just the ratio of the numerator (top number) to the denominator (bottom number). This percentage is a change in revenue over time, telling us that the value in one period of time is 25% larger than the value in another period of time. When put like that you can see how something presented as evidence for a positive change is actually fairly arbitrary without context. In order to decide what to do with this information, you actually need to understand:

1. What period of time are we measuring the change over? Covid has had a huge impact on all manner of things, not least businesses revenue. Comparing figures for 2021 to 2020 would almost certainly show a significant improvement. Doing the same comparison against a year before Covid may even show a reduction this year. Knowing the time frame being used is absolutely critical to identifying what might be driving the change you are observing.
   
   
2. How have the two revenue values been calculated? We need to check that the method being used is consistent between the two periods to know whether we are actually comparing the same things. If one metric includes different revenue streams to the other, this would not be a fair comparison.
   
   
3. What are the absolute values? Percentages are dimensionless values, simply a ratio between two numbers. That means a change in either of the two can make a huge difference to the percentage. If my numbers are small, then even a tiny change will look like a large percentage increase.

If we take all of that into account, a clearer statement to accompany the presentation of the figures would be something along the lines of:

“Between 2021 and 2022, the revenue across all of our teams increased from £10.0 Million to  £12.5 Million, representing a 25% increase.”

This is a very simple example, but numbers and charts can lend a veneer of scientific credibility to a statement that often isn’t backed up by even the most cursory of questioning. Although it’s a good thing that data is increasingly being used to influence, ultimately analysis and evidence also need to accompany it when used as part of decision making. This means asking the right questions at the right time in order to make sure everything is as it appears to be, achieving the best results for you and your business.