Hiring A VP
How and When to Do Layoffs

How Much Data Is Too Much?

Dashboard 2 car

We live in a world that is rich in analytic software and fabulously scalable databases that let you pivot, ply and bend trend lines in fantastically interesting ways. Which is great, because it enables companies to have much more comprehensive data than ever before.

However, the multitudes of data also provide ample distraction and conflicting indicators for those who are disinclined to make decisions.

To be sure, there are a few key metrics in every department that you should focus on. These include measuring Annual Recurring Revenue (ARR), burn rate, cash runway, the cost of acquiring customers, average deal size, net expansion and churn, customer satisfaction (typically the Net Promoter Score or NPS), uptime for a cloud service and so on. Even the basics are a lot to measure.

And if keeping track of these core metrics is good, maybe tracking more metrics is even better, right? In my experience, tracking metrics are like drinking shots of espresso. A little goes a long way. Too much, and you can easily introduce jitter into the system.

The problem with measuring too many metrics, is you lose focus on what is the most important metric. For example if you’re measuring the efficiency of your customer service team do you track Net Promoter Score? Or the customer satisfaction rating of each support ticket? Or the churn across customers? Or the number of support incidents per customer? Or the time it takes to resolve each incident? Or how many touch points it takes to resolve? And should that be the average or the median? Should you categorize it by customer segment? Or by subscription plan? If you measure all of these and some are trending up and some are trending down, what does that tell you about your business? And more importantly, what should you do about it?

In short, the more metrics you measure, the more likely you are to get conflicting views about how you’re doing. And when teams have conflicting metrics, they will naturally emphasize the metrics that are improving and downplay or ignore those that are going in the wrong direction.

At the executive level, it’s better to focus on a small set of metrics. Focus on the basics, like growing ARR, reducing churn and increasing uptime and customer satisfaction. Make your product easier to use, easier to buy and get customer input when you build your product roadmap.

At the departmental level, there might be a need for further drill down to get more operational detail. For example, in Marketing, it makes sense to measure the results of specific campaigns and the resulting pipeline, marketing qualified leads (MQLs), sales accepted leads (SALs) and so on.

But be careful of creating a culture where analysis takes precedence over action. Metrics and analysis are a means to an end, not an end in itself. If the data does not drive action, what is it for?

I was on the board of a company that was in the analytics space itself. They had grown to over a hundred million an annual recurring revenue (ARR) and the CEO was an absolute wizard when it came to analyzing the performance of the company. They used their own software to analyze leads, customers, sentiment, forecast, projections, churn, you name it. But they rarely hit their quarterly sales targets. I remember one board meeting where the head of Sales presented the results. He had missed his target by 60%. But he made a point of saying that his forecast of the miss was dead accurate. This is not the kind of culture you want to create.

Boards and investors often put a premium on predictability of the business. The holy grail for venture investors is to take a high growth company public and that requires a high degree of revenue predictability to gain confidence in the public markets. However, for startups, and especially those under $50 million in revenue, it is expected that there will be a high degree of volatility in the business. Typically, in the early stages of a company, the number of large revenue deals (%100k or more) is extremely lumpy. Some quarters you might get several deals and sometimes there may be none at all. That doesn’t mean the business is bad, it just means it has not yet achieved sufficient volume to be predictable. Or said differently, if you have a run rate of 10 $100k or larger deals per quarter, one or two slipping can be managed. If you only get one or two and one or both slip, that’s hard to make up.

Be careful in your quest to make the business predictable you are not over-indexing on analytics at the expense of taking action. After all, if the reporting of metrics does not result in new insights and action items, why are you collecting them?

What’s the essential data in your organization? What do you measure every week? What actions result from this data? Let me know by posting a comment below.

Comments

Feed You can follow this conversation by subscribing to the comment feed for this post.

The comments to this entry are closed.