Benchmarks

post # 128 — July 11, 2006 — a Managing post

Here’s today’s question (via email):

I am the Marketing Manager for a Technology Consulting firm in Perú. The partners and I have held long discussions over the real benefit and application of industry performance ratios such as operational ratios for our business.

What is your take on these indicators? Are they actually available and if so are they useful as benchmarks?

While there are many reputable firms who make a living producing industry reports on performance ratios, I always find them so darn expensive to buy. On the surface, they look as if they give valuable information, but the longer you think about it, the less information content they really seem to contain.

Many of these industry studies are based on information collected from the companies themselves, with little attempt at external validation. As we all know from financial scandals, there’s more than one way for a company to present any set of financial statistics, and many of these studies do not spend much time trying to achieve consistency in what they report. In many cases they cannot.

For example, you say your firm has “partners.” That probably means they would like to know a ratio like “profit per partner.” But you can play a thousand games with that – who’s a partner? What’s profit?

There is so much potential for misunderstanding when each ratio is taken in isolation.

For example, you might look at one company and see it has a margin of 50 percent, while a second has only a margin of 3 percent. Does that mean the first company is doing better? No, it could just mean that the first company is a jewellery store (high-end consultants, high fees, low-leverage) and the second company is a supermarket (low prices, lots of leverage). The supermarket could still have a better return on investment than the jewellery store, which has to finance its own very large inventory.

This seems like an obvious example, but the reasoning applies to all ratios – they only make sense when you can put them in the context of all the other ratios and extract the whole picture.

Next, you have the problem of averaging. Suppose you relied on a survey company which averaged the margins of the two companies I described above, and reported that the average margin was 26.5% Does that average number have any meaning? Probably not, but that’s what gets reported in most surveys.

A third problem I have is that I suffer the tragic history that I have a masters’ degree in mathematical statistics. That means I can never look at a survey average without asking myself “But what’s the standard deviation?” It tells you nothing to know you’re 10 percent above the standard industry ratio if they don’t tell you what the normal variation around the average is. Is 10 percent above a great performance or just “noise?” But the information to judge the variance rarely gets reported.

I could go on, but now I have to reverse my conclusion on you. In spite of all of this (and the problems of inter-regional comparisons, etc, etc.) I still would want to look at the numbers. If you know what you’re doing (that’s a big IF) and you don’t rely on them too much, then it’s better to know the reported operational ratios than to not know them.

As someone trained in numbers, I always like to say that there is no such thing as an objective, stand-alone numerical measurement system. All measures are, at best, SIGNALS to prompt further investigation and reflection. Used that way, they can be very help[ful and I’d look at the industry ratios (as long as they weren’t too expensive to get.)

Anybody else want to help the questioner with some experiences and views?