Don’t Measure, Judge!
post # 58 — April 25, 2006 — a Managing, Strategy post
The mantra of modern management is (still) ‘If you can’t measure it, you can’t manage it’ and much analytical creativity goes into trying to invent new metrics (eg ‘the balanced scorecard’) in order to justify and monitor new strategies. Corporate officers are always asking for (impossible) ROI calculations on such things as knowledge management, customer satisfaction, innovation and the like.
What this measurement mentality misses is the fact that, as human beings, we are remarkably well-equipped to judge things, and we can form an accurate picture of what is going on by looking, talking and interacting, even for only a short period of time.
Take, for example, the issue of whether or not the employees in a business are excited, enthused, energetic and engaged.
In spite of corporate efforts to design questionnaires to monitor performance (including the infamously ineffective and bureaucratic 360-degree feedback programs), anyone walking into a business unit would be able to detect in a minute the difference between a ‘turned-on’ group of employees and one where everyone was operating like a good, compliant soldier. It would have to be a very sophisticated ‘measurement’ system to capture what any one of us could detect in an instance.
Embedded in the ‘management by measurement’ mentality is a false hypothesis. That effective management can be achieved through detachment – that measurements will be reviewed by a top-level (corporate-style) manager who is not involved in a hands-on way in the business, but monitors what is going on through the scorecards.
There is much in business (perhaps most of the crucial decisions) that cannot be measured in such a way that an accurate meaning can be conveyed to the person examining ONLY the measurements. Measurements can be invaluable in giving guidance to someone who must decide and judge, but are a profoundly pathetic substitute for effective judgments.
The challenge of modern business is not to improve our measurement metrics, but to redesign managerial activities so that the organization is better able to make good judgments.
How would you think about designing an organization that outperformed others by making effective judgments? I don’t think we yet have any science on this, but it would probably include:
a) Delegation of decision making to local managers who are more likely to understand what is going on and are able to make better, informed judgments;
b) A clear set of enforced guiding principles and objectives, so that trade-offs are made consistently across the firm
c) Careful selection and apprenticeship of those who are going to be placed in judgment (managerial) roles
d) Finding ways to ensure that the judges are credible and held accountable for their effectiveness
A good starting point for reading on this topic is an article entitled ‘The New Organization’ in January 21st issue of The Economist quoting Yves Morieux of the Boston Consulting Group.
M. Morieux says that, in the organization of the future, the main tasks will be to help improve the judgment-based decision-making processes that knowledge workers use, not find better ways to measure their performance.
Anyone else have some views on how you improve an organization’s ability to judge, rather than measure?
James Bullock said:
I think you have half of it. Talking with some colleagues, some years ago a few of us astonished the other half of the conversation by claiming that “assessments” of large IT projects or SW development organizations can take as little as half a day, or even half an hour to start. Much like the “engaged employees” example above, it is obvious what is going on. What to do about it takes longer. Doing something takes longer still. The idea is the easy part.
With an assessment like this, “they” know what is going on. They are living it, after all. They’ll tell you, if you listen. So the skill is a kind of open awarness and receptivity, to take in what you are being shown and told. The antitheseis of imposing a preconcieved solution, actually, which can unfortunately look like the same rapid response.
That said, one way to describe how we work is in terms of models of how the world works. Your own work, (Mr. Maister) particularly “Managing the Professional Services Firm” describes some models that are uncommon in people’s minds, yet accurate, evocative, and predictive. Thus the power of describing the models.
I think there’s a time for reflection, and “measures” are tools of reflection, actually. “Hey, I’m pouring more hours into making software, and the useful stuff rate is going down. What’s up with that?” Not “this is wrong” but “what’s up with that?” Measures can help make it obvious when the world operates differently than we think. The measures we pursue reflect, in the end, how we think the world works. So, deciding what to collect is a way of exposing tacit assumptions. Measures are a learning tool, really, which as internalzied, leads to better judgments in the moment.
I’d like to say that this is original. Yet, it’s just what Deming really said, what Toyota has institutionalized, and what Austin wrote about in his book about measurement, among many others. In a serendipity event, just yesterday I got a note from a colleague who talks all the time about “embodiment” so traded him one of my favorite quotes:
“Software development is a performance art. Managing that doubly so.”
Still, I think the addiction to superficial measurement is as debilitating for it’s erosion of reflection as it is for usurping judgment in the moment. But that’s just my opinion, right now.
Thanks.
posted on April 25, 2006