Goodhart's Law

Goodhart's Law states " . . . when you attempt to pick a few easily defined metrics as proxy measures for the success of any plan or policy, you immediately distract or bait people into pursuing the metrics, rather than pursuing the success of the policy itself."

Better: as Andy Grove supposedly said, "For every goal you put in front of someone, you should also put in place a counter-goal to restrict gaming of the first goal."

Even better: economist Glen Whitman wrote:

With just an iota of economics training, most people catch on to the importance of incentives. "Aha! To get people to do what we want, all we have to do is reward the good stuff and punish the bad stuff!" Alas, the world is not so simple. People don't always respond to incentives in the ways you might predict. What distinguishes good economic thinking from bad is recognition of the subtle, creative, and often unforeseen ways that people respond to incentives. Ignoring the complex operation of incentives is a recipe for unintended consequences.

Comments

[john Henry]

Quite a while ago I read the story of a Russian shoe factory.

First they were judged on how many shoes they made so they made only a single size.

Then the Gosplan decided that didn't work so they judged them based on the weight of shoes produced. They produced only very heavy boots.

It went on like this. I wish I could find it again as it would be a useful example in some of my class lectures.

I suspect that it is apocryphal. On the other hand, stranger things were done in the Russian economy so it might have been true.

John Henry

[david foster]

My post "Stupidity: Communist-style and Capitalist-style" has a couple of interesting and amusing examples:

http://chicagoboyz.net/archives/5604.html

[jorgxmckie]

I try to really emphasize that idea when I teach cost accounting in public and NPO budgeting. I'll tell them that if they can name a an actual, definable goal, I can develop a metric to measure it, but I can't stop people from gaming the system.
Good examples in our systems are things like measuring policing by 'activity' [i.e. writing tickets], or in an NPO by the number of volunteers signed up [which doesn't mean they'll all want to do what you need].

[Clayton]

This is my problem with companies that focus on "stock market goals" and forgetting they have a business a run; they're not just there to please investors' every whim.

Related to the medical articles posted earlier, this is the danger in statin drug use to attack cholesterol levels. Those measurements are supposedly accurate indicators (I posted earlier about the counter claims to their accuracy). The problem we see now is that when people go after that metric, they typically make the body as a whole less healthy.