canyonwalker (![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
canyonwalker) wrote2024-08-31 07:21 pm
canyonwalker) wrote2024-08-31 07:21 pm
Entry tags:
Metrics: Gaming the System
Recently I've been writing about the use of metrics in the workplace, especially the pitfalls of using them poorly or simplistically. The pitfall I'm writing about in this chapter is how when a measure becomes the objective it ceases to be a good measure.
This is often called Goodhart's Law (Wikipedia link), based on the writings of a British economist who challenged UK economic policy in the 1970s:
Gaming the system is what happened when salesperson "Missy" scheduled bullshit sales calls to earn bonuses and accolades, as I described in a previous blog. Her behavior was a corruption of what the company intended to achieve, which was to increase sales and reward people who contributed to that. Instead Missy contributed nothing to sales, and wasted other people's time in the process, and collected extra pay for it.
It's worth noting that not all instances of gaming the system are as harmful as Missy's. Sometimes employees game the system but only cheat themselves. I've seen plenty of instances in sales, for example, where employees are asked to complete a small bit of training and fake their way through it. This has often happened when pressure is applied through management. Managers are held to account for 100% of their staff completing the task. It becomes a statistic displayed on the screen at weekly meetings of the management team in front of senior executives. Managers then turn to their employees and say things like, "Just get the green check-mark" -- putting the emphasis on securing the indicator of success instead of absorbing the training and learning from it to actually become more successful at the job.
So, do witty sayings like Goodhart's Law and Campbell's Law mean that all metrics are doomed? Hardly. They're really arguments that poor or simplistic metrics, and excessive focus on single metrics as indicators of success, cannot replace comprehensive understanding of actual success.
This is often called Goodhart's Law (Wikipedia link), based on the writings of a British economist who challenged UK economic policy in the 1970s:
"Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes."The adage was later generalized to other areas of policy, especially once stated in the less dry terms I used above. (That restatement was published by Marilyn Strathern in 1997.) But as it applies to the use of measures in business, or even in government policy, there's an earlier observation that states the problem even better, Campbell's Law:
-- Charles Goodhart, 1975
"The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."Campbell's formulation is a bit dry and academic, too, so let me offer my own restatement: When a system focuses on simple measures, there's risk people will game the system.
-- Donald T. Campbell, 1969
Gaming the system is what happened when salesperson "Missy" scheduled bullshit sales calls to earn bonuses and accolades, as I described in a previous blog. Her behavior was a corruption of what the company intended to achieve, which was to increase sales and reward people who contributed to that. Instead Missy contributed nothing to sales, and wasted other people's time in the process, and collected extra pay for it.
It's worth noting that not all instances of gaming the system are as harmful as Missy's. Sometimes employees game the system but only cheat themselves. I've seen plenty of instances in sales, for example, where employees are asked to complete a small bit of training and fake their way through it. This has often happened when pressure is applied through management. Managers are held to account for 100% of their staff completing the task. It becomes a statistic displayed on the screen at weekly meetings of the management team in front of senior executives. Managers then turn to their employees and say things like, "Just get the green check-mark" -- putting the emphasis on securing the indicator of success instead of absorbing the training and learning from it to actually become more successful at the job.
So, do witty sayings like Goodhart's Law and Campbell's Law mean that all metrics are doomed? Hardly. They're really arguments that poor or simplistic metrics, and excessive focus on single metrics as indicators of success, cannot replace comprehensive understanding of actual success.