goodhart's law
- "When a measure becomes a target, it ceases to be a good measure."
- "Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes."
- See this thread for how this plays out with rules, although it can also apply to even self-optimization where you think you're aligned
- aka Cobra effect, from an incident in British India
- great opposite-direction example, where the bit about the fire is likely to be respected more because Management isn't
3 linked references
January 2nd, 2023
- Jessica Ocean says that there's an article by Cedric Chin about W. D. Deming solving goodhart's law in the 60s
- aha, found this thread, which also talks about something basically isomorphic to lead & lag measures
- see also, re goodhart's law: