incentive design
No content on this page itself — see linked references below.
1 linked reference
February 26th, 2022
- incentives are a thing, but goodharting is also a thing. if you try to create an incentive to X, you can only ever create an incentive to appear to be doing X. which will sometimes correlate with actual X, but will also correlate with deception (including self-deception) & avoidance, particularly when X is difficult or people don't know how to do it.
- but non-created incentives don't work like this, for the most part. I assume you'd agree I have an incentive to eat. pretending to eat won't satisfy my hunger. as long as I'm steering by my actual hunger, I need to actually eat in order to solve my problem. no other incentives are needed to get me to eat, unless I've gotten majorly tangled up in some way, and if so, it's probably better for my health to get me back in touch with my own appetite rather than to layer on more external incentives.
- likewise, I have an incentive to figure out how to make sense of what's going on in my mind and in the world, because it feels good to understand what's going on. this incentivizes reading, meditating, etc, and while it can goodhart somewhat it's at least directly tracking whether I'm succeeding (as far as I can tell) in making sense of things. whereas if someone tries to get me to read or meditate or whatever, it loses track with why I might want to do those things.
- it's not to say that external incentives don't matter—they absolutely do
- eg a house that has a bunch of cheetos and milk chocolate on the counter probably has worse external incentives than the same house without those snacks there
- but removing the snacks is a very gentle nudge, compared to most other attempts to create incentives
- which are designed towards controlling behavior rather than just allowing people to do whatever they feel like but in one context rather than another
- and these incentives will inevitably goodhart
- (for people unfamiliar with goodhart's law)
- "When a measure becomes a target, it ceases to be a good measure"
- (quoting that it becomes more clear that the "removing snacks" incentive-sculpting doesn't actually create targets, which is why it doesn't goodhart. although if "ensure no bad snacks" became a target, that would absolutely goodhart)