Ungaming the counterweight

Good metrics development is hard. There’s never just one metric,Footnote 1Capital Gains, “There’s Never Just One North-Star Metric” (opens in new tab) (archived Jan 2026). since your operating context almost certainly isn’t dominated by one principal component. And your KPIs have to interrelate anyway, both for standard Goodhart reasons, but more interestingly because your metrics can degrade due to floor or ceiling effects (opens in new tab). Imagine you’re a decision-making body that must publish reasons. You decide to track time-to-decision, reasoning that it furthers transparency, but it’s easy to forget that procedural fairness constrains delivery speed. Eventually you can’t move any faster, and even slight slowdowns later on might not show up in your metric due to the floor.

This is of course why counterweight metrics exist. But what happens when your counterweight is gameable too? Imagine your decision-making body is now pitching an advisory service to proactively guide clients along the happy path. You might track the number of outreach events you organize, or the number of client questions you answer, or how quick your turnaround time is. Obviously you can game this by splitting up your team or churning out more, smaller specs, or whatever. But there’s a deeper outcome problem here because your classic counterweights — think client satisfaction — are also easy to game.

The deliberate gaming strategy is just running outreach with your friends. The unintentional version is more incisive though: your clients might not have exit options, so they’ll probably give your service five stars. This is just another shade of median mismatch; the underserved population disappears entirely when we select for the parties who don’t have other options. And on some level, that’s all your clients, because they’re all bound by the very system that’s aiming to measure them.

If you can’t manufacture accountability from within a captured system, a counterweight for your counterweight loses salience due to inherent solipsism. Gaming the counterweight basically just becomes self-referential measurement. This is a good example of what second-order cybernetics (opens in new tab) calls the eigenform. As our measurement progressively selects for an increasingly captive population, each additional measurement ends up returning the same value. Nobody who’s left has the right incentive structure to provide anything other than the five-star rating. At the limit point, you’re no better off with the counterweight than you were without it.

To escape this trap, you need an external anchor point, something that’s not captured by your own system. In the decision-maker example, this probably looks like an audit function,However, we need to be clear that even an independent audit function can be captured by a larger system. It’s a little like being at arm’s length in a very small room with no exit. or an independent researcher looking at frequency or severity of harm, or anything else that could find adversely against you even when your metrics are sky-high. It takes a bold leader to suggest a metric that rhymes with “let’s see how often we lose on judicial review”, but it’s a concrete way to avoid counterweight metrics that don’t measure anything real.

On the surface, this is the value proposition for management consultants — if your governance model or corporate strategy aren’t working, get an autonomous outside party to interview the players and recommend a structural fix. But everyone they interview will be captured too. The systems fix is to ask someone with independent accountability to observe not the players but rather the consequences, because the purpose of the system is what it does.On some level, the organization that consistently receives great feedback from its captured clientele is measuring exactly what it exists to accomplish. This outside focus on outcomes suggests an entire such network of bilateral relationships. Each is self-reinforcing but the parties are anchored to something beyond their own systemic boundary; this avoids the self-referential metrics trap and allows for properly ungamed feedback. And while individual nodes can be captured, defunded, or hollowed out from within by bad counterweights, the network topology is much more resilient.