Skip to content
Published on

How the World Works — Incentives and Systems

🇰🇷KO🇺🇸EN🇯🇵JA
Split View필사 모드
Share
Authors

Opening — Behind Strange Events, There Is Usually a Reason

Watch the news for a while and the world can look deeply irrational.

Healthy companies make decisions that hurt themselves.

Well-meaning policies produce the opposite of what they intended.

Smart people gather in a room and reach a foolish conclusion.

But step back and ask "who is rewarded, and for what?" and much of the picture snaps into focus.

People mostly behave rationally from where they stand.

What looks strange is not the behavior, but the structure that produces it.

This essay is about a few lenses for reading that structure.

We will look at how incentives quietly drive behavior, how good intentions curdle into perverse outcomes, and how a measure breaks the moment it becomes a target.

From there we move through systems thinking, feedback loops, second- and third-order effects, power laws, emergence, and leverage points.

Finally, we talk about the humility that complex systems demand.

I will lean on everyday scenes rather than jargon wherever I can.

The goal is not to hand you a formula for predicting the world.

It is to help you see, a little better, what is moving beneath the surface of events.

1. "Show Me the Incentive and I Will Show You the Outcome"

The investor Charlie Munger put it plainly.

"Show me the incentive and I will show you the outcome."

It is a simple line, but a powerful first tool for reading the world.

To understand what someone does, look less at their character or morality and more at the rewards and punishments they live under.

Munger liked to tell an old story about FedEx.

At a night sorting hub, workers could not finish their shift on time no matter how much they were urged.

As long as they were paid by the hour, there was no reason to work faster.

When the company started paying by the shift and letting people go home once the work was done, the problem all but disappeared.

The people had not changed.

The incentive had.

Incentives Speak Louder Than Words

Organizations love to post their values on the wall.

"We value quality." "We respect collaboration."

But what actually drives behavior is not the sentence on the wall; it is what leads to a promotion or a bonus.

If promotions are decided by individual output alone, no amount of shouting "collaboration" will make people collaborate.

If the metrics reward only shipping speed, quality quietly slips to the back.

When incentives and official slogans collide, the incentives almost always win.

So to understand any organization or society, read its reward structure, not its mission statement.

Good People, Bad Structures

An important corollary is that bad outcomes do not necessarily come from bad people.

Decent people produce bad results under twisted incentives.

Conversely, good incentives draw good results out of ordinary people.

Of course, incentives are not everything.

People respond not only to money and status but to intangible rewards like pride, belonging, and meaning.

Still, when you cannot make sense of why a situation unfolds the way it does, "who is rewarded here, and for what?" is almost always a good place to start.

2. Perverse Incentives and Unintended Consequences

If incentives are powerful, then badly designed incentives are powerfully dangerous.

The most famous example is the "Cobra effect."

In colonial Delhi, the authorities wanted to reduce the number of venomous cobras, so they offered a bounty for every dead cobra.

At first it seemed to work.

But soon people began breeding cobras to collect the bounty.

When the authorities noticed and scrapped the program, the now-worthless cobras were released, and the wild population ended up larger than before.

A reward meant to solve a problem made exactly that problem worse.

This pattern, where a reward intended to shrink a problem ends up feeding it, is called the Cobra effect.

What You Can Measure Versus What You Want

Perverse incentives usually share one root.

What we truly want and what we reward are not the same thing.

We want "a safe city" but reward "fewer reported crimes," which creates pressure not to record reports.

We want "a healthy codebase" but reward "a high test-coverage number," which breeds hollow tests that verify nothing.

Reward "short call times" in a call center, and agents will hang up quickly instead of solving the problem.

The harder the outcome we want is to measure, the more people optimize a measurable substitute while the real goal drifts away.

The Streisand Effect

Unintended consequences show up in the world of information too.

In 2003, the singer Barbra Streisand sued to have an aerial photo of her clifftop home removed from the internet.

The photo was part of a large public survey documenting coastal erosion, and almost no one had looked at it before the lawsuit.

Once the lawsuit became known, curiosity exploded, and the image spread to hundreds of thousands of people almost overnight.

The phenomenon in which an attempt to hide something dramatically increases attention to it is named, after this episode, the Streisand effect.

Both stories carry the same lesson.

When you design behavior, ask not only "what happens if this succeeds?" but "how will people react to this rule?"

3. Goodhart's Law — When a Measure Becomes a Target

Named after the economist Charles Goodhart, Goodhart's law can be summarized this way.

"When a measure becomes a target, it ceases to be a good measure."

The anthropologist Marilyn Strathern shaped this widely quoted phrasing.

The core idea is this.

A metric is useful when it is simply an observation that reflects reality.

But the moment that metric itself becomes the basis for reward and punishment, people start gaming the metric rather than improving reality.

The Familiar Example of Test Scores

Think of exams at school.

A test score is meant to be a proxy for how well a student understands the material.

But when a school's funding and evaluation hinge on test scores, teaching narrows to "whatever appears on the test."

Memorizing question types replaces deep understanding, and in some places outright cheating appears.

Scores go up, while the "understanding" they were supposed to measure stays flat or even declines.

The moment the measure became the target, its link to reality was cut.

How to Hold a Metric

Goodhart's law does not mean "never use metrics."

You cannot run a large organization without them.

It means you should treat a metric not as an absolute goal but as an imperfect signal that can always be distorted.

Pin all the rewards on a single number, and that number will inevitably be distorted.

So it is wiser to watch several metrics that check one another, to confirm the real situation behind the numbers qualitatively, and to keep in mind the original purpose the metric was standing in for.

A number is a map, not the territory.

Get absorbed in coloring the map nicely, and you can drift away from where you actually meant to go.

4. Systems Thinking — Stocks, Flows, and Feedback

Everything so far leads into one larger frame.

That frame is systems thinking.

Systems thinking is the habit of seeing events not as isolated points but within the web of relationships among connected elements.

The classic text in this field is Donella Meadows's book "Thinking in Systems."

Meadows describes systems in three languages.

Stocks, flows, and feedback loops.

Stocks and Flows

A stock is the amount accumulated at a given moment.

The water in a bathtub, the balance in a bank account, the technical debt a team has piled up, the trust a society holds.

A flow is the movement that raises or lowers that stock.

Water from the faucet and water down the drain, deposits and withdrawals, new debt taken on and debt paid off.

We often fixate only on the current level of the stock.

But the real behavior of a system is set by the balance between inflow and outflow.

The tub overflows not because the water is "a lot," but because the inflow exceeded the outflow for long enough.

Debt piles up because it was taken on faster than it was repaid, steadily, over time.

Blame the Individual and You Miss the System

The most practical lesson of systems thinking is this.

When bad results keep recurring, the problem usually lies in the structure, not the person.

If you swap the person in charge and the same problem returns, the position itself is probably producing the bad outcome.

A process that keeps creating bottlenecks, a team that always misses deadlines, an organization where good people burn out fast, usually has a structural cause beyond any individual.

This does not erase personal responsibility.

But if you stay stuck on "whose fault is it?", you keep replacing people while the structure remains and the problem repeats.

Only when you shift to "what behavior does this structure encourage?" does a path to a fix begin to appear.

5. Reinforcing Loops and Balancing Loops

What makes a system come alive is its feedback loops.

A feedback loop is a circle in which a system's output flows back into its input and shapes the next round of behavior.

There are broadly two kinds.

Reinforcing loops and balancing loops.

Reinforcing Loops — The Snowball

A reinforcing loop amplifies change in the same direction.

More leads to more; less leads to less.

The image is a snowball that grows as it rolls.

Compound interest, where money earns interest that becomes principal that earns still more interest, is a reinforcing loop.

Popular content that gets recommended more, becomes more popular, and is therefore recommended even more, is a reinforcing loop.

Reinforcing loops create both growth and collapse.

They can run as virtuous cycles or vicious ones.

Trust that breeds cooperation that breeds more trust, and distrust that breeds defensiveness that breeds more distrust, share the same structure.

Balancing Loops — The Thermostat

A balancing loop, by contrast, pushes a system back toward a particular state.

When things drift from the target, it nudges them back.

The classic case is a thermostat keeping a room at a steady temperature.

Too cold, and the heat turns on; reach the target, and it turns off.

Your body keeping its temperature and blood sugar steady, and a market adjusting prices to supply and demand, are balancing loops.

Balancing loops create stability, but they can also become the resistance that blocks a change we want.

The Wobble That Delays Create

Loops often contain time delays.

When there is a lag between action and result, a system tends to overshoot and overcorrect, wobbling back and forth.

Think of adjusting the water temperature in a shower.

If the temperature responds a few seconds after you turn the knob, you impatiently turn it further, then, when it turns scalding, you yank it back the other way.

The same delays hide inside economic booms and busts, gluts and shortages of inventory, and overheated then frozen hiring.

When you work with a system, remember that "no reaction right now" does not mean "no effect."

6. Second- and Third-Order Effects — And Then What?

One habit that separates good judgment from shallow judgment is asking, again and again, "and then what happens?"

Every action has a direct, first-order effect.

But that first-order effect goes on to produce second- and third-order effects.

Shallow thinking stops at the first order; deeper thinking looks past it.

Familiar Examples

Suppose a city widens a road to relieve traffic.

The first-order effect is that, briefly, the road clears.

But as the road improves, more people drive, development spreads to the outskirts, and a few years later traffic is heavier than before.

This is called "induced demand."

Rent control, which caps prices to protect tenants, is similar.

The first-order effect eases the burden on existing tenants.

But the second-order effect is that landlords build less new rental housing or let maintenance slide, and the third-order effect can be that, over time, decent housing becomes even harder to find.

The point here is not that a given policy is simply bad.

It is that if you stop at the first-order effect, you miss the real consequences that unfold afterward.

The Shadow of Good Things, the Reversal of Bad Ones

Second-order effects can even flip the sign.

Something that looks good now exacts a price later, and something that hurts now pays off later.

A convenient painkiller leads to abuse; hard exercise returns as health.

So when you make a decision, it helps to ask one more question.

"If this succeeds, what happens next?"

"And how will people react to that result in turn?"

You cannot see perfectly far ahead, but thinking even one or two steps further avoids a large share of costly mistakes.

7. Power Laws and the Pareto Principle — The Vital Few

We tend to imagine that the world is distributed fairly evenly.

We are used to bell-shaped distributions, like height or weight, where most cases cluster near the average.

But many human systems do not behave that way.

They follow steeply skewed distributions, where a small minority accounts for most of the whole.

Such distributions are called power laws.

The 80/20 Rule

More than a century ago, the Italian economist Vilfredo Pareto observed that about 80 percent of Italy's land was owned by about 20 percent of the population.

He noticed similar skews recurring in other countries and other domains.

From this came the "Pareto principle," commonly called the 80/20 rule.

It is a rule of thumb that most of the results come from a small share of the causes.

The exact figures 80 and 20 are not what matter.

What matters is that contribution is uneven and heavily skewed.

A few products make most of the revenue, a few bugs cause most of the outages, a few cities hold most of the population.

Where to Spend Your Effort

The Pareto principle carries a strong practical implication.

Not everything is equally important.

In Meadows's phrase, there is a "vital few" and a "trivial many."

So finding the small set of causes that drive outcomes and concentrating resources there is often far more effective than spreading effort evenly across everything.

There is a caveat, though.

The "trivial many" are not always safe to ignore.

Rare but catastrophic failures, in safety or trust, cannot be waved away just because they are infrequent.

The Pareto principle is a tool for setting priorities, not a license to discard the rest.

8. Emergence — The Whole Behaves Unlike Its Parts

Complex systems have a curious property.

No matter how closely you inspect the parts, you cannot predict the behavior of the whole.

This property, which appears newly at the level of the whole, is called "emergence."

A single ant is a small creature following a handful of simple rules.

But gather tens of thousands, and intricate trails and a collective intelligence appear that no single ant designed.

A single water molecule has no property of "wetness" or "waviness."

But gather countless molecules, and fluidity and waves emerge.

The Sum of Individuals Is Not the Society

The emergent view is especially useful when looking at society.

A traffic jam is not the fault of any one driver; it is a collective pattern produced by countless individual decisions interacting.

Market prices, a city's culture, an internet trend are the same.

No one created them alone, yet they rise, on their own, out of everyone's interaction.

So trying to explain a social phenomenon by individual intentions alone often misses the mark.

The answer to "why did this culture arise?" usually lies not in one person's decision but in the interaction of many people and the rules that shaped it.

Seeing From the Bottom Up

Understanding emergence also changes your sense of how to change a large phenomenon.

Issuing orders from the top rarely shifts an emergent pattern.

Rather, when you change the rules that govern individual behavior and the way interactions happen, the whole pattern gradually rises into a different shape.

This is why, to change a culture, altering the small rules and incentives people respond to every day tends to work better than chanting slogans.

9. Leverage Points — Why the Obvious Fix Is Often Wrong

One purpose of understanding a system is to know where to push so that change actually happens.

Meadows called such places "leverage points."

They are spots where a small force can produce a large change.

But she stressed a paradox again and again.

We usually do locate leverage points intuitively, yet we often push them in the wrong direction.

Low Leverage and High Leverage

The most tempting intervention is usually to change a number.

Nudge the budget up a bit, tweak the tax rate, add a few people.

Such interventions are visible and easy to execute, but they rarely change the underlying structure of the system, so their effect is often small.

Greater leverage lies in the flows of information, the rules, and the structure of incentives.

Change who can see what, and which behaviors are rewarded, and the behavior of the whole system shifts.

And the most powerful leverage lies in the goal of the system itself, and in the mindset or paradigm beneath it.

When the definition of what counts as success changes, everything below it rearranges.

Structure, Not Symptom

A common mistake follows from this.

We rush to react to visible symptoms while leaving the structure that produces them untouched.

It is like bailing out the overflowing water while never turning off the faucet.

We add more people every time there is an outage but never touch the process that causes outages; we mediate every time a conflict erupts but leave the structure that breeds conflict in place.

Real leverage usually sits where it is hard to see, in the rules, the incentives, and the goals.

So the more tempting the "most obvious fix" feels, the more it is worth pausing to ask whether you are looking at a symptom or a structure.

10. The Humility That Complex Systems Demand

The lenses so far help you see the world more clearly.

But they all point together toward one conclusion.

Before a complex system, you should be humble.

A system with many elements tangled in feedback loops, laced with delays, and prone to emergence is inherently hard to predict.

We can foresee first-order effects reasonably well, but results several steps out sit in fog.

Guarding Against the Illusion of Control

This fact calls for two attitudes at once.

On one hand, it does not mean throwing up your hands and leaving things to luck.

Understanding incentives and structure lets you make better decisions and, at least, avoid the large mistakes.

On the other hand, you should be wary of the confidence that everything can be controlled and predicted.

The grander the plan, and the harder the decision is to reverse, the larger the second-order effect you may have missed.

The Wisdom of Practice

So the way to move wisely in a complex world looks roughly like this.

Start with small, reversible experiments where you can, observe the results, and change course quickly when you are wrong.

Rather than betting everything on a single forecast, prepare for several scenarios and leave yourself some slack.

And above all, keep open the possibility that "I may not fully understand this system."

Below is a simple sketch of one reinforcing loop.

It shows how a small beginning can grow itself.

        [ Trust rises ]
              |
              v
     More attempts to cooperate
              |
              v
     Better outcomes experienced
              |
              v
   [ Trust rises further ] ---> (back to top: reinforcing loop)

Depending on its direction, this circle runs as a virtuous cycle or a vicious one.

That is why a small difference at the start widens, over time, into a large gap.

Closing — The Habit of Reading Beneath the Surface

The world is less random than we think, and at the same time less predictable than we wish.

Behind events that look irrational, there is usually a logic of its own.

The incentive someone is responding to, the structure around them, and the feedback loops that structure produces.

The lenses in this essay are connected.

Incentives drive individual behavior, those behaviors interact within a system to form feedback loops, and from those loops second-order effects and emergent patterns arise.

And over all of it, a small number of factors drives most of the results.

These lenses do not hand you the answers.

They help you ask better questions.

"Who is rewarded here, and for what?"

"If this measure becomes the target, what breaks?"

"If this succeeds, what happens next?"

"Is this a symptom, or a structure?"

Ask these questions as a habit, and the world becomes a little less bewildering and a little more understandable.

Perfect prediction is impossible, but the mere posture of reading beneath the surface brings us closer to more careful and more humble judgment.

Questions to Ponder

  1. In your organization or team, where do the values on the wall diverge from the real incentives? What are people ultimately rewarded for?

  2. Is there a metric you have recently set as a target? Look back for any signs that it began to distort once it became the goal.

  3. Take one decision you made recently and write out its second- and third-order effects, not just the first. Do you see a reaction you were missing?

  4. Find one example each of a reinforcing loop (a snowball) and a balancing loop (a thermostat) in your own life. Which would you like to grow, and which would you like to slow?

References

Discuss on Twitter • View on GitHub