Category Archives: Experiments

Make Small Mistakes: A Mood Is Not A Personality

Alex Tabarrok has a new paper out showing that easy availability of guns increases the number of suicides. (Read the comments section on that post, it’s full of good tidbits like the fact that there are more suicides farther from the equator.) This is an econometric study, but there is a psychological angle:

Our econometric results are consistent with the literature on suicide which finds that suicide is often a rash and impulsive decision–most people who try but fail to commit suicide do not recommit at a later date–as a result, small increases in the cost of suicide can dissuade people long enough so that they never do commit suicide.

In other words, while we think of some people as “suicidal” this is just the fundamental attribution error rearing its ugly head. Another example:

  • That other person didn’t signal before changing lanes because they’re a bad driver.
  • I didn’t signal before changing lanes because I forgot, I’m tired, my kids are yelling in the backseat, etc.

People’s behavior is determined by the situation, their feelings are transient or generated on the spot. Very little of their behavior can be pinned to permanent characteristics or explicit intentions. But our first inclination is the opposite. If this  idea tickles your fancy — if you’d like to learn a lot more about how the situation can affect your behavior — read The Person and the Situation, even though it didn’t make my list of top 5 behavioral economics books and even though Malcolm Gladwell wrote the Introduction. You should also browse this deck of cards showing how the physical design of the environment can affect your actions.

There are also fascinating implications for the study of crime. Gary Becker revolutionized the field by pointing out that crime isn’t done by “criminals” — it’s done by ordinary humans who face different costs and benefits than the rest of us. Of course, this isn’t the final word. Most crimes are crimes of passion; between a fifth and a third of prisoners were drinking at the time of their offense. To prevent crime, we don’t need to make 25 year sentences longer, we need to somehow get around all that System 1 decision-making. And an important new paper shows that Cognitive Behavioral Therapy is staggeringly effective:

The intervention … included … in-school programming designed to reduce common judgment and decision-making problems related to automatic behavior and biased beliefs, or what psychologists call cognitive behavioral therapy (CBT). Program participation reduced violent-crime arrests during the program year by … 44 percent … the benefit-cost ratio may be as high as 30:1 from reductions in criminal activity alone.

The paper also finds improved schooling outcomes. Reducing the importance of your automatic decision-making can have huge benefits. The alternate strategy — what Tabarrok’s paper on suicide suggests — is that you should also lower the size of the mistake that your System 1 self can make. Your impulsive self can really screw you over if you’re not careful: make sure he doesn’t have a gun.

I Might Have Been Wrong: On Experimentation

In my previous post I railed against a political fundraising technique I saw in the wild (on Twitter). I was upset because the experimental literature suggested that the technique they used was a waste of money, and I went on to lament the campaign’s lack of interest in science.

But it’s possible they were a step ahead of me. After all, a true science-driven campaign would be doing their own experimenting because of external validity concerns in the paper I mentioned. So maybe I was just in one of several treatment groups. If that’s the case I hope they’re tracking “disdainful blog posts” as an outcome variable of interest.

Why (Behavioral) Science Matters

Almost a month ago Bill de Blasio, Democratic nominee for NYC mayor, tweeted something that made me angry. If I were to donate to his primary campaign, he (or a 19 year old unpaid intern) proclaimed, my contribution would be matched 6 to 1. So now we know that de Blasio — unlike Barack Obama — is not running his campaign according to the latest research findings.

Bad economics from a mayoral hopeful.

Bad economics from a mayoral hopeful.

Political fundraising has long been an inexact science, so in the mid 2000s, two economists partnered up with a US non-profit and ran an experiment about what works best when a charity is raising money. Specifically, they looked at the effect of matching funds. Are people actually more likely to donate if their donations are matched? Their findings were surprising.

It turns out that offering a 1:1 match made people more likely to donate and raised the total amount of money that the charity raised. But higher matches — which essentially give donors more bang for their charitable buck — have no additional effect. 6 to 1 sounds high, but the evidence demonstrates that a one-to-one match would have worked just as well and so five sixths (83%) of that donor money is being wasted on a match when it could just go to general operating expenses.

That’s right, folks: a political candidate brazenly ignored a six-year-old paper from the American Economic Review. Pitchforks and torches would not be an over-reaction.

No, the real reason this matters is as a signal. Every organization on the planet, whether a political campaign, a business, or a government agency, at some level needs to influence people’s behavior. Traditionally this has been done based on intuition but now we can use state-of-the-art scientific knowledge about behavior. Hire a chief behavioralist, or just someone whose life was changed by Thinking Fast and Slow. It would have taken a day, tops, for someone to review the literature on the science of fundraising, but no — instead the campaign was guided by hoary rules of thumb that have never been exposed to the scientific method (tagline: “the known universe’s best tool for knowledge accumulation”).

Of course research matters elsewhere as well. From seawalls to transit expansion to services for the indigent, every policy decision that de Blasio will have to make as mayor can be research-informed — and good decisions are those that use prior knowledge intelligently. If de Blasio doesn’t see science as important for his campaign, if he can’t see how it would help him win election to office, what are the odds that he will use it to make better policy? Pick your political candidates based on how much they strive to use the best possible information to make their decisions —  vote for fans of scientific research.

Portfolio Theory and the Laboratories of Democracy

The common thinking is that the USA benefits by having many states because they will have different policies and we can thereby learn which policies are most effective.1 This idea is intuitive but actually I think the opposite is the case: as government gets larger it becomes more (theoretically) capable of doing the kind of experimentation that leads to better policy.

Think of a government as holding a portfolio of policies. Each one has an expected return and a variance.2 Some policies are less likely to succeed than others, but they make up for it by having higher potential upside or being cheaper up front. However you wouldn’t want to construct a portfolio out of only risky policies because you don’t want your whole government program to fail.

So it’s elementary that one governing body making 50 policies will have more experimentation–and learn more, and have better policy next year–than 50 governing bodies each making 1 policy. In fact in the latter case they would probably all take the safest possible policy.

To be sure, states may may differ on what they think is the safest possible policy (or what a good outcome looks like). For example, several states are moving toward legalizing marijuana and the federal government has not done so. But state “experimentation” likely crowds out federal experimentation. If we had more real, by-design experiments on the federal level, that would be the weak point that advocacy groups3 would attack. State level policy is a hole in the dike; the pressure would come through somewhere else if this channel were blocked off.

Moreover, even though states differ, they are still all doing what they think best–they are not, by and large, consciously experimenting. And experimentation has to be conscious because experiments can be designed to maximize what you learn from them.

Think for a minute about classroom size. States (or districts) have different maximum classroom size rules, so we can use differences in outcome to see the effects of classroom size on learning. But then we learn only about how classroom size matters between 20 and 30 students in a class under standard teaching methods.  On the other hand, a conscious experiment would introduce much more variation and we might find something new entirely. This is another way of saying that state “experimentation” can only tell us where the local maximum is, but we might be stuck on a very short hill next to a high mountain.

One other concern is that some policies (for example, voting rules) are harder to experiment with on a centralized level because the unit of analysis is the government. That’s a fair point–but most policy is not like this.

For most policies, looking at portfolio management theory suggests that a centralized government is likely to take more risks, do more experimentation and learn more than decentralized states.

  1. This idea is known as “laboratories of democracy” and was brought to my attention by the Charles Pierce series of the same name. []
  2. Whether you think about this on the level of “the government wanting to be effective” or “politicians wanting to seem effective to get re-elected” the analysis is the same though some of the results may be different. []
  3. Or lobbies, or special interests. []