Steven Levitt misrepresenting his source

Steven Levitt has risen to stardom by riding on the overblown rhetoric resting on the overblown claims of Freakonomics. It is, unfortunately, in the nature of popular books that they oversimplify and over-claim. Academic literature is supposed to be different: rigorous, cautious and, of course, peer-reviewed for accuracy.

Of course, if all those attributes really applied, a career like that of Levitt would have been impossible, since the econometric methodology he employs is far too weak to be able to produce with any credibility the kind of results Levitt is aiming at. It is clear, therefore, that within the community within which Levitt works, some standards of critical thought have been suspended. However, even when credulity is being stretched and poorly supported statements are taken as proven, one may still hope for the superficial ground rules to apply. Specifically, for example, one hopes that when previous research is cited and summarized the findings of the research are fairly represented.

In 2004 Levitt published a paper in the Journal of Economic Perspectives called “Understanding Why Crime Fell in the 1990s: Four Factors that Explain the Decline and Six that Do Not”. As is already clear from the title, Levitt strikes a confident tone.

As an aside, I note that this confident tone is rather surprising in face of the fact that none of Levitt’s four preferred factors accounts for the dramatic rise in crime in the late 60’s and early 70’s. Levitt simply ignores this issue completely, despite the fact that he surely has considered it, as the article has a section about extending the analysis backwards from 1990. But Levitt conveniently extends it only back to 1973. In Freakonomics he (and/or his co-author Stephen Dubner) add the assertion that the “1960s and 1970s were, in retrospect, a great time to be a street criminal in most American cities” since it was “the heyday of a liberal justice system and the criminals’ rights movement”, without explaining in what way this was a change from the 1950s or early 1960s during which incarceration rates were equal or lower than those of the 1960s and 1970s.

My focus here, however is on a much more specific point – namely one of the very few pieces of evidence that Levitt musters to support his “high confidence” claim that “the strong economy of the 1990s” was not a major factor in the decline in crime.

On page 170 Levitt refers to a paper by Stephen Machin and Costas Meghir named “Crime and Economic Incentives”. Levitt says

Empirical estimates of the impact of macroeconomic variables on crime have been generally consistent across studies: Freeman (1995) surveys earlier research, and more recent studies include Machin and Meghir (2000), Gould, Weinberg and Mustard (1997), Donohue and Levitt (2001) and Raphael and Winter-Ebmer (2001). Controlling for other factors, almost all of these studies report a statistically significant but substantively small relationship between unemployment rates and property crime. A typical estimate would be that a one percentage point increase in the unemployment rate is associated with a one percent increase in property crime. […] Studies that have used other measures of macroeconomic performance like wages of low-income workers come to similar conclusions (Machin and Meghir, 2000; Gould, Weinberg and Mustard, 1997). Based on these estimates, the observed 2 percentage point decline in the U.S unemployment rate between 1991 and 2001 can explain an estimated 2 percent decline in property crime (out of an observed drop of almost 30 percent)[.]

Having read the description of the Machin and Meghir paper, it is surprising to find the following statement in the abstract of the paper: “We carry out a number of experiments with different wage measures, including a wage measure that accounts for the effects of changes in the composition of employment. These reinforce the picture of a strong impact of wages on crime.”

In the body of the paper, in the section “Empirical Findings” Machin and Meghir say (p. 18):

The marginal effect reported in the last row of [Table 1] reveals that an area with a 10 percent higher 25th percentile wage is predicted to have a 0.7 percentage point lower crime rate, which viewed against the actual value of the crime rate (about 8 percent, or 80 crimes per 1000 people, in the 1990s) is quite a high number.

If, following Levitt’s standard methodology, we assume the uncalculated risk of taking such estimates literally, then the 12% increase in the 20th percentile of the distribution of real income in the U.S. over the period 1991-2001 (from $18,693 to $21,046), should have resulted in over 10% reduction in the property crime – as high as the strongest effect in Levitt’s list (legalized abortion, Table 5, p. 184).

Far from being supportive of Levitt’s conclusion, the paper he cites as supportive is therefore actually presenting a diametrically opposing view. Levitt is thus grossly misrepresenting the paper. Behind his confident tone lies not only the weakness of his method, but also the breaking of the simplest rules scientific conduct. His respected status is a testimony to the weakness of the academic system in producing high quality scientific work.

Advertisement