Does the death penalty lead to a lower number of homicides? A recent paper by Charles Manski and John Pepper argues that, on the basis of existing US data, we do not know. Both positive and negative effects of the application of the death penalty are consistent with the observed homicide rates in the US. The argument itself is not new (see for example this 2006 paper by John J. Donohue III and Justin Wolfers) but Manski and Pepper’s text is still very interesting and highly instructive.

Manski and Pepper strip the problem to the core. Say we only have four observation points – the average yearly homicide rates for ’75 and ’77 in two sets of states that either did (A) or (B) did not reinstate the death penalty after the moratorium was lifted with the 1976 Gregg decision. So in 1975 both sets of states did not have a death penalty while in 1977 group A had reinstated it. I reworked the table with the rates into the figure below. The red dots show the homicide rates in the ‘death penalty’ states and the blue ones in the remaining ones.

The authors show that on the basis of these four numbers, there are at least three point estimates of the effect of the death penalty that we can derive from the data depending on the assumptions that we are willing to make.

First, we can assume that the selection of individual states into the two groups (‘death penalty’ or not) is independent of the potential outcomes (as if the assignment is random). In that case we can just compare the *contemporaneous* rates of the two groups in 1977 and we will conclude that the effect of the death penalty is +2.8 (meaning that the death penalty increases the homicide rate).

Second, we can assume that the mean ‘treatment’ (having the death penalty) response is the same for the treated (A) and the untreated (B) groups. In that case we can look at the* before/after* change in the homicide rates for group A between 1975 and 1977 and we will conclude that the effect is – 0.6 (meaning that the death penalty decrease the homicide rate).

Third, we can assume the the ‘treatment’ response is linear and homogeneous across states and dates. In that case we will arrive at the *difference-in-differences* estimate which compares the change between 1975 and 1977 in group A with the change between 1975 and 1977 in group B which in the particular case will lead to the conclusion that the effect of the death penalty is +0.5 (in the ‘treated’ group A the change is a reduction of 0.6 but in the ‘untreated’ group B the reduction is greater at 1.1).

Not only are these three estimates quite different, but each of them is based on easily challenged assumptions. The paper proceeds to present an alternative ‘partial identification’ approach which relies on weaker assumptions but this comes at a price – we can only identify *bounds* on the potential effect of the death penalty rather than precise numbers. The authors examine a number of alternatives, but in all cases the conclusion is that the possible deterrent effect of the death penalty is contained within large intervals which inevitably contain zero. In short, under weaker assumptions we cannot even say whether the death penalty has a positive or a negative effect on homicide rates.

The paper does a remarkable job in showing clearly how the seeming precision of point estimates is only possibly after the researcher makes a series of questionable assumptions. I am also convinced that in many policy settings having a *reliable interval* of possible values for the effect of a treatment (policy) is more useful than having a seemingly precise but unreliable and ultimately invalid *point estimate*. I know too little about the exact method applied in the paper to arrive at these intervals (this 2007 book by Manski seems like a good introduction) to assess the details but the approach seems more than reasonable.

One of the best things about the paper is that it deliberately sidelines statistical issues in order to focus squarely on the problem of inference. The problems of the uncertainty of point estimates, correct standard errors, etc. come *on top *of the general problem of inference. In many presentations the statistical issues are conflated with the more, uhm, fundamental problem of deriving a valid statement about the effect of a treatment (policy, institution, etc.) on the basis of observational data.

Finally, it is worth drawing some policy implications from the fact that scientific research cannot provide an answer to the question about the potential deterrent effect of the death penalty. I guess the current message ‘*it could be negative or positive, but in any case it is likely to be small’* implies that deterrence should not play a major role in the debates about the future of the death penalty. Different arguments, and different frames, should be the basis of discussion and decision since we simply don’t know whether a deterrent effect exists. In fact, in their recent book Frank R. Baumgartner, Suzanna De Boef, and Amber Boydstun leave only a very minor role (if at all) for the deterrence frame as a driver of death penalty policy change. Instead, they argue that it is the ‘discovery of innocence’ that is responsible for the ongoing transformation of the policy in the US, but their book and their approach are worth a separate post.