Skip to content

Category: Observational studies

Is unit homogeneity a sufficient assumption for causal inference?

Is unit homogeneity a sufficient condition (assumption) for causal inference from observational data? Re-reading King, Keohane and Verba’s bible on research design [lovingly known to all exposed as KKV] I think they regard unit homogeneity and conditional independence as alternative assumptions for causal inference. For example: “we provide an overview here of what is required in terms of the two possible assumptions that enable us to get around the fundamental problem [of causal inference]” (p.91, emphasis mine). However, I don’t see how unit homogeneity on its own can rule out endogeneity (establish the direction of causality). In my understanding, endogeneity is automatically ruled out with conditional independence, but not with unit homogeneity (“Two units are homogeneous when the expected values of the dependent variables from each unit are the same when our explanatory variables takes on a particular value” [p.91]). Going back to Holland’s seminal article which provides the basis of KKV’s approach, we can confirm that unit homogeneity is listed as a sufficient condition for inference (p.948). But Holland divides variables into pre-exposure and post-exposure before he even gets to discuss any of the additional assumptions, so reverse causality is ruled out altogether. Hence, in Holland’s context unit homogeneity can indeed be regarded as sufficient, but in my opinion in KKV’s context unit homogeneity needs to be coupled with some condition (temporal precedence for example) to ascertain the causal direction when making inferences from data. The point is minor but can create confusion when presenting unit homogeneity and conditional independence side by side as alternative assumptions for inference.

Social science in the courtroom

Everyone who is interested in the sociology of science, causal inferences from observational data, employment gender discrimination, judicial sagas, or academic spats should read the latest issue of Sociological Methods & Research. The whole issue is devoted to the Wal-Mart Stores,Inc. v. Dukes et al. case – “the largest class-action employment discrimination suit in history”, with a focus on the uses of social science evidence in the courtroom.  The focal point of contestation is the report of Dr. Bielby – an expert for the plaintiff. In a nutshell, the report says that the gender bias in promotion decisions at Wal-Mart can be attributed to the lack of efforts to create a strong corporate culture and limit the discretion managers have in promotion decisions, which in turn allows for biased decisions. The evidence is mostly 1) a literature review that supports the causal links between corporate policies and corporate culture, corporate culture and individual behavior, discretion and biased individual behavior, and corporate policies and outcomes, and 2) description of the corporate policies and culture at Wal-Mart which points to a relatively weak policy towards gender discrimination and considerable discretion for managers in promotion decisions. Dr. Bielby describes the method as follows: “…look at distinctive features of the firm’s policies and practices and … evaluate them against what social scientific research shows to be factors that create and sustain bias and those that minimize bias” [the method is designated as “social framework analysis”]. What gives the case broader significance (apart from the fact that it directly concerns between half a million and a million and a half…

What makes a video go viral?

Internet Marketing expert Dr Brent Coker claims to have developed an algorithm that can predict which ad movies will go viral on YouTube. I don’t plan a career move to advertising but was nevertheless intrigued by the claim from a research methods & design perspective. Unfortunately, there is very little information available (yet?) and what information is available makes me a bit skeptical about the reliability of the conclusion. Still, Dr Coker’s approach might make for a nice discussion in the context of a Research Design course since it touches upon a question students can relate to, and raises various issues from operationalization to theory specification to theory testing. In short, according to Dr Coker, “there are four elements that need to be in place for a branded movie to become viral: (1) congruency, (2) emotive strength, (3) network-involvement ratio, and (4) paired meme synergy”. Congruency is the consistency of the video’s theme with brand knowledge. Disgust and fear, for example, imply powerful emotive strength. The network-involvement ratio refers to how relevant the message is to the seeded network. The last element ‘paired meme synergy’ means that certain memes are effective when paired with certain other memes. “For example, impromptu entertainment acts appeared to work when paired with ‘Eyes Surprise’. When paired with ‘bubblegum nostalgia’, the … pair doesn’t work. Anticipation works with Voyeur, but not on its own. And so forth.” As I said, there is not much information available on the research design, but from what I can gather, the predictive algorithm is based on an inductive approach: analyze movies that did go…

Foreign media exposure and democratization

This paper [ungated; longer version] has it all: a  design based on a ‘natural experiment’, recently declassified East German public opinion surveys, and a counterintuitive result – exposure to West German TV increased support for the the East German communist regime. Here is the abstract: In this case study of the impact of West German television on public support for the East German communist regime, we evaluate the conventional wisdom in the democratization literature that foreign mass media undermine authoritarian rule. We exploit formerly classified survey data and a natural experiment to identify the effect of foreign media exposure using instrumental variable estimators. Contrary to conventional wisdom, East Germans exposed to West German television were more satisfied with life in East Germany and more supportive of the East German regime. To explain this surprising finding, we show that East Germans used West German television primarily as a source of entertainment. Behavioral  data on regional patterns in exit visa applications and archival evidence on the reaction of the East German regime to the availability of West German television corroborate this result. The ‘randomization’ is based on the fact that the penetration of West German TV in East Germany was determined by topographical features. The area around Dresden is the main one which had no access so it serves to anchor the comparisons. The effect sizes reported in the empirical analysis are not great – the different models and estimators show a positive effect of exposure to West German TV in the range…

The deterrent effect of the death penalty

Does the death penalty lead to a lower number of homicides? A recent paper by Charles Manski and John Pepper argues that, on the basis of existing US data, we do not know. Both positive and negative effects of the application of the death penalty are consistent with the observed homicide rates in the US. The argument itself is not new (see for example this 2006 paper by John J. Donohue III and Justin Wolfers) but Manski and Pepper’s text is still very interesting and highly instructive. Manski and Pepper strip the problem to the core. Say we only have four observation points – the average yearly homicide rates for ’75 and ’77 in two sets of states that either did (A) or (B) did not reinstate the death penalty after the moratorium was lifted with the 1976 Gregg decision. So in 1975 both sets of states did not have a death penalty while in 1977 group A had reinstated it. I reworked the table with the rates into the figure below. The red dots show the homicide rates in the ‘death penalty’ states and the blue ones in the remaining ones. The authors show that on the basis of these four numbers, there are at least three point estimates of the effect of the death penalty that we can derive from the data depending on the assumptions that we are willing to make. First, we can assume that the selection of individual states into the two groups (‘death penalty’…