What makes a video go viral?

Internet Marketing expert Dr Brent Coker claims to have developed an algorithm that can predict which ad movies will go viral on YouTube. I don’t plan a career move to advertising but was nevertheless intrigued by the claim from a research methods & design perspective. Unfortunately, there is very little information available (yet?) and what information is available makes me a bit skeptical about the reliability of the conclusion. Still, Dr Coker’s approach might make for a nice discussion in the context of a Research Design course since it touches upon a question students can relate to, and raises various issues from operationalization to theory specification to theory testing.

In short, according to Dr Coker, “there are four elements that need to be in place for a branded movie to become viral: (1) congruency, (2) emotive strength, (3) network-involvement ratio, and (4) paired meme synergy”. Congruency is the consistency of the video’s theme with brand knowledge. Disgust and fear, for example, imply powerful emotive strength. The network-involvement ratio refers to how relevant the message is to the seeded network. The last element ‘paired meme synergy’ means that certain memes are effective when paired with certain other memes. “For example, impromptu entertainment acts appeared to work when paired with ‘Eyes Surprise’. When paired with ‘bubblegum nostalgia’, the … pair doesn’t work. Anticipation works with Voyeur, but not on its own. And so forth.”

As I said, there is not much information available on the research design, but from what I can gather, the predictive algorithm is based on an inductive approach: analyze movies that did go viral and see what their characteristics are. Such an approach would be OK to generate ideas, but one should be careful overselling the inductively-identified “solution” as a predictive algorithm which has been properly tested. An obvious next step would be to see whether the “solution” predicts outside the sample it was derived from, and maybe Dr Coker is working on that stage now. I wonder, however, whether the rather flexible definitions of some of the predictive elements make a testing of the approach feasible even in principle. It seem hard to identify the ‘network-involvement ration’, for example, prior to observing the outcome. The meme-pairing idea is interesting, but again: if there is no clear idea why certain memes should go together, there is a high risk of the analysis just playing catch-up with the data.

For example, how would you score this awesome recent viral video (my take would be Disruption Destruction + Performance + Skill Bill + Simulation Trigger, for the list of possible memes see here)?: 

P.S. On a somwhat related note: The Atlantic has a feature on the rise of big data which says that Google runs ”100-200 experiments on any given day, as they test new products and services, new algorithms and alternative designs”.

Foreign media exposure and democratization

This paper [ungated; longer version] has it all: a  design based on a ‘natural experiment’, recently declassified East German public opinion surveys, and a counterintuitive result – exposure to West German TV increased support for the the East German communist regime. Here is the abstract:

In this case study of the impact of West German television on public support for the East German communist regime, we evaluate the conventional wisdom in the democratization literature that foreign mass media undermine authoritarian rule. We exploit formerly classified survey data and a natural experiment to identify the effect of foreign media exposure using instrumental variable estimators. Contrary to conventional wisdom, East Germans exposed to West German television were more satisfied with life in East Germany and more supportive of the East German regime. To explain this surprising finding, we show that East Germans used West German television primarily as a source of entertainment. Behavioral  data on regional patterns in exit visa applications and archival evidence on the reaction of the East German regime to the availability of West German television corroborate this result.

The ‘randomization’ is based on the fact that the penetration of West German TV in East Germany was determined by topographical features. The area around Dresden is the main one which had no access so it serves to anchor the comparisons.

The effect sizes reported in the empirical analysis are not great – the different models and estimators show a positive effect of exposure to West German TV in the range of  0.15 to 0.26  on a 4-point scale of support for the East German communist regime (survey data gathered in 1988/1989 among young people). But even if we take the empirical findings to imply that exposure to West German TV definitely did not decrease support for the communist regime, the conclusions are important and significant. The mechanism proposed to explain the counterintuitive findings is that watching West German TV actually made life more bearable and provided an escape from the dull communist reality.

One of the authors has another piece using a very similar setup based again on the natural experiment of West German TV penetration in which he argues that West German TV exposure did not affect participation in the protest participation in the East German Revolution in 1989.

Great staff!

The deterrent effect of the death penalty

Does the death penalty lead to a lower number of homicides? A recent paper by Charles Manski and John Pepper argues that, on the basis of existing US data, we do not know. Both positive and negative effects of the application of the death penalty are consistent with the observed homicide rates in the US. The argument itself is not new (see for example this 2006 paper by John J. Donohue III and Justin Wolfers) but Manski and Pepper’s text is still very interesting and highly instructive.

Manski and Pepper strip the problem to the core. Say we only have four observation points – the average yearly homicide rates for ’75 and ’77 in two sets of states that either did (A) or (B) did not reinstate the death penalty after the moratorium was lifted with the 1976 Gregg decision. So in 1975 both sets of states did not have a death penalty while in 1977 group A had reinstated it. I reworked the table with the rates into the figure below. The red dots show the homicide rates in the ‘death penalty’ states and the blue ones in the remaining ones.

The authors show that on the basis of these four numbers, there are at least three point estimates of the effect of the death penalty that we can derive from the data depending on the assumptions that we are willing to make.

First, we can assume that the selection of individual states into the two groups (‘death penalty’ or not) is independent of the potential outcomes (as if the assignment is random). In that case we can just compare the contemporaneous rates of the two groups in 1977 and we will conclude that the effect of the death penalty is +2.8 (meaning that the death penalty increases the homicide rate).

Second, we can assume that the mean ‘treatment’  (having the death penalty) response is the same for the treated (A) and the untreated (B) groups. In that case we can look at the before/after change in the homicide rates for group A between 1975 and 1977 and we will conclude that the effect is – 0.6 (meaning that the death penalty decrease the homicide rate).

Third, we can assume the the ‘treatment’ response is linear and homogeneous across states and dates. In that case we will arrive at the difference-in-differences estimate which compares the change between 1975 and 1977 in group A with the change between 1975 and 1977 in group B which in the particular case will lead to the conclusion that the effect of the death penalty is +0.5 (in the ‘treated’ group A the change is a reduction of 0.6 but in the ‘untreated’ group B the reduction is greater at 1.1).

Not only are these three estimates quite different, but each of them is based on easily challenged assumptions. The paper proceeds to present an alternative ‘partial identification’ approach which relies on weaker assumptions but this comes at a price – we can only identify bounds on the potential effect of the death penalty rather than precise numbers. The authors examine a number of alternatives, but  in all cases the conclusion is that the possible deterrent effect of the death penalty is contained within large intervals which inevitably contain zero. In short, under weaker assumptions we cannot even say whether the death penalty has a positive or a negative effect on homicide rates.

The paper does a remarkable job in showing clearly how  the seeming precision of point estimates is only possibly after the researcher makes a series of questionable assumptions. I am also convinced that in many policy settings having a reliable interval of possible values for the effect of a treatment (policy) is more useful than having a seemingly precise but unreliable and ultimately invalid point estimate. I know too little about the exact method applied in the paper to arrive at these intervals (this 2007 book by Manski seems like a good introduction) to assess the details but the approach seems more than reasonable.

One of the best things about the paper is that it deliberately sidelines statistical issues in order to focus squarely on the problem of inference. The problems of the uncertainty of point estimates, correct standard errors, etc. come on top of the general problem of inference. In many presentations the statistical issues are conflated with the more, uhm, fundamental problem of deriving a valid statement about the effect of a treatment (policy, institution, etc.) on the basis of observational data.

Finally, it is worth drawing some policy implications from the fact that scientific research cannot provide an answer to the question about the potential deterrent effect of the death penalty. I guess the current message  ‘it could be negative or positive, but in any case it is likely to be small’ implies that deterrence should not play a major role in the debates about the future of the death penalty. Different arguments, and different frames, should be the basis of discussion and decision since we simply don’t know whether a deterrent effect exists. In fact, in their recent book Frank R. Baumgartner, Suzanna De Boef, and Amber Boydstun leave only a very minor role (if at all) for the deterrence frame as a driver of death penalty policy change. Instead, they argue that it is the ‘discovery of innocence’ that is responsible for the ongoing transformation of the policy in the US, but their book and their approach are worth a separate post.