RE-DESIGN

Natural experiments of history? Not really, but still a fine book

Published by demetriodor on December 11, 2011

Natural experiments are a fine (and fun) way to study questions where the researcher doesn’t have control over the assignment of cases. But the label ‘natural experiment’ can get abused – not all comparisons are ‘natural experiments’. Nature needs to intervene into the assignment of cases in a way that can be credibly regarded as random in order to approximate the experimental method (e.g. here). Jared Diamond and Paul Robinson have collected seven essays in a book entitled “Natural Experiments of History”. But from the seven studies, only one or two might be regarded as a true ‘natural experiment’ – the rest are just more or less systematic comparisons. It is still a fine book – I found all seven essays interesting and stimulating. But they are not natural experiments; in fact, Diamond and Robinson themselves seem to retract from the label in the concluding chapter of the book. For example, in his chapter Patrick Kirch studies Polynesian cultural evolution. The three islands of Mangaia, Marquesas, and Hawai”i end up with quite different social and political institutions despite being populated by the same people. To his credit, Kirch uses ‘controlled comparisons’ instead of ‘natural experiments’ to describe his approach. But does the fact that the three islands were settled by people coming from the same homeland in Western Polynesia allow us to ‘control’ for the pre-settlement characteristics of the people who inhabited Mangaia, Marquesas, and Hawai’i? The explorers leaving in search of new homes are seldom a representative sample from…

Is unit homogeneity a sufficient assumption for causal inference?

Published by demetriodor on December 6, 2011

Is unit homogeneity a sufficient condition (assumption) for causal inference from observational data? Re-reading King, Keohane and Verba’s bible on research design [lovingly known to all exposed as KKV] I think they regard unit homogeneity and conditional independence as alternative assumptions for causal inference. For example: “we provide an overview here of what is required in terms of the two possible assumptions that enable us to get around the fundamental problem [of causal inference]” (p.91, emphasis mine). However, I don’t see how unit homogeneity on its own can rule out endogeneity (establish the direction of causality). In my understanding, endogeneity is automatically ruled out with conditional independence, but not with unit homogeneity (“Two units are homogeneous when the expected values of the dependent variables from each unit are the same when our explanatory variables takes on a particular value” [p.91]). Going back to Holland’s seminal article which provides the basis of KKV’s approach, we can confirm that unit homogeneity is listed as a sufficient condition for inference (p.948). But Holland divides variables into pre-exposure and post-exposure before he even gets to discuss any of the additional assumptions, so reverse causality is ruled out altogether. Hence, in Holland’s context unit homogeneity can indeed be regarded as sufficient, but in my opinion in KKV’s context unit homogeneity needs to be coupled with some condition (temporal precedence for example) to ascertain the causal direction when making inferences from data. The point is minor but can create confusion when presenting unit homogeneity and conditional independence side by side as alternative assumptions for inference.

Hyperlinks

Published by demetriodor on December 5, 2011

Medical marijuana laws decrease road fatalities I wonder what’s really driving this unlikely association Beauty not enough to make a picture memorable Did they try sex and gore? Which Nations Conform Most? Interesting, but how much variation among citizens of one country compared to the differences between countries? Out of Arabia? A fascinating hypothesis about early human migration

Emigrants vs. Settlers

Published by demetriodor on December 4, 2011

In his contribution to ‘Natural Experiments in History‘ James Belich argues that shifting attitudes towards emigration in Britain and the US were essential for the settler explosions in the American West, Australia, Canada, New Zealand and South Africa. Belich puts the shift in attitudes between 1810 and 1820 and illustrates the transformation with the contest between the use of ’emigrant’ and ‘settler’ on the pages of the Times of London. Always on the lookout for potential application of the awesome power of Google N-gram, I checked whether the shift of attitudes and vocabulary is visible in the larger body of English-language literature indexed by Google N-gram as well. Here is the graph: ‘Settler’ gets more popular than ’emigrant’ indeed! But the shift occurs a bit later with an initial catch-up around 1930 and the ultimate win of ‘settler’ around 1970. Interestingly, in the corpus of British books, ‘settler’ never surpasses ’emigrant’ in popularity, while in American books the two terms are practically even between 1830 and 1865 when ‘settler’ overtakes ’emigrant’ for good. Actually, it is ‘pioneer’ that rises in popularity beyond ’emigrant’ around 1810 and then surpasses both ’emigrant’ and ‘settler’ after 1845: Overall, Belich’s transformation in attitudes and vocabulary towards emigration seems reflected in literature, although the shift occurs later, and is much stronger for American English.

Social science in the courtroom

Published by demetriodor on December 2, 2011

Everyone who is interested in the sociology of science, causal inferences from observational data, employment gender discrimination, judicial sagas, or academic spats should read the latest issue of Sociological Methods & Research. The whole issue is devoted to the Wal-Mart Stores,Inc. v. Dukes et al. case – “the largest class-action employment discrimination suit in history”, with a focus on the uses of social science evidence in the courtroom. The focal point of contestation is the report of Dr. Bielby – an expert for the plaintiff. In a nutshell, the report says that the gender bias in promotion decisions at Wal-Mart can be attributed to the lack of efforts to create a strong corporate culture and limit the discretion managers have in promotion decisions, which in turn allows for biased decisions. The evidence is mostly 1) a literature review that supports the causal links between corporate policies and corporate culture, corporate culture and individual behavior, discretion and biased individual behavior, and corporate policies and outcomes, and 2) description of the corporate policies and culture at Wal-Mart which points to a relatively weak policy towards gender discrimination and considerable discretion for managers in promotion decisions. Dr. Bielby describes the method as follows: “…look at distinctive features of the firm’s policies and practices and … evaluate them against what social scientific research shows to be factors that create and sustain bias and those that minimize bias” [the method is designated as “social framework analysis”]. What gives the case broader significance (apart from the fact that it directly concerns between half a million and a million and a half…

The best abstract ever

Published by demetriodor on December 1, 2011

The best abstract of an academic paper ever? Probably yes. [via Boing Boing] P.S. Now that the problem of writing the perfect abstract is solved, you might wanna check how to write the perfect title.

RE-DESIGN Posts