{"id":210,"date":"2011-12-11T15:33:11","date_gmt":"2011-12-11T15:33:11","guid":{"rendered":"http:\/\/rulesofreason.wordpress.com\/?p=210"},"modified":"2011-12-11T15:33:11","modified_gmt":"2011-12-11T15:33:11","slug":"natural-experiments-of-history-not-really-but-still-a-fine-book","status":"publish","type":"post","link":"http:\/\/re-design.dimiter.eu\/?p=210","title":{"rendered":"Natural experiments of history? Not really, but still a fine book"},"content":{"rendered":"

Natural experiments are a fine (and fun) way to study questions where the researcher doesn’t have control over the assignment of cases. But the label ‘natural experiment’ can get abused – not all comparisons are ‘natural experiments’. Nature needs to intervene into the assignment of cases in a way that can be credibly regarded as random in order to approximate the experimental method (e.g. here<\/a>).<\/p>\n

Jared Diamond and Paul Robinson have collected seven essays in a book<\/a> entitled “Natural Experiments of History”. But from the seven studies, only one or two might be regarded as a true ‘natural experiment’ – the rest are just more or less systematic comparisons. It is still a fine book – I found all seven essays interesting and stimulating. But they are not natural experiments; in fact, Diamond and Robinson themselves seem to retract from the label in the concluding chapter of the book.<\/p>\n

For example, in his chapter Patrick Kirch studies Polynesian cultural evolution. The three islands of Mangaia, Marquesas, and Hawai”i end up with quite different social and political institutions despite being populated by the same people. To his credit, Kirch uses ‘controlled comparisons’ instead of ‘natural experiments’ to describe his approach. But does the fact that the three islands were settled by people coming from the same homeland in Western Polynesia allow us to ‘control’ for the pre-settlement characteristics of the people who inhabited Mangaia, Marquesas, and Hawai’i? The explorers leaving in search of new homes are seldom a representative sample from the original population. They could be the most adventurous, skilled, risk-taking individuals who depart in search of better fortunes. Or they could be the most aggressive, rebellious, conflict-seeking and uncooperative individuals who get expelled from their original island. In any case, they are likely to be different<\/em> than the median individual of the source population. As a result, we cannot really rule out that Mangaia, Marquesas, and Hawai’i develop different and progressively more complex political organizations because of differences between the original inhabitants rather that the ecological variables discussed by Kirch. My aim here is not to criticize the substance of Kirch’s research – I actually find his conclusions rather compelling, but to illustrate how self-selection can undermine the comparisons when the assignment of people to islands is not entirely random.<\/p>\n

In fact, from the seven chapters the comparison of different Polynesian islands is one of the more persuasive ones from a methodological point of view. Only Banhijit Banerjee and Lakshmi Iyer’s essay on the impact of land tenure institutions on public goods provision in India comes closer to the use of a selection mechanism that can be regarded as a natural experiment. But even there, if the British governors who opted for village-based ownership were also more likely to engage in other ‘progressive’ projects (thus establishing the initial infrastructure for public good provision), then the differences we observe today between Indian regions might not be attributable to the different land tenure regimes during colonial rule.<\/p>\n

Even if not based on true natural experiments, the seven essays collected in this little book have a lot to offer. For historians especially, some of the methodological approaches might be eye-opening and hopefully lead to more rigorous search for patterns in history. \u00a0If the price for bringing methodological insight to a bigger and more diverse audience is a catchy title that is not entirely fair to the content of the book<\/a>, so be it.<\/p>\n","protected":false},"excerpt":{"rendered":"

Natural experiments are a fine (and fun) way to study questions where the researcher doesn’t have control over the assignment of cases. But the label ‘natural experiment’ can get abused – not all comparisons are ‘natural experiments’. Nature needs to intervene into the assignment of cases in a way that can be credibly regarded as random in order to approximate the experimental method (e.g. here). Jared Diamond and Paul Robinson have collected seven essays in a book entitled “Natural Experiments of History”. But from the seven studies, only one or two might be regarded as a true ‘natural experiment’ – the rest are just more or less systematic comparisons. It is still a fine book – I found all seven essays interesting and stimulating. But they are not natural experiments; in fact, Diamond and Robinson themselves seem to retract from the label in the concluding chapter of the book. For example, in his chapter Patrick Kirch studies Polynesian cultural evolution. The three islands of Mangaia, Marquesas, and Hawai”i end up with quite different social and political institutions despite being populated by the same people. To his credit, Kirch uses ‘controlled comparisons’ instead of ‘natural experiments’ to describe his approach. But does the fact that the three islands were settled by people coming from the same homeland in Western Polynesia allow us to ‘control’ for the pre-settlement characteristics of the people who inhabited Mangaia, Marquesas, and Hawai’i? The explorers leaving in search of new homes are seldom a representative sample from…<\/p>\n

Continue readingNatural experiments of history? Not really, but still a fine book<\/span><\/a><\/div>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false},"categories":[19,29],"tags":[115,316,356,372,432,433,466,468,506,507],"jetpack_featured_media_url":"","jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7g3hj-3o","jetpack-related-posts":[{"id":195,"url":"http:\/\/re-design.dimiter.eu\/?p=195","url_meta":{"origin":210,"position":0},"title":"Emigrants vs. Settlers","date":"December 4, 2011","format":false,"excerpt":"In his contribution to 'Natural Experiments in History' James Belich \u00a0argues that shifting attitudes towards emigration in Britain and the US were essential for the settler explosions in the American West, Australia, Canada, New Zealand and South Africa. Belich puts the shift in attitudes between 1810 and 1820 and illustrates\u2026","rel":"","context":"In "Bibliometry"","img":{"alt_text":"","src":"https:\/\/i2.wp.com\/re-design.dimiter.eu\/wp-content\/uploads\/2011\/12\/chart2.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":220,"url":"http:\/\/re-design.dimiter.eu\/?p=220","url_meta":{"origin":210,"position":1},"title":"Slavery, ethnic diversity and economic development","date":"December 14, 2011","format":false,"excerpt":"What is the impact of the slave trades on economic progress in Africa? Are the modern African states which 'exported' a higher number of slaves more likely to be underdeveloped several centuries afterwards? Harvard economist Nathan Nunn addresses these questions in his chapter for the \"Natural experiments of history\" collection.\u2026","rel":"","context":"In "Development"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/re-design.dimiter.eu\/wp-content\/uploads\/2011\/12\/slave-trades.jpg?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":158,"url":"http:\/\/re-design.dimiter.eu\/?p=158","url_meta":{"origin":210,"position":2},"title":"Foreign media exposure and democratization","date":"November 9, 2011","format":false,"excerpt":"This paper [ungated; longer version] has it all: a \u00a0design based on a 'natural experiment', recently declassified East German public opinion surveys, and a counterintuitive result - exposure to West German TV\u00a0increased support for the the East German communist regime. Here is the abstract: In this case study of the\u2026","rel":"","context":"In "Natural experiments"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":544,"url":"http:\/\/re-design.dimiter.eu\/?p=544","url_meta":{"origin":210,"position":3},"title":"Ethnic job discrimination in the Netherlands","date":"October 29, 2012","format":false,"excerpt":"I have more than one reason to care about job discrimination based on ethnicity in the Netherlands. A new study shows that there is plenty to worry about. In short, the researchers sent identical job applications varying only the name - Dutch vs. ethnic (Antillean, Surinamese, Turkish, Moroccan). The 'Dutch'\u2026","rel":"","context":"In "Field experiments"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":685,"url":"http:\/\/re-design.dimiter.eu\/?p=685","url_meta":{"origin":210,"position":4},"title":"A lesson about discrimination you can never forget","date":"February 15, 2013","format":false,"excerpt":"What happens when you tell a bunch of kids that the blue-eyed children are better than the browned-eyed? Watch this video to find out. But be warned - this is one of the most disturbing things I have seen in a long while. It is not that one is unaware\u2026","rel":"","context":"In "Anthropology"","img":{"alt_text":"","src":"https:\/\/i2.wp.com\/img.youtube.com\/vi\/8bWlTZZN3DY\/0.jpg?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":486,"url":"http:\/\/re-design.dimiter.eu\/?p=486","url_meta":{"origin":210,"position":5},"title":"Facebook does randomized experiments to study social interactions","date":"June 15, 2012","format":false,"excerpt":"Facebook has a Data Science Team. And here is what they do: Eytan Bakshy [...] wanted to learn whether our actions on Facebook are mainly influenced by those of our close friends, who are likely to have similar tastes. [...] So he messed with how Facebook operated for a quarter\u2026","rel":"","context":"In "Advertising research"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=\/wp\/v2\/posts\/210"}],"collection":[{"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=210"}],"version-history":[{"count":0,"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=\/wp\/v2\/posts\/210\/revisions"}],"wp:attachment":[{"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=210"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=210"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=210"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}