Social science research is supposed to work like this: 1) You want to explain a certain case or a class of phenomena; 2) You develop a theory and derive a set of hypotheses; 3) You test the hypotheses with data; 4) You conclude about the plausibility of the theory; 5) You write a paper with a structure (research question, theory, empirical analysis, conclusions) that mirrors the steps above. But in practice, social science research often works like this: 1) You want to explain a certain case or a class of phenomena; 2) You test a number hypotheses with data; 3) You pick the hypotheses that matched the data best and combine them in a theory; 4) You conclude that this theory is plausible and relevant; 5) You write a paper with a structure (research question, theory, empirical analysis, conclusions) that does not reflect the steps above. In short, an inductive quest for a plausible explanation is masked and reported as deductive theory-testing. This fallacy is both well-known and rather common (at least in the fields of political science and public administration). And, in my experience, it turns out to be tacitly supported by the policies of some journals and reviewers. For one of my previous research projects, I studied the relationship between public support and policy output in the EU. Since the state of the economy can influence both, I included levels of unemployment as a potential omitted variable in the empirical analysis. It turned out that lagged unemployment is positively related to the volume of policy output. In the paper, I mentioned this result in passing…
Research Design Matters