I have always considered scatterplots to be the best available device to show relationships between variables. But it must be even better to have the regression table and a full description of the results in addition, right? Not so fast: A new paper shows that professional economists make largely correct inferences about data when looking at a scatterplot, but get confused when they are shown the details of the regressions next to the scatterplot, and totally mess it up when they are shown only the numbers without the plot! Wow! If you needed any more persuasion that graphing your data and your results are more important than those regression tables with zillions of numbers, now you have it. P.S. The authors of this research could have done a better job themselves in communicating visually their findings… [via Felix Salmon] The illusion of predictability: How regression statistics mislead experts Emre Soyer& Robin M. Hogarth Abstract Does the manner in which results are presented in empirical studies affect perceptions of the predictability of the outcomes? Noting the predominant role of linear regression analysis in empirical economics, we asked 257 academic economists to make probabilistic inferences given different presentations of the outputs of this statistical tool. Questions concerned the distribution of the dependent variable conditional on known values of the independent variable. Answers based on the presentation mode that is standard in the literature led to an illusion of predictability; outcomes were perceived to be more predictable than could be justified by the model. In particular, many respondents failed to take…
Research Design Matters