Felix Salmon<\/a>]<\/p>\n <\/p>\n
The illusion of predictability: How regression statistics mislead experts<\/strong>
\nEmre Soyer& Robin M. Hogarth
\n<\/span><\/span><\/em>Abstract
\n<\/strong>Does the manner in which results are presented in empirical studies affect perceptions of the predictability of the outcomes? Noting the predominant role of linear regression analysis in empirical economics, we asked 257 academic economists to make probabilistic inferences given different presentations of the outputs of this statistical tool. Questions concerned the distribution of the dependent variable conditional on known values of the independent variable. Answers based on the presentation mode that is standard in the literature led to an illusion of predictability; outcomes were perceived to be more predictable than could be justified by the model. In particular, many respondents failed to take the error term into account. Adding graphs did not improve inferences. Paradoxically, when only <\/span><\/em><\/span><\/em>graphs were provided (i.e., no <\/span>regression statistics), respondents were more accurate. The implications of our study suggest, inter alia, the need to reconsider how to present empirical results and the possible provision of easy-to-use simulation tools that would enable readers of empirical papers to make accurate inferences.<\/p>\n","protected":false},"excerpt":{"rendered":"I have always considered scatterplots to be the best available device to show relationships between variables. But it must be even better to have the regression table and a full description of the results in addition, right? Not so fast: A new paper shows that\u00a0professional economists make largely correct inferences about data when looking at a scatterplot, but\u00a0get\u00a0confused when they are shown the details of the regressions next to the scatterplot, and totally mess it up when they are shown only the numbers without the plot! Wow! If you needed any more\u00a0persuasion that graphing your data and your results\u00a0are\u00a0more important than\u00a0those regression tables with zillions of numbers,\u00a0now you have it. P.S. The authors of this\u00a0research could have done a better job themselves in communicating visually their findings… [via Felix Salmon] The illusion of predictability: How regression statistics mislead experts Emre Soyer& Robin M. Hogarth Abstract Does the manner in which results are presented in empirical studies affect perceptions of the predictability of the outcomes? Noting the predominant role of linear regression analysis in empirical economics, we asked 257 academic economists to make probabilistic inferences given different presentations of the outputs of this statistical tool. Questions concerned the distribution of the dependent variable conditional on known values of the independent variable. Answers based on the presentation mode that is standard in the literature led to an illusion of predictability; outcomes were perceived to be more predictable than could be justified by the model. In particular, many respondents failed to take…<\/p>\n
Continue readingScatterplots vs. regression tables (Economics professors edition)<\/span><\/a><\/div>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false},"categories":[11],"tags":[162,208,272,350,351,479,512,555,579],"jetpack_featured_media_url":"","jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p7g3hj-80","jetpack-related-posts":[{"id":767,"url":"http:\/\/re-design.dimiter.eu\/?p=767","url_meta":{"origin":496,"position":0},"title":"Predicting movie ratings with IMDb data and R","date":"March 2, 2014","format":false,"excerpt":"It's Oscars season again so why not explore how predictable (my) movie tastes are. This has literally been a million dollar problem\u00a0and obviously I am not gonna solve it here, but it's fun and slightly educational to do some number crunching, so why not. Below, I will proceed from a\u2026","rel":"","context":"In "Data visualization"","img":{"alt_text":"","src":"https:\/\/i1.wp.com\/re-design.dimiter.eu\/wp-content\/uploads\/2014\/03\/figure9.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":220,"url":"http:\/\/re-design.dimiter.eu\/?p=220","url_meta":{"origin":496,"position":1},"title":"Slavery, ethnic diversity and economic development","date":"December 14, 2011","format":false,"excerpt":"What is the impact of the slave trades on economic progress in Africa? Are the modern African states which 'exported' a higher number of slaves more likely to be underdeveloped several centuries afterwards? Harvard economist Nathan Nunn addresses these questions in his chapter for the \"Natural experiments of history\" collection.\u2026","rel":"","context":"In "Development"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/re-design.dimiter.eu\/wp-content\/uploads\/2011\/12\/slave-trades.jpg?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1013,"url":"http:\/\/re-design.dimiter.eu\/?p=1013","url_meta":{"origin":496,"position":2},"title":"The political geography of human development","date":"November 12, 2018","format":false,"excerpt":"The research I did for the previous post on the inadequacy of the widely-used term 'Global South' led me to some surprising results about the political geography of development. Although the relationship between latitude and human development is not linear, distance from the equator turned out to have a rather\u2026","rel":"","context":"In "Data visualization"","img":{"alt_text":"","src":"https:\/\/i2.wp.com\/re-design.dimiter.eu\/wp-content\/uploads\/2018\/11\/f3_hdi_eq.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":260,"url":"http:\/\/re-design.dimiter.eu\/?p=260","url_meta":{"origin":496,"position":3},"title":"When 'just looking' beats regression","date":"January 30, 2012","format":false,"excerpt":"In a draft paper\u00a0currently under review I argue that the institutionalization of\u00a0a common EU\u00a0asylum policy has not led to a race to the bottom with respect to asylum applications, refugee status grants, and some other indicators. The graph below traces the number of asylum applications lodged in 29 European countries\u2026","rel":"","context":"In "Time series analysis"","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/re-design.dimiter.eu\/wp-content\/uploads\/2012\/01\/asylumapplications5.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":969,"url":"http:\/\/re-design.dimiter.eu\/?p=969","url_meta":{"origin":496,"position":4},"title":"The 'Global South' is a terrible term. Don't use it!","date":"November 6, 2018","format":false,"excerpt":"The Rise of the 'Global South' The 'Global South' and 'Global North' are increasingly popular terms used to categorize the countries of the world.\u00a0According to Wikipedia, the term 'Global South' originated in postcolonial studies, and was first used in 1969. The Google N-gram chart below shows the rise of the\u2026","rel":"","context":"In "Classification"","img":{"alt_text":"","src":"https:\/\/i2.wp.com\/re-design.dimiter.eu\/wp-content\/uploads\/2018\/11\/f2_hdi_eq.png?resize=350%2C200","width":350,"height":200},"classes":[]},{"id":1053,"url":"http:\/\/re-design.dimiter.eu\/?p=1053","url_meta":{"origin":496,"position":5},"title":"Modeling mortality","date":"January 20, 2021","format":false,"excerpt":"To grasp the true impact of COVID-19 on our societies, we need to know the effect of the pandemic on mortality. In other words, we need to know how many deaths can be attributed to the virus, directly and indirectly. It is already popular to visualize mortality in order to\u2026","rel":"","context":"In "Data visualization"","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=\/wp\/v2\/posts\/496"}],"collection":[{"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=496"}],"version-history":[{"count":0,"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=\/wp\/v2\/posts\/496\/revisions"}],"wp:attachment":[{"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=496"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=496"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/re-design.dimiter.eu\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=496"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}