The Outlier Bias: A Disproportionate Influence of Outliers in the Perception of Correlations

Abstract

While there is a preponderance of existing research examining how people interpret scatterplots and estimate correlations, there is a lack of research examining how the presence of outliers affects such processes. This study describes two experiments that start to fill this void and explain how people interpret outliers in scatterplots. In Experiment 1, participants were asked to estimate best-fit lines for scatterplots. Using the difference in estimates for nearly-identical scatterplots differentiated only by a single outlier, the correlational weight that participants assigned to individual outliers was calculated. Participants overweighted the effects of outliers on correlations, and held a positivity outlier bias in which they were especially influenced by outliers that strengthened the correlation. Experiment 2 limited the amount of perceptual information in order to determine whether this overweighting and positivity bias reflected a distortion in perception or working memory. The same effects were found across the board, implying that this phenomenon at least partly originates in working memory. Finally, the micro and macro level implications of these findings are discussed, and a new, proportionally representative scatterplot design is proposed to mitigate the effects of the outlier bias in data visualization.