Is what we see really there? - A discussion of visual inference
Often times, when we are ‘looking’ at data in plots, we find a set of interesting points or identify a pattern that is interesting. What is the significance of a finding like that? Visual inference gives us protocols that help us to quantify the strength of a visual finding in a framework similar to to confirmatory statistical hypothesis testing. Visual inference helps analysts determine if structure is real or spurious. In this framework, plots take on the role of test statistics, and human cognition the role of statistical tests. Statistical significance of visual ``discoveries’’ is measured by having the human viewer compare the plot of the real dataset with collections of plots of simulated or null datasets. We will discuss some applications of visual inference and aspects of visual tests, such as determining the power of competing designs.