Small samples & the hot hand fallacy

Extreme outcomes

In his ground-breaking book Thinking Fast and Slow psychologist Daniel Kahneman invited the reader to consider why some small, rural, Republican towns in America’s South, West, and Midwest might’ve recorded the highest rates of kidney cancer (perhaps due to the clean-living rural lifestyle, lack of pollution, and access to fresh food?).

Confusingly some small, rural, Republican towns in the South, West, and Midwest had also recorded the lowest rates of kidney cancer (maybe due to the challenges of living in remote regions and related poverty?).

The suggested explanations are nonsensical, of course.

A simpler answer is that small populations are statistically more likely to produce extreme outcomes.

Poor intuitive statisticians

We see this phenomenon often enough in real estate magazines, with remote suburbs that you’ve never heard of featuring prominently in the ‘best performing suburbs’ lists.

It all sounds impressive until you work out that the sample size is half a dozen transactions, and the same suburbs will likely feature in the worst performers soon enough.

While such examples can seem very obvious, Kahneman’s research found that most of us are poor intuitive statisticians, and consequently nowhere near as rational as we think we are.

False prophets

This law (or should that be fallacy?) of small numbers extends far beyond rates of kidney cancer or real estate trends, to sports stars experiencing apparent ‘hot streaks’, or random results from roulette wheels.

Far too often we place undue emphasis or reliance on the returns of small samples.

Occasionally even a sample of one is enough!

Financial gurus can make their name from just one good year or open-ended prediction, though human nature dictates that they’ll downplay or downright ignore their poor forecasts.

It’s said that forecasts tell you more about the forecaster than they do the future, and we can all tend to rate commentary that confirms our underlying beliefs higher than predictions that contradict them.

By way of a contemporary example of a time-bound prediction, look no further than the targets for where the S&P 500 would finish in 2018.

A few face-burning (& face-saving) end-of-year rallies dragged the index back up to 2,500, yet still none of the predictions was even remotely close to the mark.

The less said about the above forecasts the better, except that a cyclical investment approach based on fundamentals might fare better!

Be wary of small samples and the hot hand fallacy.