A small note on noise

There is a famous Tufte line about charts: that the ones we trust most are usually the ones designed best, not the ones that are most true. I think about it often.

Most of the time, when we squint at a time series and feel like we are seeing a pattern, we are seeing noise that happens to point in a direction. The brain hates randomness — it always finds an arrow. The discipline is to ask, before reading anything off the chart, what kind of squiggle pure chance would have produced on the same axes. Usually, it would have produced one a lot like the one in front of you.

This is the part of statistics that does not make for great Twitter content. It says: probably nothing happened. It says: wait. It says: yes, you found a five-sigma event in a dataset of fifty million possible comparisons, please sit down.

I am writing this down because I want to keep myself honest. Every post that I write about data, finance, or anything in between will have to pass a small private test: would I still believe this if the noise had been a little louder?

Most of the time, the answer will be no. Those posts will not be published.