Correlation Surprises and Why Spreadsheets Lie Quietly

Spreadsheets have a calm authority. A correlation of 0.23 looks like a fact. In a committee meeting it can end debate. I have learned to treat that calm as a warning. Correlation estimates are compressed history. They are not obligations the future signed.

The first quiet lie is sample choice. Five years of data can miss the regime you are about to enter. Twenty years can mix eras with different policy backdrops until the average means little. High-frequency estimates can look stable while missing slow-burn structural breaks. None of this makes estimation wrong; it makes single-number certainty wrong.

The second lie is symmetry. People hear “low correlation” and imagine protection. Low average correlation can coexist with high correlation in the left tail. That is when you needed diversification most. Looking only at full-sample coefficients is how portfolios discover, too late, that hedges were fair-weather friends.

The third lie is dimensionality theatre. Expanding from ten assets to two hundred can improve in-sample optimisation while adding estimation error and operational complexity. Optimisers are especially eager to exploit noise. Without constraints, shrinkage, or robust methods, “optimal” weights can be a portrait of your dataset’s quirks.

Digital assets intensify the problem. Short histories, evolving market structure, and episodes of extreme co-movement with risk appetite make pairwise statistics brittle. A crypto sleeve may diversify idiosyncratic equity stories for months and then trade like a high-beta tech proxy in a liquidity shock. Both observations can be true. Your process must allow both.

What do I do instead of worshipping the matrix? I use correlations as one input among scenarios. I compare calm-period versus stress-period relationships. I ask whether an economic channel explains the dependence—shared cash-flow discounting, shared funding, shared investor base. If the only link is “they both went up,” I am suspicious of relying on their independence.

I also prefer simple robustness checks over ornate precision. Cap weights. Impose minimum diversification. Penalise turnover. Re-estimate under rolling windows and see whether recommended holdings thrash. If a tiny data tweak rearranges the whole portfolio, the model is too confident for the information it has.

Human judgment still belongs in the loop—not as mystic vibes, but as a veto against nonsense. If a model wants a huge short in an instrument you cannot borrow reliably, or a hedge that only works if a market stays open, the spreadsheet is incomplete. Completeness includes market plumbing.

None of this argues against quantitative tools. It argues against confusing formatted outputs with understanding. The best analysts I know can explain a risk position without opening a file, then open the file to check they are not fooling themselves. That order matters.

Spreadsheets lie quietly because they rarely raise their voice. Your job is to keep asking what they omitted until the silence gets uncomfortable—and informative.

Missing data and stale marks create false calm. Illiquid sleeves that rarely update can look uncorrelated simply because they are unobserved. That is accounting diversification, not economic diversification. Force honest marks or haircut the comfort.

Currency overlays and parent-currency reporting can also invent correlations. Two assets may diversify locally and still co-move after translation. Know which currency your risk is actually measured in.

Machine-learning covariance estimates are not exempt from these problems; they can be worse if they fit noise with confidence intervals that look scientific. Complexity is not robustness. Sometimes a shrinkage estimator and a sceptical human beat a glittering model.

When presenting to committees, I try to show ranges and stress plots, not a single glamorous heatmap. Heatmaps persuade. Ranges inform. Choose informing.

If your diversification story collapses whenever you change the start date by a year, you do not have a story. You have a coincidence with formatting.

Liquidity betas hide inside correlation stories. Assets may show moderate price correlation and still share a common need for the same scarce financing in stress. Funding risk is a correlation channel that price history understates until the day dealers step back.

I keep a short list of “break glass” assumptions—what would have to be true for my diversification claim to fail. Revisiting that list beats polishing a denser matrix nobody will read under pressure.

Committees should reward analysts who surface uncomfortable dependence, not only those who present tidy heatmaps. Incentives write the real model.

Teach juniors to break a matrix by hand on a whiteboard before trusting software. If they cannot sketch the economic story of the largest off-diagonal numbers, they are not ready to optimise anything.