Model Risk Is Still Human Risk

We talk about model risk as if it lives inside equations. Part of it does: wrong assumptions, fragile calibrations, silent data issues, overfitting dressed as intelligence. The larger part lives in people—who trusts the output, who can override it, who owns the consequence when the model is confidently wrong. Model risk is still human risk.

The seduction of clean outputs

A number with three decimals feels more true than a messy judgment. Machine-learning scores and automated signals intensify that feeling. Clean outputs are useful. Clean outputs without ownership are hazardous. Someone must still ask whether the input regime matches the training regime, whether costs are realistic, and whether the recommended size respects liquidity.

In calm markets, models drift into authority because they have not failed recently. That is when governance should tighten, not loosen.

Overrides tell the cultural truth

Every shop has overrides. Healthy shops log them, expire them, and review patterns. Unhealthy shops treat overrides as proof of superior intuition whenever they win and as irrelevant noise whenever they lose. That asymmetry trains the organisation to dismantle its own controls.

Documentation is a risk control

If you cannot explain what a model is optimising, what it ignores, and how it is monitored, you cannot manage it. “It is complex” is not an explanation. Complexity that cannot be narrated at the decision layer does not belong in an unsupervised capital path.

AI quant work needs adult supervision

I organised research around AI-driven analysis because machines help with scale and consistency. I did not organise it to abolish accountability. Intelligent risk management and automated execution only deserve those adjectives inside explicit envelopes. Outside envelopes, automation accelerates mistakes.

Schedule model reviews while P&L is dull. Waiting until breakage in a crisis guarantees panic rebuilding. Include vendor and data dependencies. A delayed feed can freeze a desk that believes itself sophisticated.

A practical ownership rule

Incentives around model authority

If bonuses depend on shipping models quickly, provenance and monitoring will be underfunded. If committees fear looking anti-innovation, weak models will be waved through. Align incentives with stewardship: reward clean kills, clean documentation, and early detection of drift as much as reward new signal launches.

Create a path for junior staff to challenge model outputs without career risk. Many model disasters include someone who was uncomfortable and quiet. Psychological safety is a risk control.

Layered defences

Use multiple imperfect defences: unit tests on data joins, champion-challenger models where proportionate, hard exposure caps, and human review for material notionals. No single defence is enough. Layers buy time.

When a model succeeds, do not skip the autopsy of why. Lucky fits should not become lore. When a model fails, avoid humiliation rituals that teach hiding. Seek correctable causes.

Remember the punchline: the equation is not the risk owner. A named human is. Write the name down.

Every material model needs a named owner, a review cadence, a known failure signature, and a kill or degrade path. If that sounds heavy, the model may be too important to run casually or too unimportant to defend.

Board-level questions worth asking

Which models can move capital without a human? What is the largest automated notionals path? When was the last kill-switch test? Who owns model inventory? Boards that only hear about accuracy metrics miss the human system around the model.

Demand answers in plain language. If plain language is impossible, the system is not ready for the authority it has been given.

Incident theatre versus incident learning

After a model incident, avoid theatre that only seeks a scapegoat. Seek the chain: data, assumption, override, incentive, and monitoring gap. Fix the chain. Then practise the kill-switch again. Learning without rehearsal is half-done.

Keep a model inventory that executives can read. Unknown models with capital authority are organisational debt. Inventory days are hygiene days. Hygiene is how human risk around models stays bounded.

Name the owners. Test the stops. Remember that intelligence without stewardship is just speed.

Incident theatre versus incident learning

After a model incident, avoid theatre that only seeks a scapegoat. Seek the chain: data, assumption, override, incentive, and monitoring gap. Fix the chain. Then practise the kill-switch again. Learning without rehearsal is half-done.

Keep a model inventory that executives can read. Unknown models with capital authority are organisational debt. Inventory days are hygiene days. Hygiene is how human risk around models stays bounded.

Name the owners. Test the stops. Remember that intelligence without stewardship is just speed.

Equations fail. Humans decide how expensive the failure becomes. Build for that reality.