Simpson’s Paradox in Trading: When Aggregation Reverses Algorithm Rankings
Introduction
When evaluating trading algorithms, it is common to compare their average performance across the entire dataset. But if algorithms are used differently across market regimes, aggregated performance can produce a completely false ranking. The example below shows how an algorithm that is worse in every regime can appear superior overall.
Explanation of the Table
Rows represent market regimes: Easy and Hard.
Columns show:
Each algorithm’s average PnL per trade in that regime,
The share of trades that each algorithm executed in that regime, relative to its own total flow.
| Market Regime | Algo A: avg PnL / trade (bps) | Share of A flow | Algo B: avg PnL / trade (bps) | Share of B flow |
|---|---|---|---|---|
| Easy | 2 | 90% | 4 | 20% |
| Hard | -1 | 10% | 0 | 80% |
Calculations
Per-regime comparison (higher is better):
Easy: \(2 < 4 \Rightarrow B\) better
Hard: \(- 1 < 0 \Rightarrow \text{ }B\) better
Overall expected PnL for Algo A
\[E\lbrack PnL \mid A\rbrack = 0.90 \cdot 2 + 0.10 \cdot ( - 1) = 1.8 - 0.1 = 1.7\ bps\]
Overall expected PnL for Algo B
\[E\lbrack PnL \mid B\rbrack = 0.20 \cdot 4 + 0.80 \cdot 0 = 0.8 + 0 = 0.8\ bps\]
Conclusions
Algo B outperforms Algo A in every regime.
Aggregated over all trades, Algo A appears better (1.7 bps > 0.8 bps).
The ranking is reversed: the inferior algorithm per regime looks superior overall.
For Further Reading
An extended discussion of Simpson’s paradox in a trading context can be found in Handbook of Price Impact Modeling (2023) by Kevin T. Webster.
