Is B Really Better Than A? A Trading Strategy Comparison

How a single-value performance metric can distort the ranking of trading strategies — and how to avoid falling for it and gain deeper insight into strategy comparison.

Picture two trading strategies, A and B. You run backtests for them, compare the performance metric, and B comes out ahead. Case closed — B is the better strategy?

Not necessarily. A single performance value is just one draw from a whole distribution of possible outcomes. If B's distribution is wide and A's is narrow, that "winning" draw from B might be nothing more than a lucky tail observation, while A's number reflects what it typically delivers. Compare two single points, and you can't tell genuine outperformance from pure luck.

This is precisely the issue that motivated me to write my newest paper: how do we compare trading strategies fairly once we stop reducing their performance to a single scalar and instead look at the full distribution behind it? I rigorously investigated that problem using a GAMLSS/ZAGA approach to precisely compare two trading strategies applied to the S&P 500 — buy-and-hold and a second one based on machine learning, namely a polynomial-kernel SVM strategy (SVMP).

I share my paper below:

Regime-Conditional Distributional Comparison of Trading Strategies: A GAMLSS/ZAGA Framework Applied to the S&P 500
arXiv: https://arxiv.org/abs/2606.31251
SSRN: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=7058418