PDF Textbook

Scientific Backtesting

A Methodological Framework for Inference and Explainability in Trading Models

📚 PDF Textbook (38 pages) 📖 Price: $29 + VAT
Get the PDF Textbook
Scientific Backtesting textbook cover

Core Skills You’ll Master

By the end of this PDF Textbook, you will be able to:

  • Conduct scientifically valid backtests that separate training and testing phases rigorously
  • Use CPCV and other methods to derive estimates of expected performance
  • Apply statistical inference to assess the reliability of model’s performance
  • Detect backtest overfitting using stimulant/destimulant logic
  • Evaluate the minimum sample size and control for multiple comparison bias
  • Implement Explainable AI (XAI) to interpret the drivers of trading model performance
Get the PDF Textbook $29 + VAT ISBN 978-83-980747-1-1

Who Is This PDF Textbook For?

This PDF Textbook is ideal for:

  • Quantitative researchers and data scientists designing interpretable trading models who seek to connect statistical inference with model explainability
  • Academics and graduate students in finance, econometrics, or data science seeking a rigorous foundation in backtesting
  • Financial analysts and portfolio managers who want to validate strategies beyond traditional performance metrics
  • Developers and quants integrating XAI into trading research to make model behavior explainable
  • Anyone interested in transforming backtesting from a black-box tool into a scientific methodology

PDF Textbook Description

This PDF Textbook presents essential components of a scientific and transparent framework for backtesting trading models—bridging statistical inference and explainable artificial intelligence (XAI). It systematizes the backtesting process as an instrument of empirical research, rather than a mechanical and “black-box” validation tool.

Step by step, you will:

  • Understand Marcos López de Prado’s three laws of backtesting and how they underpin scientific validation in using trading model’s backtests
  • Explore and compare Walk-Forward (WF), Cross-Validation (CV), and Combinatorial Purged Cross-Validation (CPCV) methods, learning when and how to apply each
  • Learn how to perform statistical inference from testing paths and why conventional backtests often fail to provide statistically meaningful conclusions
  • Study the Backtest Overfitting Problem, extended here to distinguish between stimulant and destimulant performance measures—an author’s original contribution that refines how overfitting is defined and understood
  • Address the Minimum Sample Size and Multiple Comparison problems to ensure statistically reliable conclusions
  • Discover how Explainable Artificial Intelligence (XAI)—using Logistic Regression, Decision Trees, LIME, SHAP, and Explainable Boosting Machines (EBM)—can explain the variability of backtest results, revealing why a model performs as it does, not just how much it performs

The PDF Textbook emphasizes transparency, reproducibility, and interpretability—transforming backtesting into a scientific ecosystem for evaluating and explaining trading models.

Requirements / Prerequisites

  • Intermediate understanding of financial data, trading models, backtesting concepts, and performance measures
  • Familiarity with basic statistics, hypothesis testing, calibration, estimation, and optimization
  • General understanding of how quantitative models are evaluated, validated, and interpreted in practical applications
  • Interest in quantitative methods that emphasize transparency, inference, and interpretability

PDF Textbook Contents

View full detailed table of contents
  • 1. Introduction
  • 2. Laws of Backtesting
    • 1. Marcos’ first law of backtesting
    • 2. Marcos’ second law of backtesting
    • 3. Marcos’ third law of backtesting
  • 3. Backtesting Methods
    • 1. Intuition test
    • 2. Naïve Backetsting
    • 3. The Walk-Forward (WF) Methods
      • 1. The WF with Rolling Window
      • 2. The WF with Expanding Window
      • 3. Advantages and Disadvantages of the WF Methods
    • 4. The Cross-Validation (CV) Method
      • 1. The CV Method with k Folds
      • 2. Advantages and Disadvantages of the CV Method
    • 5. The Combinatorial Purged Cross-Validation (CPCV) Method
      • 1. CPCV with Spiting, Purging, and Embargoing
      • 2. Advantages and Disadvantages of the CPCV Method
    • 6. Comparison of the WF, CV, and CPCV Methods
    • 7. Statistical Inference from Paths of Testing Sets
    • 8. Performance Measures
    • 9. Optimization Methods
  • 4. The Backtest Ovefitting Problem
  • 5. The Minimum Sample Size Problem
  • 6. Multiple Comparison Problem
  • 7. Explainable Artificial Intelligence (XAI) in Backtesting
    • 1. Rationale for XAI Methods in Backtesting
    • 2. Explainable AI Methods Application in Backtesting
      • 1. Logistic Regression
      • 2. Decision Tree
      • 3. Local Interpretable Model-Agnostic Explanations (LIME)
      • 4. Shapley Additive Explanations (SHAP)
      • 5. Explainable Boosting Machines (EBM)
  • 8. Conclusions
  • 9. References

About the Author

Dr. Krzysztof Ozimek, PRM Quantitative Investment & Trading Research Educator with over 30 years of experience teaching advanced finance, econometrics, and quantitative trading methods. His educational resources are science-based and carefully designed.

FAQ

Q: What is this PDF Textbook about?
A: A scientific framework for backtesting that integrates statistical inference on performance measures with their explanation using Explainable Artificial Intelligence (XAI), addressing methodological issues such as overfitting, sample size, and multiple comparisons.

Q: How is it different from other backtesting books?
A: It firmly links backtesting to statistical inference and model explainability, introduces the stimulant and destimulant performance measures in overfitting analysis, and presents a novel application of Explainable AI (XAI) to performance explanation.

Q: Do I need to code?
A: This PDF Textbook does not require coding skills. Its purpose is to build an understanding of the scientific and methodological foundations of backtesting—how trading models are evaluated, validated, and interpreted—rather than to implement any toolkit in code.

Q: Is it practical or theoretical?
A: It is primarily methodological—bridging theory and practice to show how scientific principles apply to real backtesting procedures. The emphasis is on achieving reliability through statistical inference and interpretability within backtesting frameworks.

Q: What’s new here?
A: Two original contributions by the author:

  • A reformulated definition of backtest overfitting distinguishing between stimulant and destimulant performance measures.
  • The specific application of Explainable AI (XAI) to explain and interpret trading models performance measures.
Get the PDF Textbook $29 + VAT ISBN 978-83-980747-1-1

Cite This Work

Ozimek, K. (2026). Scientific backtesting: A methodological framework for inference and explainability in trading models (1st ed.). Eduinn Krzysztof Ozimek.

BibTeX citation
@book{ozimek2026scibacking,
  author    = {Ozimek, Krzysztof},
  title     = {Scientific Backtesting},
  subtitle  = {A Methodological Framework for Inference and Explainability in Trading Models},
  edition   = {1},
  publisher = {Eduinn Krzysztof Ozimek},
  address   = {Warsaw, Poland},
  year      = {2026},
  isbn      = {978-83-980747-1-1},
  url       = {https://www.drkrzysztofozimek.com/scientific-backtesting-textbook/},
  orcid     = {0009-0005-5210-9205}
}
ISBN 978-83-980747-1-1 ORCID 0009-0005-5210-9205

Accessibility

PDF documents included in this bundle are text-selectable and compatible with screen readers.