Synthetic Data in Inferential Backtesting

1. Introduction

The Single-Path Constraint

Financial model validation faces a structural constraint that no accumulation of historical data can resolve: the observed market record represents only one realized trajectory of an underlying stochastic process. Regardless of how extensive or carefully curated the historical sample may be, it cannot reveal the distribution of outcomes that could plausibly have unfolded under the same underlying dynamics. Every period of calm conceals the turbulence that did not materialize; every crisis masks the recoveries that never arrived. This irreducible singularity of the historical path is not a data problem—it is a fundamental epistemological limitation of empirical finance.

Classical backtesting inherits this limitation in full. Even in its most disciplined form—the Walk-Forward framework, which enforces strict chronological separation between learning and evaluation phases—a trading model is ultimately calibrated on one observed past and evaluated against one observed future. The resulting performance estimates are structurally thin: a single favorable or adverse market regime can dominate the conclusion, rendering the evaluation fragile and difficult to generalize. No amount of methodological rigor in the evaluation architecture can compensate for the fact that only one realization of the market process is ever available.

Synthetic Data: A Different Purpose

Synthetic data generation has emerged as one response to this limitation. In its most common framing, the objective of synthetic data is similarity: producing artificial datasets that are statistically indistinguishable from real ones, preserving distributional properties, autocorrelation structure, tail behavior, and cross-sectional dependence with the fidelity of a well-calibrated generative model. This textbook acknowledges that tradition but departs from it in a deliberate and explicit way. Here, synthetic data is not employed as a substitute for historical observations, nor is its quality judged primarily by how closely it replicates the empirical record. Instead, synthetic data serves an entirely different purpose: the construction of controlled alternative market environments in which trading models can be calibrated, stress-tested, and evaluated across a distribution of plausible scenarios rather than a single observed path. The goal is not faithful reproduction but structured exploration.

The Structural Challenge of OHLC Modeling

Compounding this methodological departure is a further distinctive feature of the framework developed here: the object being modeled is not a generic financial time series. The literature on financial time series modeling spans a wide spectrum—from univariate models of closing prices or log-returns, through unconstrained multivariate systems capturing cross-asset dependencies, to structural econometric frameworks. What is comparatively rare, and what this textbook addresses directly, is the modeling of open–high–low–close (OHLC) time series as a coherent and structurally constrained object. OHLC data are not simply four parallel time series that happen to be observed together. They carry an inherent geometric ordering constraint—the low cannot exceed the high, and both the open and the close must lie within the intraperiod range—that is not an assumption imposed from outside but a structural necessity native to the data itself. Any modeling framework that ignores or violates these constraints produces outputs that are financially incoherent by construction.

This constraint makes OHLC modeling a genuinely non-trivial problem. Applying a standard univariate model collapses the rich intraperiod structure entirely. Applying a generic multivariate framework without internal restrictions generates synthetic paths that routinely violate the ordering conditions. The latent decomposition proposed in this textbook—representing OHLC data through midpoint, half-range, and the relative intraperiod positions of open and close—is a purpose-built response to this structural challenge. It is not the only conceivable approach, but it is a carefully constructed one that enforces geometric coherence by design rather than by post-hoc correction. Readers will find that understanding this internal mechanism, and the modeling choices it entails, opens a conceptual space that generic time series frameworks do not reach.

Framework Overview

To operationalize this approach, the textbook develops an integrated three-pillar framework:

• Simulation of synthetic in-sample OHLC paths, based on an ARMA–GARCH–Copula latent-variable architecture that preserves key empirical properties of financial price data while generating multiple alternative realizations of historical market dynamics (Chapter 4);

• Forecasting of synthetic out-of-sample OHLC paths, using the TOTO time-series foundation model as a generative engine that produces probabilistic Monte Carlo forecasts of future price trajectories conditioned on the observed in-sample history (Chapter 5);

• A synthetic backtesting framework that integrates both components within a Walk-Forward evaluation paradigm, replacing the single-path OOS evaluation of classical backtesting with a distribution of synthetic performance trajectories encoded in a structured performance matrix (Chapter 6).

Methodological Building Blocks

Two methodological components serve as essential technical building blocks for this architecture. The Directional Change (DC) framework, developed in Chapter 3, provides a threshold-based mechanism for identifying local extrema in price series and partitioning the in-sample interval into structurally coherent subblocks. The spliced cumulative distribution function (CDF) methodology, introduced in Section 4.1, provides a flexible approach to marginal distribution modeling that combines kernel density estimation for the central region with Generalized Pareto tails, enabling accurate representation of both bulk behavior and extremes. Both components are introduced not as standalone contributions but as functional elements serving the simulation engine.

This textbook has an educational objective rather than an advisory or mentoring purpose. Its aim is to propose structured approaches, stimulate critical reflection, and encourage readers to develop and refine their own methods within a coherent analytical framework. The presentation moves deliberately from foundational ideas, through formal mathematical construction, to their translation into executable simulation procedures. At each stage, the reader is invited not merely to follow the proposed methodology but to interrogate it, extend it, and adapt it to their own analytical contexts.

Nature and Epistemological Position

This textbook is deliberately not written as a research monograph intended for publication in a peer-reviewed journal. Many formal statistical justifications, asymptotic derivations, and empirical verifications that would be expected in that context are intentionally absent. This is not an oversight but a considered choice reflecting the book's primary objective: to rebuild and expand the reader's consciousness and skills about backtesting as a research tool and a practice, and to do so by enriching it with the conceptual and computational machinery of synthetic data generation.

The objective of this textbook is not to establish formal statistical theorems regarding synthetic or probabilistic backtesting. It is to construct a conceptual and computational framework within which such questions can be explored. The framework is presented as a methodological platform—a structured starting point from which the reader can develop, challenge, and extend the ideas introduced here. Formal verification of the framework's statistical properties, investigation of its asymptotic behavior, and empirical evaluation across diverse asset classes and market regimes are questions that remain deliberately open, constituting a natural agenda for further research.

This framing has direct implications for how the mathematics in this textbook should be read. The quantitative constructions presented—the latent OHLC decomposition, the spliced marginal distributions, the ARMA–GARCH–Copula simulation engine, the projection-based OHLC correction, the performance matrix—are not offered as theoretically optimal solutions to formally posed problems. They are offered as carefully reasoned, internally coherent, and practically implementable tools that open the door to a richer mode of backtesting. The reader is encouraged to treat them as building blocks: to understand why each choice was made, to recognize where alternative choices are possible, and to use the accompanying scripts as a laboratory for exploring those alternatives.

The book should therefore be evaluated as a methodological exploration and conceptual framework builder, not as a fully verified econometric theory of synthetic backtesting. It occupies a space that is relatively rare in the quantitative finance literature: between a textbook and a research agenda, between a technical manual and a conceptual provocation. Its value lies not in what it conclusively establishes but in what it makes thinkable and computable that was not so before.

Educational Scope and Accompanying Scripts

The R and Python scripts accompanying this textbook are not intended to serve as production-grade components of a synthetic market simulation environment. Rather, they should be viewed as early-stage, enhancement-ready, yet functionally complete simulation tools whose primary role is educational: to complement the conceptual and mathematical framework presented in the text, and to serve simultaneously as practical implementations and as structured templates for experimentation, modification, and further development. The scripts are organized as follows:

• dc_extrema()—implements the Directional Change extrema detection procedure; available in both R and Python;

• spliced_kde_gpd()—implements the spliced KDE–GPD marginal distribution model; available in both R and Python;

• simulate_ohlc_paths()—the central simulation engine of the framework, implementing the full ARMA–GARCH–Copula OHLC simulation procedure; available in both R and Python;

• toto_forecast() and toto_forecast_ohlc()—implement the TOTO-based synthetic OOS forecasting procedure; available in Python only.

The Python-only restriction on the TOTO scripts is a deliberate and transparent design decision rather than an omission. The TOTO foundation model is distributed via Hugging Face and its open implementation is Python-native, relying on the PyTorch and Hugging Face Transformers ecosystems for which no mature or stable R equivalent currently exists. Introducing an R wrapper around these dependencies would create an artificial and fragile layer of interoperability that would obscure the underlying mechanics rather than illuminate them. The constraint therefore reflects the current state of the deep learning tooling landscape and is intended to be clearly communicated to the reader from the outset.

Structure of the Textbook

The textbook is organized as follows. Chapter 2 establishes the conceptual foundations, introducing synthetic data and the Walk-Forward backtesting framework as the two methodological pillars on which the remainder of the book rests. Chapter 3 develops the Directional Change framework for local extrema identification, providing the structural block-partitioning mechanism used in synthetic in-sample path construction. Chapter 4 presents the ARMA–GARCH–Copula simulation framework for synthetic OHLC paths, including the spliced marginal distribution model, the latent variable representation, and optional stress and drift enforcement mechanisms. The copula component plays a specific and indispensable role within this architecture: while the ARMA–GARCH models govern the marginal dynamics of each latent component separately, the copula captures their joint dependence structure, enabling the simulation of coherent multivariate paths that respect the cross-sectional relationships present in the data. Copula theory itself is a well-developed field with an extensive body of introductory and advanced literature. It is not developed independently in this textbook; readers seeking a structured treatment of copula foundations and their practical implementation are directed to Ozimek (2025c) and Ozimek (2025d), which provide open educational courses covering copula theory, generation, and visualization, equipped with supplementary educational materials. Chapter 5 introduces TOTO (Time Series Optimized Transformer for Observability), a foundation model for time series, and its integration into a synthetic out-of-sample forecasting procedure for OHLC time series. Foundation models represent one of the most actively developing frontiers in predictive artificial intelligence—large neural architectures pretrained on broad and heterogeneous corpora, capable of producing forecasts in a zero-shot manner without task-specific retraining, or further adapted through fine-tuning when domain data are available. The landscape of time series foundation models is rapidly expanding, encompassing models such as TimeGPT, Lag-Llama, Chronos, Moirai, and TimesFM, among others (Peixeiro, 2026). TOTO is selected here for a specific and technically motivated reason: unlike models that produce deterministic or interval-based forecasts, TOTO generates probabilistic forecasts through a mixture-based output distribution, enabling direct Monte Carlo simulation of future paths—a capability that is essential for the synthetic OOS evaluation paradigm developed in this textbook. It should be emphasized that classical time series models could in principle also serve as Monte Carlo engines; the choice of TOTO reflects not an exclusive reliance on foundation models but a deliberate decision to engage with this emerging class of AI-driven forecasting tools, whose potential for financial time series applications remains an open and promising area of research. Chapter 6 integrates all preceding components into the synthetic backtesting framework, covering the generation of synthetic in-sample and out-of-sample environments, model calibration approaches, and the statistical evaluation of trading models.

Readers seeking an immediate orientation to the overall architecture of the framework are encouraged to consult Section 6.1 early in their reading. The conceptual diagram presented there as Figure 6.1 provides the most informative single illustration of the entire synthetic backtesting framework developed in this textbook, mapping the relationships between all components across the Walk-Forward paradigm. It may serve as a useful navigational reference to which the reader can return as successive building blocks are introduced in the preceding chapters. Figure 6.1 is included in the PDF Preview.