
Out-of-sample testing: why most backtests look better than they are
How walk-forward splits, embargo periods and a multiple-testing haircut separate a durable trading signal from one that only fits its own history.
Insights series
Method-first papers from our quantitative and technology researchers. Each one states its hypothesis, the data used and the ways it could be wrong.
Publications

How walk-forward splits, embargo periods and a multiple-testing haircut separate a durable trading signal from one that only fits its own history.

Why revised economic data and restated financials inflate backtests, and how storing every value by the date it was known keeps research honest.
About the series
Research papers explain the methods behind our systematic strategies: how we build datasets, how we test signals and how we decide whether a result is real or an accident of the sample.
Papers are written by researchers in our Quantitative Trading group in Chicago and our Technology and Research team in San Francisco. They are aimed at allocators, consultants and students who want to judge a systematic process on its methods rather than its marketing.
Every paper follows the same structure as our internal research: a stated hypothesis, the data and its limitations, the test, and the result, including results that did not hold up. Any backtest shown is hypothetical and labeled as such. Read more about our research process.
What every paper contains
Written before the test is run, with the evidence that would prove it wrong.
Sources, dates and transformations, stored as they were known at the time.
What the test cannot show, and how many variations were tried before this one.
New research papers are announced in the Lenox Brief, our weekly email on rates, credit and systematic markets.