Quantor
Methodology · 7 min read

We pre-registered a strategy test. It failed. Here's every number.

Every bot vendor eventually announces a "new class of signals." We designed one too — mean reversion in calm markets. Then we did the thing marketing departments never do: we committed the pass/fail rules to git before running the test, ran it exactly once, and promised to publish whatever came out. It failed. Here's the corpse, in full.

The idea (it sounds so reasonable)

Buy statistically oversold dips — price stretched two standard deviations below its moving average — but only when the market is calm: low volatility, no crash in progress. Exit when price reverts to the mean. A wide disaster stop, an exit if the regime turns violent, a time stop if nothing happens. The story writes itself: in quiet markets, impatient sellers overshoot, and you get paid for catching the overshoot.

We'd tested plain mean reversion before, inside our 264 walk-forward study — it failed there. The new hypothesis was that the calm-regime filter and a redesigned exit would rescue it. That's a legitimate new question. It's also exactly the kind of question that produces overfit garbage if you let yourself tweak parameters after seeing results.

The discipline: decide the rules before you see the answer

So we did it the way medical trials do it, not the way bot marketing does it:

  • Arithmetic first. Before any backtest, we computed whether the idea could clear costs at all: median capturable move ≈1.6% on 4-hour bars against a 0.30% round-trip cost, predicting about +0.40% per trade if the win rate hit 65% and stop-outs stayed near 20%. Two whole timeframes died at this stage — 1-hour (costs eat the capture) and daily (three trades in nine years) — excluded before testing, not after.
  • Pre-registration. Every parameter, threshold, cost model, evaluation window, and all seven pass/fail gates were written into a document and committed to git before the run — including the kill line: "if stop-outs exceed 20–25% of trades, mean reversion likely dies."
  • One shot. The test runs once, on pinned data (~9 years of real Binance BTC and ETH 4-hour bars, through 2026-07-19). No re-tuning, no re-running, no window shopping. Any follow-up idea is a new pre-registration in a trials ledger that now stands at 346 — so every past attempt discounts the statistics of the next one.
  • A hostile review before committing. A second, independent AI engine reviewed the pre-registration adversarially and made it stricter — it cut our planned parameter variants from six trials to two, closed a subtle stop-accounting loophole, and renamed a soft verdict category into what it really is: replication failure.

The result: it died, and the autopsy is clean

BTC (4h)ETH (4h)Assumed in advance
Closed trades15651
Win rate39.1%43.1%65%
Trades ending at the stop56%45%20–25% = kill line
Expectancy per trade (after costs)−0.48%−0.39%predicted +0.40% / +0.38%
Net over the full period−53.9%−19.0%
Profit factor0.570.62gate: ≥1.15

Six of the seven gates failed, on both symbols. The pre-registered program verdict: FAIL.

The stop-out rate is the whole story. The design assumed that in calm markets, a two-sigma break below the mean is usually an overreaction that snaps back. The data says otherwise: more often than not, it's the start of a move. The falling knife cuts even in quiet markets. The 65%-win assumption wasn't a rounding error — reality came in at 39%. That's not a parameter to tune; that's a hypothesis to bury.

Why this is good news for you (though not for our strategy).

The test cost one run and zero re-tunes, and it produced a decisive answer. That is what pre-registration buys: failures die fast and cheap, and they stay dead — nobody gets to resurrect this idea by quietly sweeping parameters until something looks good. When a vendor shows you only winning backtests, now you know exactly what machinery is missing.

The same day's second finding: a real premium, currently too thin

We also measured perpetual-futures funding — the fee leveraged longs pay shorts — across the full history since 2019, as a candidate delta-neutral carry strategy. The premium is real: it averaged roughly +12%/year gross on BTC and +14% on ETH, with remarkably shallow negative stretches. But it is strongly cyclical — the fat years were the 2020–21 bulls — and in 2026 to date it pays about 1%/year, which is what an exchange's savings product pays with none of the leverage, liquidation risk, or new infrastructure. So we declared a re-open trigger (a sustained month above ~10% annualized) and built nothing. Measuring first is cheaper than building first.

Why we publish failures

Because a published failure is what makes a pass mean anything. Our PAPER bot results on live market data are signed with Ed25519 keys — losses included — and our research negatives go up in full, like this one. If we ever do claim a strategy passed a pre-registered test, you'll be able to hold that claim against exactly this standard, ledger and all. A track record you can verify — including the corpses — is the entire product. It's the same discipline behind Verified by Quantor, where we apply this kind of audit to other people's trading records.

No Quantor strategy is claimed to have a return edge. What our testing supports is narrower and honest: rules-based execution, hard risk limits, and lower drawdown in 108 of 264 tested configurations — partly the mechanical effect of holding cash part of the time. Risk reduction, not alpha. Crypto trading carries substantial risk of loss — Quantor does not guarantee returns.