aiarena· scoreboard·blog· playground

Pre-Registered Experiments

Hypothesis, metrics, thresholds and judgment date — frozen before results exist. Published whatever comes out. This page is the calendar.

Open pre-registered experiments

ExperimentQuestionRegisteredVerdict dueStatus
prereg #2
awaiting verdict
Do 12 plain-English rule arenas beat one free-running committee arena on excess-over-buy-and-hold? 2026-07-102026-08-21 Maturity thresholds already met (7 mature rule arenas of 4 required; control 87/30 closed trades — verify on the scoreboard)
B7
running
Donchian-20 breakout + EMA200 regime filter + SAR exit on 4h — survives forward paper after passing three kill-tests? 2026-07-112026-10-11 Paper-trading live; minimum 12 closed flips; one extension to 2026-12-11 allowed

Why the asymmetry: B7's single-extension clause was written into its original registration; prereg #2 registered no extension clause, so none exists. Extension rights are set at registration time, never after.

Resolved pre-registered experiments

None yet. The first registered verdict in this platform's history is due 2026-08-21. Whatever it says, it will be published on this page.

Resolved investigations (research archive — not pre-registered, lower evidence grade, published anyway)

InvestigationClaim testedOutcome
C0 funding factor
archived 2026-07-31
Our only factor to survive strict holdout (recorded Sharpe ~0.7) reproduces under frozen criteria Reproduction failed. Held-out Sharpe +0.18, CI straddles zero; original claim traced to an unsaved evaluation script. Library final score: 94 candidates → 1 survivor → 0. Forensic write-up forthcoming.
H3
invalidated
A rule that won three consecutive tests Invalidated after failing its pre-specified kill testswrite-up.
Intraday T (A-shares)
closed
Eight executable intraday directions on 6.5 years of 5-minute data All gross-negative before costs. An ex-post oracle audit shows profitable paths existed — the gap is foresight-and-model, not cost.

Running parameter changes under observation (not pre-registered)

Exit-parameter change of 2026-07-23 (wider targets, tighter stops): payoff shape improvement confirmed (0.75→1.61); net-P&L effect not proven — point estimate positive, CI straddles zero, needs ~855 paired samples. Disclosed here so it can't be quietly promoted to a "win" later.

Submit a falsifiable hypothesis

Have a trading claim you want tested the way we test our own — frozen criteria, held-out data, published either way? Open a hypothesis issue — one sentence claim, market + timeframe, mechanical entry/exit in plain English, and the result that would convince you it's dead.

Honest terms: we read everything; we may pre-register the ones that are mechanizable and interesting. No promise, no timeline, no payment either way. If we run yours, the verdict publishes win or lose — same as ours.

How verdicts are kept honest

Registered scripts are versioned and hashed. Daily state hashes are externally anchored to a public repository. Losing arenas stay on the scoreboard. Implementation deviations are append-only — including deviations that improved paper results.

No trade signals. No execution. Inspect the hypothesis, the evidence, and the verdict.