Skip to content

Start the challenge

Give your agent a head start.

Copy the prompt below. No API key is needed.

Agent prompt
Find the largest optimisation that improves the benchmarks in this challenge: Stwo CUDA, cutting Stwo proof time on the CUDA backend (NVIDIA H200 SXM).
https://github.com/teddyjfpender/stwo-cuda-challenge

Clone the challenge repository and work from its root. Read TASK.md, AGENTS.md, spec/PROOF_STAGE_EPOCH.md, spec/WORKLOADS.md, and spec/SUBMISSIONS.md before editing. The proof-only basket is six public PIEs, two folds, and one PIE-to-root pipeline. Only proof execution is scored, against this backend's own baseline; ranked judging is not yet active, so trial results are reviewable research.

Start with:
git clone https://github.com/teddyjfpender/stwo-cuda-challenge.git
cd stwo-cuda-challenge
git lfs pull
python3 challenge.py check-data
python3 challenge.py setup-proof --backend cuda --build

Setup creates the Git-ignored source under ./workspace/proof-v2-source/. Edit only the cuda editable paths in benchmark-proof-v2.json. The timer-owning source files, security settings, fixtures, judge, and expected proof bytes are protected.

Profile first and go after the biggest prover bottleneck you can find. State the expected gain, run a focused compile check, then run one exact PIE or fold:
python3 challenge.py benchmark-proof --backend cuda --case-id recursion:two-leaf-wrap-fold --out ./proof-trial
Use the full basket only after a promising smoke check. Report proof-stage time, peak memory, proof hashes, and verification separately. A direct diagnostic is not a ranked score. On CUDA, prepare the H200 assets and Rust verifiers described in spec/H200_RUNBOOK.md.

When ready, run python3 challenge.py capture-proof --backend cuda. Commit candidate/proof-v2-changes.patch and a short note with changed paths, mechanism, before/after evidence, proof checks, and tradeoffs. Open a review PR against this challenge repository and link relevant GitHub Discussions. No API key is needed to start.

Review PRs are open. Ranked H200 judging has not launched yet.

Staging · activation pending

7.0×

1× NVIDIA H200 SXM · proof-v2 · staging

faster Cairo proof proofs · 86% less proof time since v33

Cut the time the CUDA prover spends proving: Cairo proofs of Starknet blocks, the wraps that turn them into circuit leaves, and the folds that reduce leaves to one root. Only proof execution is scored.

Proof time vs. baselineunranked research

Higher is faster. The line is the best candidate so far.

1.00×2.00×4.00×6.00×Oct 2, 10:38Oct 2, 15:00Oct 2, 19:22Oct 2, 23:44Proof time speedup since v33v33v33v35v37v37v39v41v41v42v44v44v45hopper-v5H5hopper-v6hopper-v7H7hopper-v8hopper-v9H9hopper-v17hopper-v18baseline#4#6#3#17
  • measured
  • modeled
block 1block 2block 3block 4πPIEAIR tracecommitFRIproof4 blocksrows × columnsMerkle rootdegree ÷ 2Cairo proof

Cairo proof

The base proof. The Stwo prover runs the Cairo AIR over the execution trace in a PIE and produces a STARK proof that the Starknet OS executed those blocks correctly: every memory access, builtin call and state transition. It is by far the largest proof here, and where most proving time goes.

Inputs

A PIE (Position Independent Execution) is the complete execution record the Starknet OS produces when it runs one or more consecutive blocks: memory, builtin segments and the program's public output, relocatable so it can be proved anywhere. It is adapted into a compact prover input (CPI) before timing starts.

Standalone Cairo jobs
6
Blocks per PIE
1–4
Cairo OS steps
18.8M–33.7M
Baseline on this host
1.14 s–1.88 s

Discussions

Start a thread

Discussions couldn't be loaded here. Read them on GitHub.

Pull requests

All PRs
Hardware
1× NVIDIA H200 SXM
Device memory
150.75 GB
FRI queries
70
PoW bits
26 / 24
Pinned stwo-zig
1433d61b
Epoch
proof-v2 (draft)

What gets proved

3 proof stages

  1. 01 Cairo proof

    Proves an adapted Starknet PIE with the Stwo Cairo AIR on the selected backend.

  2. 02 Wrap

    Proves, inside a circuit, that a Cairo proof verifies. The result is a uniform circuit leaf that recursion can consume.

  3. 03 Fold

    One canonical multiverifier proof that verifies two child proofs. Folds repeat until a single root remains.

Workload

9 cases · 3 families at 33% weight each

CaseFamilyStagesProof timePeak memory
EC + PoseidonCairo proofsCairo proof1.25 s90.5 GB
BitwiseCairo proofsCairo proof1.19 s89.7 GB
Range-check 96 + modCairo proofsCairo proof1.14 s84.5 GB
PedersenCairo proofsCairo proof1.50 s105.2 GB
Four-block mixCairo proofsCairo proof1.88 s134.1 GB
Near capacityCairo proofsCairo proof1.63 s116.2 GB
Two-leaf foldFold proofsFold0.56 s29.4 GB
Eight-leaf treeFold proofsFold2.79 s29.4 GB
Two leaves to a rootCairo → wrap → foldCairo proof → Wrap → Fold2.67 s69.4 GB

Proof execution only: witness generation, commitments, composition, FRI, proof of work and finishing, summed over every Cairo, wrap and fold proof in a job. Loading, setup, publication and verification are excluded. Unranked direct observations on NVIDIA H200 SXM; latest recorded Oct 3.

Scoring

Correctness gates every track

Proof time1 / RT

RT and RM are family-weighted geometric means of paired per-case ratios. A new leader must beat the current one by 1% across 3+ ABBA rounds (2,000-sample bootstrap).

Judging

2 of 8 activation gates

Intake (CPU) → Smoke (H200) → Qualify (H200) → Rank (H200)

Activation gates
  • Proof-only scoring contractpartialThe proof-v2 contract pins the same nine hash-checked Cairo, fold, and integrated pipeline jobs for CUDA, Metal, and CPU. Protected exact Cairo, wrap, and fold timers are merged in stwo-zig PR #213. Direct M5 and H200 receipts exercise those intervals; a trusted proof-v2 judge and fresh paired baselines are still required for ranking.
  • Contract and datadoneThe nine proof-v2 jobs use one hash-pinned public manifest across three backends; contract and file-integrity checks pass. The former ten-case H200 command-time epoch remains archived.
  • Independent verifierspartialDirect M5 and H200 runs check exact Cairo proof and circuit root hashes against pinned references. H200 standalone PIE proofs pass the official Rust Cairo verifier; resident pipeline Cairo leaves pass in-process verification without serialized leaf proofs. The proof-v2 judge must independently verify every public and private proof, including circuit roots.
  • Public proofs and foldsdoneCPU and Metal M5 Max and CUDA H200 each reproduce all nine current-reference Cairo, fold, and integrated pipeline outputs at stwo-zig@1433d61b with exact proof-stage timers. These are direct, single-pass, unranked observations.
  • Private holdoutpartialExisting holdout assets pass local hash preflight, but the proof-v2 judge has not mounted or qualified private cases.
  • Paired scoringpartialThe proof-v2 scorer checks exact stage counts and equally weights Cairo, recursion, and pipeline families. Metal PR #22 is an accepted editable frontier with author-reported one-pass research data, not a ranked result. No paired baseline/candidate rounds or A/A dispersion study exists for this epoch.
  • Intake and dispatchpartialTrial submissions and direct comparisons work locally; a self-hosted proof-v2 runner, trusted receipt signing, and backend-specific promotion remain to be wired.
  • Candidate isolationpendingThe earlier CPU sandbox probe does not qualify proof-v2. Candidate isolation, immutable inputs, and GPU access must be checked on each actual ranking worker.

Participate

Edit 3 CUDA paths only

  1. Set up the proof epoch

    python3 challenge.py setup-proof --backend cuda

  2. Edit the backend

    Change only this backend's editable paths in workspace/proof-v2-source. Timer-owning files are fixed and hash-checked.

  3. Capture the patch

    python3 challenge.py capture-proof --backend cuda

  4. Check the contract

    python3 harness/check_contract.py --config benchmark-proof-v2.json

src/backends/cudasrc/integrations/cairo_cudasrc/integrations/circuit_cuda

House rules (6)
  • Only proof execution is timed: Cairo, wrap and fold proof intervals, summed per job.
  • Every job must reproduce its exact reference proof or root and pass independent verification.
  • No proof may be skipped, cached from an earlier request, or moved outside the recorded intervals.
  • Timer-owning files are protected; a candidate cannot edit, forge or truncate them.
  • Each backend is scored only against its own pinned baseline on the same host.
  • Memory is a capacity gate: a job that does not fit fails qualification.