01CONTROLLED · COMPLETE
Approval binding
reference-approval-binding-v1
- Runs
- 2
- Model calls
- 0
- Network
- off
- Artifacts
- 4
python3 labs/reproduce.py reference-approval-binding-v1 --output reproduction-report.jsonInspect the experiment first REPRODUCE · OFFLINE · PUBLIC-SAFE
Do not merely trust the result. Rebuild it yourself.
Start from a clean public-repository checkout, rebuild a deterministic experiment offline with the Python standard library, compare every committed artifact byte-for-byte, and emit a public-reviewable reproduction report.
- MODEL CALLS
- 0
- NETWORK
- NOT REQUIRED
- DEPENDENCIES
- PYTHON STDLIB
- EXPERIMENTS
- 10
CLEAN COMMIT → RUN → VERIFY → SUBMIT
Four steps to a reviewable reproduction
No API key, model account, network access, or dependency installation is needed. Git and Python 3.12+ are enough.
- 01
Pin the source state
Use a full public Git commit and confirm that the tracked worktree is clean.
- 02
Rebuild declared artifacts
Choose one Controlled experiment; the script rebuilds its traces, result, and report in memory.
- 03
Verify integrity
Compare committed artifacts byte-for-byte and verify the generated report_sha256.
- 04
Submit the outcome as observed
Keep reproduced, failed, or not-comparable exactly as observed; never rewrite a negative result as success.
QUICK STARTreference-tool-roundtrip-v1
git status --short
python3 labs/reproduce.py reference-tool-roundtrip-v1 --output reproduction-report.json
python3 labs/reproduce.py --verify-report reproduction-report.json10 CONTROLLED EXPERIMENTS
Choose a mechanism to begin
This inventory comes directly from the experiment Registry. Every entry has a fixed fixture, scenario, two runs, and committed structured artifacts.
02CONTROLLED · COMPLETE
Checkpoint and rollback
reference-checkpoint-rollback-v1
- Runs
- 2
- Model calls
- 0
- Network
- off
- Artifacts
- 4
python3 labs/reproduce.py reference-checkpoint-rollback-v1 --output reproduction-report.jsonInspect the experiment first 03CONTROLLED · COMPLETE
Context budget and truncation
reference-context-budget-v1
- Runs
- 2
- Model calls
- 0
- Network
- off
- Artifacts
- 4
python3 labs/reproduce.py reference-context-budget-v1 --output reproduction-report.jsonInspect the experiment first 04CONTROLLED · COMPLETE
Context compaction
reference-context-compaction-v1
- Runs
- 2
- Model calls
- 0
- Network
- off
- Artifacts
- 4
python3 labs/reproduce.py reference-context-compaction-v1 --output reproduction-report.jsonInspect the experiment first 05CONTROLLED · COMPLETE
Memory retrieval
reference-memory-retrieval-v1
- Runs
- 2
- Model calls
- 0
- Network
- off
- Artifacts
- 4
python3 labs/reproduce.py reference-memory-retrieval-v1 --output reproduction-report.jsonInspect the experiment first 06CONTROLLED · COMPLETE
Project trust boundary
reference-project-trust-v1
- Runs
- 2
- Model calls
- 0
- Network
- off
- Artifacts
- 4
python3 labs/reproduce.py reference-project-trust-v1 --output reproduction-report.jsonInspect the experiment first 07CONTROLLED · COMPLETE
Sandbox and network policy
reference-sandbox-network-v1
- Runs
- 2
- Model calls
- 0
- Network
- off
- Artifacts
- 4
python3 labs/reproduce.py reference-sandbox-network-v1 --output reproduction-report.jsonInspect the experiment first 08CONTROLLED · COMPLETE
Session replay and branch
reference-session-replay-v1
- Runs
- 2
- Model calls
- 0
- Network
- off
- Artifacts
- 4
python3 labs/reproduce.py reference-session-replay-v1 --output reproduction-report.jsonInspect the experiment first 09CONTROLLED · COMPLETE
Stream normalization
reference-stream-normalization-v1
- Runs
- 2
- Model calls
- 0
- Network
- off
- Artifacts
- 4
python3 labs/reproduce.py reference-stream-normalization-v1 --output reproduction-report.jsonInspect the experiment first 10CONTROLLED · COMPLETE
Tool request/result roundtrip
reference-tool-roundtrip-v1
- Runs
- 2
- Model calls
- 0
- Network
- off
- Artifacts
- 4
python3 labs/reproduce.py reference-tool-roundtrip-v1 --output reproduction-report.jsonInspect the experiment first OUTCOME IS EVIDENCE
All three outcomes must be preserved
Reproduction is not a scorecard that accepts only green results. Failures and non-comparable runs can expose platform differences, stale assumptions, or tooling defects.
Clean, pinned source rebuilt every declared artifact exactly and is ready for human submission review.
The run was comparable, but an input, execution, or artifact check failed; that result is still valuable.
The observation could not bind to a clean, pinned Git state; fix the environment without raising the evidence level.
Automation can verify the report, not who ran it.
Claim independent reproduction only after running it yourself in your own environment. Submit the original JSON, actual command, deviations, relevant affiliations, and an explicit statement that you are not acting for the project maintainer. CI, maintainer reruns, and AI agents acting for the maintainer do not satisfy the external-participant requirement.