Inside Coding Agents
v0.2 preview
EXPERIMENT · CONTROLLED · PASSED

Reference Harness stream normalization probe

Can the Reference Harness normalize a fragmented model stream into one deterministic event sequence without depending on hidden model state?

reference-stream-normalization-v12 repetitions5 metrics
Evidence boundary

This is a controlled experiment. It supports only the listed subjects, fixture, configuration, and outputs; it does not generalize to native product behavior or cross-agent ranking.

STATUSpassed
SUCCESSFUL2/2
FIXTUREd369431d683d
PROMPT51f9c1719823
SUBJECTS + CONTROLS

Which variables are fixed?

Both repetitions will preserve stream order, mark the final model response, produce the expected answer, and share one normalized Trace fingerprint.

SDK

Reference Harness

reference-harness-2026-08-10-controlled

Model
scripted-model-v1
Provider
local-fixture
Comparability
not-comparable
Inspect snapshot →
CONTROLS

Run manifest

network
false
temperature
0
writes
false
sandbox
lesson-owned-temporary-fixture
RUNS + METRICS

Are repeated runs consistent?

python3 labs/runner.py reference-stream-normalization-v1 --check
RUN 01passed

reference-stream-normalization-v1-run-001

  • event-order-validpass
  • observations-matchpass
  • final-answer-matchpass
  • fixture-unchangedpass
  • deterministic-replaypass
3f234ce09a5c8cd997c743cb9f5459496a78de8f32b3524b5ff06cd912db9ef6Replay 7 events →
RUN 02passed

reference-stream-normalization-v1-run-002

  • event-order-validpass
  • observations-matchpass
  • final-answer-matchpass
  • fixture-unchangedpass
  • deterministic-replaypass
3f234ce09a5c8cd997c743cb9f5459496a78de8f32b3524b5ff06cd912db9ef6Replay 7 events →
SUCCESS CRITERIA
  1. Both repetitions expose the declared final stream frame and completed stop reason.
  2. Both repetitions return the exact expected answer and share one normalized Trace fingerprint.
LIMITATIONS
  • The Scripted Model and Reference Harness are deterministic teaching components, not Native vendor evidence.
  • The probe validates the declared normalization path only; it does not cover malformed or vendor-specific streams.
  • The experiment does not measure latency, token usage, cost, or model capability.
GENERATED REPORT

Reference Harness stream normalization probe

  • Status: PASSED
  • Mode: controlled
  • Experiment: reference-stream-normalization-v1

Question

Can the Reference Harness normalize a fragmented model stream into one deterministic event sequence without depending on hidden model state?

Reproduction

python3 labs/runner.py reference-stream-normalization-v1 --check

The command rebuilds the expected artifacts in memory and exits non-zero if a committed trace, result, fixture digest, prompt digest, or report differs.

Result

2 of 2 repetitions passed all declared metrics.

  • reference-stream-normalization-v1-run-001: passed, 7 events, fingerprint 3f234ce09a5c8cd997c743cb9f5459496a78de8f32b3524b5ff06cd912db9ef6
  • reference-stream-normalization-v1-run-002: passed, 7 events, fingerprint 3f234ce09a5c8cd997c743cb9f5459496a78de8f32b3524b5ff06cd912db9ef6

Metrics

  • event-order-valid: pass
  • observations-match: pass
  • final-answer-match: pass
  • fixture-unchanged: pass
  • deterministic-replay: pass

Input digests

  • Fixture: d369431d683da5c2cad36bb6dc6f76d429c2efbdc29171d5c8d9d4d7c3c575aa
  • Prompt: 51f9c1719823cfcedfe60c50fe59fefe0167e444b30430b9018d72468c931fb4

Evidence boundary

  • The Scripted Model and Reference Harness are deterministic teaching components, not Native vendor evidence.
  • The probe validates the declared normalization path only; it does not cover malformed or vendor-specific streams.
  • The experiment does not measure latency, token usage, cost, or model capability.
Back to Lab →