Lab L1 — The Harness Swap Experiment

Phase 1 · HARNESS — Lab L1
Status: Authored & Empirically Verified.
Student Lab Package: Authenticated Direct Download from VTAlgo Platform
Branches: main (clean starter code for students) · solution (reference architecture & instructor playbook).
Video Master: 05:25 min @ 4K UHD 60fps · Audio Elmer Oficial (-16 LUFS, Zero BGM) · English Subtitles (ASS 82pt).
Canonical Path: Student repo only — not E:\bridle, not the Kratos live product.

1. Laboratory Objective

Demonstrate empirically on the student's local machine that:

$$\text{Model} \neq \text{Agent}$$

and that holding the foundation model and engineering tasks constant while swapping the execution control plane yields dramatic, measurable divergence in success rate, token consumption, latency, and catastrophic failure resistance.

2. The Controlled Experimental Matrix

To isolate the harness as a first-class independent variable with scientific rigor, two canonical variables are held strictly invariant:

$$\text{MODEL} = \text{CONSTANT} \quad (\text{Qwen 3.8 Flash @ Temperature } 0.0, \ \text{Seed Deterministic})$$ $$\text{TASK} = \text{CONSTANT} \quad (\text{Suite of 5 Automated Software Engineering Tasks in Go})$$ $$\text{HARNESS} = \text{VARIABLE} \quad (\text{Harness A vs. Harness B})$$

                   ┌─────────────────────────────────────────┐
                   │       THE HARNESS SWAP EXPERIMENT       │
                   └────────────────────┬────────────────────┘
                                        │
                 ┌──────────────────────┴──────────────────────┐
                 ▼                                             ▼
       ┌───────────────────┐                         ┌───────────────────┐
       │     HARNESS A     │                         │     HARNESS B     │
       │ (Market Baseline) │                         │ (HEFESTO Control) │
       ├───────────────────┤                         ├───────────────────┤
       │ • Plaintext ReAct │                         │ • Formal FSM      │
       │ • Raw Error Dumps │                         │ • 3-Tier Firewall │
       │ • Unbounded Loops │                         │ • Typed Contracts │
       │ • No Rollback     │                         │ • Atomic Rollback │
       └───────────────────┘                         └───────────────────┘

3. The 5 Engineering Benchmark Tasks

The student agent must diagnose, repair, and verify five non-trivial Go engineering challenges against automated unit tests:

Task ID	Domain Challenge	Engineering Invariant
`task01_ast`	Go AST Analysis	Parse AST trees, detect unexported struct fields, and generate typed getters.
`task02_data_race`	Concurrency & Mutex	Diagnose a high-contention memory race condition detected by `go test -race` and introduce fine-grained mutex synchronization.
`task03_deadlock`	Channel Deadlock	Resolve circular channel dependency in a goroutine worker pool.
`task04_mem_leak`	Memory Profiling	Identify unclosed goroutines leaking memory in a streaming subscriber pipeline.
`task05_sql_acid`	Transaction Isolation	Enforce ACID isolation and rollback semantics on partial batch inserts.

4. Empirical Benchmark Results

Measured on console under identical hardware and environment constraints:

Metric	Harness A (Market Baseline)	Harness B (HEFESTO Architecture)	Delta / Impact
Benchmark Success Rate	5 / 5	5 / 5	Parity achieved
Total Token Consumption	29,975 tokens	8,591 tokens	-71.3% token reduction
Mean Execution Latency	123.5 seconds	34.0 seconds	3.6x overall speedup
Task 02 (Data Race) Tokens	11,449 tokens	496 tokens	-95.7% token reduction
Task 02 (Data Race) Time	4m 20s (runaway loop)	12.3 seconds (single turn)	21x latency acceleration

Why Did Harness A Collapse on Task 02?

When go test -race failed, Harness A dumped a raw 200-line concurrency stack trace into the plaintext context. Recency bias trapped the model in a stochastic collapse attractor, hallucinating syntactic patches over already broken code for 4 minutes.

In contrast, Harness B's Observation Sanitizer extracted only the data race line invariant and injected clean AST context. The model solved the race condition in a single surgical 12-second turn.

5. Hands-On Student Protocol

Step 1: Download & Unpack the Private Student Package

Descargá el paquete oficial hefesto-lab1-harness-swap.zip directamente desde el aula autenticada de VTAlgo LAB: Descargar Laboratorio (ZIP)

*(El paquete está protegido y solo es accesible para alumnos con sesión activa y serial canjeado).*

Descomprimí el archivo en tu máquina y entrá a la carpeta:

cd hefesto-lab1-harness-swap

*(El archivo incluye el repositorio local de Git preconfigurado con las ramas main de trabajo y solution de referencia).*

Step 2: Verify Baseline Go Environment

go version
go test ./...

Step 3: Run the Comparative Benchmark

go run ./cmd/benchmark --compare

*Observe console telemetry: Harness A collapses into stochastic hallucination loops on concurrency and schema tasks (0/5 passed, high token burn), while on main Harness B awaits your implementation of the 3-Tier Admission Firewall and ACID State Rollback in internal/harness_b/.*

Step 4: Implement Milestones 1 & 2

Complete the TODOs in:

internal/harness_b/firewall.go (Inspect, SanitizeCompilerOutput, CompactHistory)
internal/harness_b/ledger.go (CommitSnapshot, Rollback)

Re-run the benchmark to reach production certification SLA ($\ge 80\%$):

go run ./cmd/benchmark --compare

Step 5: Reference Solution & Instructor Playbook

To inspect the reference architecture and complete analysis:

git checkout solution
cat INSTRUCTOR_PLAYBOOK.md
go run ./cmd/benchmark --compare