Module VIII — Multi-Harness Systems & Adapter Architecture

Phase 6 · MULTI-HARNESS SYSTEMS — Module VIII
Status: Authored & Empirically Verified.
Lecture Components: 6 FHD 1080p master videos + Lab L8.
Canonical Core Axiom:

NEVER COUPLE GOVERNANCE TO A PHYSICAL AGENT HARNESS:
ADAPT TO A STRONGLY-TYPED CONTRACT, ISOLATE RUNTIMES, AND ROUTE WITH DETERMINISTIC FALLBACKS.


1. The Multi-Harness Reality & Framework Monoculture

In enterprise-scale autonomous software engineering, no single agent framework or execution harness suffices. Disparate harnesses excel across distinct operational boundaries:

• Native Go Harnesses: Built for extreme throughput, sub-millisecond process dispatch, and deterministic AST parsing.
• Python / Scientific Harnesses: Specialized in tensor operations, statistical evaluations, and data science pipelines.
• Third-Party CLI Agents: Standalone binaries (such as specialized refactoring tools or terminal coding engines) operating on local git trees.

The amateur anti-pattern is forcing a monolithic monoculture—attempting to rewrite every tool into a single framework or coupling system orchestrators directly to upstream vendor SDKs. The senior architecture pattern enforces Inversion of Control (IoC): the control plane interacts strictly with a unified, strongly-typed adapter interface, isolating process crashes, API deprecations, and runtime shifts behind physical boundary adapters.

┌────────────────────────────────────────────────────────────────────────┐
│                        HEFESTO ORCHESTRATOR / DAG                      │
└───────────────────────────────────┬────────────────────────────────────┘
                                    │ Unified TaskEnvelope
                                    ▼
┌────────────────────────────────────────────────────────────────────────┐
│                        HARNESS ADAPTER INTERFACE                       │
│        Execute(ctx, env) · Cancel(ctx) · Capabilities() -> Bitmask     │
└───────┬───────────────────────────┼────────────────────────────┬───────┘
        │                           │                            │
        ▼                           ▼                            ▼
┌───────────────┐           ┌───────────────┐            ┌───────────────┐
│ Mock / Fast   │           │ Subprocess    │            │ Remote / HTTP │
│ Adapter       │           │ CLI Adapter   │            │ Gateway       │
│ (In-Memory    │           │ (Ring Buffers,│            │ (Rate Limits, │
│  Unit Tests)  │           │  Job Objects) │            │  Token Ledger)│
└───────────────┘           └───────────────┘            └───────────────┘

2. The Canonical Adapter Contract

In pure idiomatic Go, the adapter contract decouples execution, lifecycle management, and telemetry streaming:

type Adapter interface {
    // Name returns the canonical identifier of the harness.
    Name() string
    
    // Capabilities returns the immutable bitmask of supported primitives.
    Capabilities() Capability
    
    // Execute runs a task envelope to completion or until context cancellation.
    Execute(ctx context.Context, env TaskEnvelope) (ExecutionResult, error)
    
    // Cancel terminates the active execution and sweeps orphaned process subtrees.
    Cancel(ctx context.Context) error
}

By standardizing on this interface, the overarching control plane treats in-memory test mocks, sandboxed OS CLI processes, and distributed network agents identically.

3. Capability Discovery & Bitmask Negotiation

Harnesses present acute functional asymmetries: some support native structured tool calls, others only understand plaintext conversation, and others operate strictly through terminal diffs.

HEFESTO resolves this via strongly-typed bitmasks evaluated in $O(1)$ time:

type Capability uint32

const (
    CapTools       Capability = 1 << 0 // Native structured tool calling
    CapMCP         Capability = 1 << 1 // Model Context Protocol support
    CapTerminal    Capability = 1 << 2 // Interactive shell / sandbox access
    CapGitPatches  Capability = 1 << 3 // Unified git diff / patch emission
    CapStreaming   Capability = 1 << 4 // Real-time token streaming
    CapCheckpoints Capability = 1 << 5 // Transactional state snapshots
)

Prior to dispatching work, the orchestrator conducts a Capability Handshake: 1. The TaskEnvelope declares mandatory invariants (RequiredCaps) and soft preferences (PreferredCaps). 2. The dispatcher checks (adapter.Capabilities() & env.RequiredCaps) == env.RequiredCaps. 3. If an invariant is missing, the dispatcher aborts immediately with zero token waste, routing work to an eligible fallback adapter. 4. If non-critical features are missing, the adapter applies graceful degradation polyfills (e.g., regex JSON extractors for engines lacking native tool schemas).

4. CLI Process Encapsulation & Bounded Ring Buffers

Many industrial agents exist only as command-line utilities. Encapsulating them safely requires OS-level process fences:

┌─────────────────────────────────────────────────────────────┐
│                 SUBPROCESS ADAPTER ENCLAVE                  │
│                                                             │
│   ┌────────────────────┐            ┌───────────────────┐   │
│   │   Parent Process   │  Scan Stdio│   Bounded Ring    │   │
│   │   (Go Controller)  ├───────────►│   Buffer (64KB)   │   │
│   └─────────┬──────────┘            └───────────────────┘   │
│             │ Context Deadline                              │
│             ▼                                               │
│   ┌─────────────────────────────────────────────────────┐   │
│   │             OS PROCESS FENCE / JOB OBJECT           │   │
│   │   Windows: KillOnJobClose · Linux: cgroups / SIGKILL│   │
│   │   Ephemeral Git Worktree · Strict RAM / CPU Ceiling │   │
│   └─────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────┘

1. Deadlock & Leak Prevention: Naive synchronous pipe reading exhausts RAM or triggers deadlocks when OS pipe buffers fill. We deploy non-blocking concurrent scanner goroutines piped into bounded circular ring buffers with strict byte limits. 2. Zombie Process Elimination: Subprocesses frequently spawn child compilers or runtime nodes. When a timeout occurs, killing only the parent leaves orphaned zombies. The adapter attaches processes to POSIX process groups or Windows Job Objects with KillOnJobClose, ensuring uncatchable SIGKILL tree termination. 3. Workspace Isolation: Subprocess CLI agents execute inside disposable Git worktrees on temporary directories, preventing rogue scripts from modifying the host repository.

5. Normalized Invocation Contracts: TaskEnvelope & ExecutionResult

To prevent the "Tower of Babel" problem across divergent engine schemas, HEFESTO establishes strict input/output contract symmetry:

Ingress: The Immutable TaskEnvelope

• TaskID & CorrelationID: Traceability across distributed multi-agent DAGs.
• Requirement & Invariants: Plaintext specifications paired with formal machine-checkable constraints.
• AllowedPaths & Hashes: Whitelist of mutable file paths anchored to cryptographically verified SHA-256 baseline hashes.
• ResourceQuotas: Hard caps on timeout duration, token consumption, and memory.

Egress: The Distilled ExecutionResult

• Status Verdict: Strongly-typed outcome (Success, ValidationFailed, Timeout, Aborted).
• Unified Diff: Ground-truth filesystem patch extracted directly via Git, discarding model claims.
• Sanitized Telemetry: Terminal escapes (ANSI) stripped; stdout/stderr captured in bounded logs.
• Normalized Accounting: Microsecond CPU vs network latency, exact token counters, and financial cost.

Every adapter implements two pure functions: $$\text{CompileIngress}: \text{TaskEnvelope} \longrightarrow \text{HarnessIngress}$$ $$\text{ReduceEgress}: (\text{ExitCode}, \text{Stdio}, \text{Worktree}) \longrightarrow \text{ExecutionResult}$$

6. Heterogeneous Runtimes & Orthogonal Factorization

To eliminate vendor lock-in, HEFESTO factorizes agentic architectures into five mutually orthogonal dimensions:

$$\text{Architecture} = \text{Role} \times \text{Model} \times \text{Harness} \times \text{Provider} \times \text{Tooling}$$

• Role: Hard authority boundaries, system prompts, and permissions.
• Model: Cognitive capacity (Claude 3.5 Sonnet, DeepSeek-R1, GPT-4o, Qwen 2.5).
• Harness: The execution engine and state machine (Go native, Aider CLI, Python agent).
• Provider: Transport, credential rotation, and billing (Anthropic, Bedrock, Azure, OpenRouter).
• Tooling: Deterministic action primitives and environment sandboxes.

Swapping any dimension (e.g., migrating from Claude via Anthropic to Qwen via local vLLM) requires zero code refactoring in the harness or orchestrator. All traffic is routed through a centralized, high-concurrency Provider-Agnostic Gateway in pure Go.

7. Cross-Harness Orchestration & Cascade Fallbacks

In mission-critical infrastructure, single-engine failure halts operations. Multi-harness architectures treat engines as redundant nodes under four dispatch policies:

1. Cost-First: Routes deterministic and low-complexity tasks to local lightweight harnesses. 2. Latency-First: Prioritizes native Go-compiled harnesses for sub-millisecond dispatch. 3. Capability-First: Matches tasks to harnesses presenting specialized primitives (e.g. AST manipulators). 4. Redundancy-First: Executes identical task envelopes across two distinct harnesses in parallel.

Cascade Fallback Protocol

When an active harness fails (timeout, quota breach, or verification rejection): 1. Atomic Rollback: The orchestrator executes git reset --hard on the ephemeral worktree. 2. Context Purification: Corrupted terminal buffers and conversational attractor traps are purged. 3. Envelope Repackaging: The original task envelope is combined with a structured failure diagnostic. 4. Secondary Dispatch: Work is transferred to an alternate harness in a completely sanitized environment.

8. Summary of Module Deliverables