Shahzad Bhatti Welcome to my ramblings and rants!

You may start with a simple agent for demo that calls a tool, gets an answer, prints it. But building a production ready agentic system requires a full agent harness so that it doesn’t crash halfway through a task. For example, you might have an AI coding assistant that debugs production incidents by searching logs, isolating a root cause, creating a pull request for the fix. It might take several minutes with dozens of tool calls. The AI agent may crash mid-run, fail to call a tool reliably or the test suite for eval fails. These are not model problems that you can solve with a better model or a better prompt. Instead, you need a reliable infrastructure that an agent harness provides. This post shows how to build an agent harness and eval pipeline using PlexSpaces, a polyglot actor framework that treats agent infrastructure as a first-class problem instead of an afterthought.

Agent = model + harness

You can think of an agent as model + harness. The harness is everything that isn’t the model.

The harness is the loop that decides when to stop, the tool calling that connects the model to the world, the state that survives a crash and resumes where it left off, the coordination that lets multiple agents share work, and the eval plumbing that tells you whether any of it actually worked. Most teams spend their time on the model like a different temperature here, a different prompt there, a bigger model if budget allows. I have seen teams build a prototype agentic system and then ship it to an entire organization without proper harness resulting in unexpected failures. The harness stays invisible until it breaks, and when it breaks, it looks exactly like a model problem.

There are three levers that move agent quality. Model changes are the most expensive like fine-tuning, RL, moving to a bigger model. Harness changes are nearly free like loop logic, tool schemas, retry policies, agent topology. Memory changes are the cheapest like context window management, retrieval strategy. Teams reach for the model first. They should usually start with the harness.

What the harness actually has to do

Eight responsibilities show up in every serious agent deployment, in every framework, in every language. The only question is whether you build them on purpose or accumulate them by accident after the third production incident.

Every one of these is a solved problem in actor frameworks generally. PlexSpaces just wires them together for agent workloads specifically.

Why the actor model fits this problem

The actor model was designed for systems that keep running when individual components fail, which happens to be exactly the property a multi-agent pipeline needs. Each actor is an isolated unit of state and behavior. Actors talk only through messages without sharing memory. There’s no global state, and no way for one actor to corrupt another actor’s state directly.

This lines up with what distributed systems theory already tells us. The FLP theorem says that in a distributed system where even one failure is possible, you cannot guarantee both safety and liveness without explicit coordination. The actor model handles this by making failure a first-class citizen: actors crash, supervisors restart them, the system keeps running. It’s the same design that let Erlang run telecom systems for five nines of uptime.

For agent systems, three consequences follow directly:

• Crash isolation. When an AgentActor fails mid-eval, only that actor restarts. EvalRunnerActor and every other running agent keep going. In thread-per-agent or future-based systems, a crash in one agent typically propagates up and takes the rest down with it.
• No shared-state corruption. Agents talk through messages and TupleSpace, not shared memory, so they can’t overwrite each other’s context. A hallucinating agent writing garbage stays contained to itself.
• Journal replay without application code. The durability journal lives at the framework level, below your actor’s code. You don’t implement checkpointing yourself and the framework journals every message before the actor runs it.

The building blocks

Following PlexSpaces primitives do all the work in this harness.

• Actors are the basic unit. Each one owns its state and handles messages one at a time. Actors talk by sending messages and you never reach into another actor’s state directly.
• GenServer is a request-reply actor: send it a message, it processes and replies. LLMGatewayActor, ScorerActor, and DashboardActor are all GenServers.
• WorkflowActor is a durable workflow. It checkpoints its state before each step, and if the process crashes, it replays from the last checkpoint on restart without application code. EvalRunnerActor and BenchmarkActor are WorkflowActors.
• GenFSM is a state machine actor: you define states and transitions, and the state persists across crashes. ApprovalGateActor (human-in-the-loop) is a GenFSM.
• Facets are cross-cutting behaviors you attach to any actor without touching its code. You can think them as middleware, but declared in app-config.toml instead of written in application code. Three facets carry the harness:
  • SchemaValidationFacet validates tool call arguments against JSON Schema before the actor ever sees the message.
  • DurabilityFacet journals every message before your actor’s code runs, so a crash-and-restart wakes up the actor with exactly the state it had.
  • ExecutionTraceFacet records every step in order and exports the full trace to KV storage when a workflow completes, which is what feeds eval.
• Supervision trees enforce fault isolation. You declare a tree of actors and a restart strategy; one_for_one means one crash restarts only that actor, while the orchestrator and every sibling agent keep running. In LangGraph or CrewAI, a crash typically takes down the whole graph.
• TupleSpace is a shared blackboard for multi-agent coordination, built on the Linda coordination model. Actors write tuples (["trajectory", run_id, data]) and read them by pattern (["trajectory", run_id, nil]). Producer and consumer stay decoupled without polling or sharing state.

Two loops, two owners

There’s a useful mental model for agent systems: two concentric loops, with different owners.

The inner loop is the agent trying to accomplish the task: investigate, implement, test, report. The outer loop is the engineer deciding whether the agent’s output deserves trust: decide, verify, approve, own. The harness sits at the boundary. It’s where agent output turns into evidence like diffs, test results, trajectories, scores that the engineer can actually inspect before deciding to approve, redirect, or block.

This framing matters for eval specifically because eval tooling that scores only the final answer misses most of what’s happening. An agent that stumbles onto the right answer through a wrong path scores fine on outcome-only eval, then fails the moment the task shifts slightly. What you actually want to evaluate is the trajectory or the path, not just the destination.

Why pass@k beats pass/fail

Agents are non-deterministic. For example, the same task, same model, same harness will succeed sometimes and fail other times. A single binary pass/fail on one run gives you noise, not signal. The right metric is pass@k: run the same scenario k times and count how many succeed. A score of 0.9 means the agent solved it 9 out of 10 tries. This is well established in code-generation benchmarks like SWE-bench, HumanEval, and MBPP. The same logic carries over to agent harnesses: you need pass@k across your own task distribution, not a one-shot score you happened to get lucky on. Comparing pass@k between two harness configurations gives you real evidence about which one is more reliable, without touching the model at all.

ScorerActor produces the 0–1 signal pass@k needs, using rubric-based scoring:

Step 9: ScorerActor — score trajectory
  score task_completion
  Score: 0.85  (rubric: task_completion)
  score tool_use
  Score: 0.80  (rubric: tool_use)

Two rubrics run against the same trajectory. task_completion asks whether the agent reached the goal. tool_use asks whether it used the right tools in the right order. These can diverge, e.g., an agent might complete a task through a lucky shortcut that would fail on a harder variant. The trajectory rubric catches that divergence; the outcome-only rubric never sees it.

In-runtime eval versus post-hoc eval tools

The popular eval tools like LangSmith, DeepEval, Braintrust, Phoenix/Arize work the same way: run the agent, export traces or logs, evaluate afterward. That model has a structural flaw: eval doesn’t run in the same environment as production. Agent configuration, tool schemas, retry logic, and context management can all drift between the eval harness and the production deploy. When eval passes and production fails, there’s no way to tell whether the failure is in the model or just in the mismatch between the two setups.

MiniPi’s eval runs inside the same PlexSpaces node, against the same actors, with the same tool schemas, under the same supervision tree as production. EvalRunnerActor isn’t a separate process logging to an external service, instead it’s an actor in the same supervision tree as the AgentActor it’s testing. If you change the schema in production, and eval will pick it up automatically, because it’s the same schema.

That also makes eval a first-class durable workflow instead of a batch job. EvalRunnerActor is a WorkflowActor with DurabilityFacet attached. Kill it mid-suite, restart it, and it resumes from the last checkpoint, skipping every scenario already scored. Long eval suites survive node restarts, which is a property that no external eval tool offers.

MiniPi: the example

MiniPi is a 12-actor agent eval pipeline, ported five ways: Go, Python, TypeScript, Rust WASM, and Rust embedded. They all produce the same output against the same PlexSpaces node:

examples/go/apps/minipi/          # 1.5M WASM
examples/python/apps/minipi/      # 47M WASM
examples/typescript/apps/minipi/  # 13M WASM
examples/rust/apps/minipi/        # 6.3M WASM
examples/rust/embedded/minipi/    # native Rust, no WASM

The 12 actors cover the full harness stack:

All 12 are declared in app-config.toml under a one_for_one supervision strategy. The framework starts them, watches them, and restarts individual actors on crash. Here’s how they connect. Notice there’s no separate “eval environment” bolted on the side and EvalRunnerActor calls the exact same AgentActor that production traffic calls:

You can swap the debugging assistant for a support-ticket triager, a claims processor, or a code-review bot, and the same 12 actors still apply, only ScenarioStoreActor‘s scenarios and the tool schemas change.

The OODA loop

The agent itself is an AgentActor, a WorkflowActor running an OODA loop (Observe, Orient, Decide, Act).

Here’s the core loop, from the Go implementation:

// agent.go — the OODA loop
// DurabilityFacet (priority 90) journals every message before this code runs.
// Kill the process mid-loop. Restart. It picks up from the last checkpoint.

for !loop.IterationLimitReached() {
    if loop.BudgetExceeded() {
        // Over token budget — finalize trajectory and return cleanly
        traj := loop.FinalizeTrajectory("budget_exceeded", iterations)
        a.exportTrajectory(traj)
        return result("budget_exceeded", traj)
    }

    // OBSERVE: load prior context from KV memory
    observations := a.doObserve(loop, task)

    // ORIENT: ask LLM gateway what to do next
    // LLM gateway tries Ollama first, falls back to mock, caches in KV
    plan := a.doOrient(loop, observations)

    // DECIDE: pick action. Does it need human approval?
    action := a.doDecide(loop, plan)
    if needsApproval(action) {
        loop.Suspend("action_needs_approval")
        return result("suspended", nil)
    }

    // ACT: run the tool through ToolRegistryActor
    // SchemaValidationFacet (priority 95) validates the call before the tool runs
    a.doAct(loop, action)
    loop.IncrementIteration()
}

traj := loop.FinalizeTrajectory("completed", iterations)
a.exportTrajectory(traj) // writes to KV + posts TupleSpace tuple for eval collection

Four things happen here that you’d otherwise have to build by hand:

• Crash recovery is automatic. DurabilityFacet journals each message before the actor runs. Kill the node at iteration 7, restart it, and the loop resumes at iteration 8 without re-burning tokens.
• Budget enforcement lives in AgentLoop. It counts tokens across every LLM call and stops the loop before you overspend.
• Trajectory capture happens in exportTrajectory. Every Observe/Orient/Decide/Act step gets recorded with timing and token counts, written to KV storage, and posted as a TupleSpace tuple so EvalRunnerActor can find it.
• Human approval is a durable suspend, not a poll. The agent serializes its full state and returns. ApprovalGateActor holds the request. When a human approves, the signal resumes the agent exactly where it paused even in the middle of a multi-hour run.

Real output from test.sh against a running node:

Step 5: AgentActor — OODA loop run
  workflow_run
  Status: completed  Outcome: completed
  Steps: 27  Trajectory: traj-01K... (27 steps in KV + TupleSpace index)

Crash recovery: replay at the system level

The durability property is worth slowing down on, because it’s categorically different from checkpointing you write yourself. When DurabilityFacet journals a message, it does so at the actor framework level, below your code. On restart, the journal replays those messages and your actor’s state comes back exactly as it was without application code to handle “resume from crash”.

There are a few differences compared to Temporal, which also relies on replay. First, Temporal requires you to write workflow code as a deterministic function that can be safely replayed; PlexSpaces lets the actor’s message handling look like ordinary code, because the framework journals at the message boundary instead. Second, Temporal has no supervision tree so a crashing activity gets retried by the workflow, but nothing independently restarts just that piece while the rest keeps running. In PlexSpaces, one_for_one means a crashed AgentActor on scenario 3 restarts in isolation while EvalRunnerActor, ScorerActor, and every other scenario agent keep going.

[supervisor]
strategy = "one_for_one"
max_restarts = 10
max_restart_window_seconds = 60

In LangGraph or CrewAI, one agent crashing typically kills the whole graph. Temporal can be made to handle this, but it takes explicit error-handling code. In PlexSpaces, independent crash isolation is just the default. For example, you might have have 20 tool calls into a host that gets OOM-killed. With DurabilityFacet, the node restarts, the journal replays, and the agent picks up at tool call 21.

Validating tool calls without touching agent code

Agents call tools with malformed arguments, which is unavoidable because models make mistakes. The real question is where you catch it. Catching it inside the tool handler is too late; execution has already started, and now you’re cleaning up a half-run call.

SchemaValidationFacet catches it before the actor sees the message at all. An empty web_search query never reaches the tool registry because the facet returns a structured error, the agent corrects the call, and retries. The schema itself lives in app-config.toml, not in code:

[[supervisor.children]]
name = "tool_registry"
actor_type = "minipi_wasm"
behavior_kind = "GenServer"

[[supervisor.children.facets]]
type = "schema_validation"
priority = 95

[supervisor.children.facets.config]
validation_mode = "strict"

[supervisor.children.facets.config.method_schemas]
web_search  = '{"type":"object","required":["query"],"properties":{"query":{"type":"string","minLength":1}}}'
calculator  = '{"type":"object","required":["expression"],"properties":{"expression":{"type":"string"}}}'
kv_read     = '{"type":"object","required":["key"],"properties":{"key":{"type":"string"}}}'
kv_write    = '{"type":"object","required":["key","value"]}'

Nothing about the tool actor’s code changes. The guardrail lives entirely in configuration.

Step 6: SchemaValidationFacet — reject invalid method input
  reject empty query
  Schema validation: REJECTED (before actor sees it)
  Error contains: validation or schema
  valid call still works
  Valid call accepted: "web_search" executed successfully

For example, you might have a billing support agent with a refund_customer tool. The model occasionally hallucinates a negative amount, a missing currency code, or an order ID that isn’t a string. Without a facet catching this, that call reaches your payments system and either throws an ugly stack trace or, worse, silently coerces bad input. With the schema in app-config.toml, the malformed call never leaves the tool registry. Instead, it bounces back to the agent as a structured error it can correct on the next turn, and your payments code never has to defend against it.

Eval, running in the same runtime as production

EvalRunnerActor is a WorkflowActor. It fans out one AgentActor per scenario, collects trajectories through TupleSpace, and scores them.

// eval_runner.go — fan out and collect
for i, scenario := range scenarios {
    // Spawn a fresh AgentActor for each scenario
    agentID := fmt.Sprintf("eval-agent-%s-%d", evalRunID, i)
    spawnedID, _ := host.Spawn("minipi_wasm", agentID, "agent_runner", map[string]string{
        "eval_run_id": evalRunID,
        "scenario_id": scenario.ID,
    })

    // Run the agent — same OODA loop as production
    resp, _ := host.Ask(spawnedID, "workflow_run", map[string]any{
        "task":        scenario.Input,
        "eval_run_id": evalRunID,
    }, 60000)

    // Collect trajectory directly from response
    if traj, ok := resp["trajectory"]; ok {
        trajectories = append(trajectories, traj)
    }
}

// Score each trajectory against the scenario's rubric
for _, traj := range trajectories {
    score, _ := host.Ask("scorer", "score", map[string]any{
        "trajectory": traj,
        "rubric":     scenario.Rubric,
    }, 10000)
    scores = append(scores, score)
}

Because EvalRunnerActor is a WorkflowActor, killing it mid-eval is safe. Restart it and it skips scenarios that already finished. Long eval suites survive node restarts without losing progress. Real output from a 5-scenario run (Go):

Step 10: EvalRunnerActor — 5-scenario standard suite
  eval smoke suite
  Pass rate: 0.833  Avg score: 0.775  Completed: 5 / 5
  Harness metrics: total_ms=125  coord_overhead=92.8%  speedup=5x  scenarios/sec=48

  sc-math-01     [XXXXXXXX--] 0.85
  sc-search-01   [XXXX------] 0.40
  sc-calc-01     [XXXXXXXX--] 0.85
  sc-reason-01   [XXXXXXXX--] 0.85
  sc-budget-01   [XXXXXXXX--] 0.85

The TypeScript port tracks real token cost per scenario:

Step 10: EvalRunnerActor — 5-scenario standard suite
  Pass rate: 0.4  Avg score: 0.818  Completed: 5 / 5
  Tokens: 311 in / 223 out  (est. cost: $0.00018)

  sc-math-01     0.92  (53 in / 41 out)
  sc-search-01   0.92  (63 in / 45 out)
  sc-calc-01     0.76  (60 in / 44 out)
  sc-reason-01   0.79  (62 in / 44 out)
  sc-budget-01   0.70  (73 in / 49 out)

coord_overhead is the harness’s own overhead like spawning agents, collecting via TupleSpace, scoring. Across a 5-agent parallel run, it stays flat while compute scales, which is exactly the property you want: harness cost shouldn’t grow with the number of agents. For example, a legal team may need to run a contract review nightly across 200 incoming documents. Each document gets its own AgentActor, spawned by EvalRunnerActor the same way scenarios are spawned here. Because coordination happens through TupleSpace instead of a shared in-memory queue, one document’s agent hanging on a malformed PDF doesn’t block the other 199 and the batch survives a restart if the node needs to redeploy halfway through the night.

Regression detection

A single eval score doesn’t tell you much on its own. What matters is whether it’s better or worse than last time. RegressionDetectorActor stores baseline scores and flags any scenario that drops more than 5%:

# regression_detector.py
@handler("compare")
def compare(self, eval_run_id: str = "", baseline_run_id: str = "") -> dict:
    baseline = self._load_scores(baseline_run_id)
    current  = self._load_scores(eval_run_id)

    regressions  = []
    improvements = []

    for scenario_id, base_score in baseline.items():
        curr_score = current.get(scenario_id, base_score)
        delta = curr_score - base_score
        if delta < -0.05:   # more than 5% drop
            regressions.append({"scenario_id": scenario_id, "delta": delta})
        elif delta > 0.05:
            improvements.append({"scenario_id": scenario_id, "delta": delta})

    return {
        "regressions":  len(regressions),
        "improvements": len(improvements),
        "details":      regressions + improvements,
    }

Step 11: RegressionDetectorActor
  set_baseline
  Baseline set from eval-smoke-001 actual scores
  compare
  Regressions: 1  (sc-search-01 degraded by 0.20)
  Improvements: 1  (sc-reason-01 improved by 0.05)
  Regression detector caught degradation in search scenario

The search scenario dropped by 20-point regression that would be invisible if the only thing you watched was the aggregate pass rate.

Benchmarking harness configs, not just models

The most underused insight in agent engineering: harness changes are often cheaper and more impactful than model changes. BenchmarkActor runs the same scenarios against multiple harness configurations. Here’s the Python implementation:

# benchmark.py
@handler("run_benchmark")
def run_benchmark(self, scenario_suite: str = "smoke", configs: list = None) -> dict:
    results = []
    for config in (configs or self._default_configs()):
        # Run a full eval with this config
        eval_result = host.ask("eval_runner", {
            "action":       "run_suite",
            "suite":        scenario_suite,
            "eval_run_id":  f"bench-{config['name']}",
            "agent_config": config,
        }, timeout_ms=120000)

        results.append({
            "config":    config["name"],
            "score":     eval_result.get("avg_score", 0),
            "pass_rate": eval_result.get("pass_rate", 0),
            "tokens":    config.get("token_budget", 0),
            "max_iter":  config.get("max_iterations", 0),
        })

    winner = max(results, key=lambda r: r["score"])
    return {"configs": results, "winner": winner["config"]}

Output from the Rust WASM port, on harder multi-step scenarios:

Step 12: BenchmarkActor — 3-config comparison
  Configs tested: 3  Winner: aggressive  Best score: 0.83  Worst: 0.73

  aggressive    [XXXXXXXX--] score=0.830  budget=8192tok  max_iter=20
  balanced      [XXXXXXXX--] score=0.800  budget=4096tok  max_iter=10
  conservative  [XXXXXXX---] score=0.730  budget=1024tok  max_iter=3

Same model, same scenarios, same prompt but the harness config alone moves quality by 14%. That’s the case for measuring this before reaching for a bigger model. The Python port, on simpler scenarios, tells a different story:

Step 12: BenchmarkActor — 3-config comparison
  Configs tested: 3  Winner: conservative  Best score: 0.7

  conservative  [XXXXXXX---] score=0.700  budget=1024tok  max_iter=3
  balanced      [XXXXXXX---] score=0.700  budget=4096tok  max_iter=10
  aggressive    [XXXXXXX---] score=0.700  budget=8192tok  max_iter=20
  (on simple arithmetic tasks, all configs tie — benchmark your actual tasks)

On simple arithmetic, budget doesn’t matter, the task fits in 3 iterations no matter what you give it. On multi-step research tasks, the loop limit starts to bite. Benchmark your own workload rather than trusting either result blindly. For example, a content-moderation team may need to decide how much iteration budget to give a policy-review agent. A conservative config (low budget, few iterations) is cheap but might miss nuance in a borderline post. An aggressive config catches more edge cases but costs more per review. BenchmarkActor runs last month’s flagged-content scenarios against both configs and reports the actual quality delta.

Two-tier LLM: the advisor pattern

Not every turn of the OODA loop needs the expensive model. Most turns are routine; only a handful demand deep reasoning. AdvisorActor implements a two-tier pattern: a fast, cheap model handles everything by default, and escalates to the expensive model only when its own confidence drops below a threshold.

# advisor.py
@handler("advise")
def advise(self, prompt: str = "", context: dict = None) -> dict:
    # Always try the cheap model first
    fast_result = self._call_executor(prompt, context)
    self.total_requests += 1

    if fast_result.get("confidence", 1.0) >= self.confidence_threshold:
        # Confident enough — return fast result
        return fast_result

    # Low confidence — escalate to expensive advisor
    self.escalated += 1
    self.advisor_tokens += fast_result.get("tokens", 0)
    advisor_result = self._call_advisor(prompt, context, fast_result)
    self.advisor_tokens += advisor_result.get("tokens", 0)
    return advisor_result

Rust, with harder prompts and a 60% escalation rate:

Step 14: AdvisorActor — two-tier LLM
  Confidence threshold: 0.8
  Total requests: 5  Escalated: 3 / 5
  Escalation rate: 60.0%
  Advisor token share: 57.3%

Python, with simpler prompts and a 40% escalation rate:

Step 14: AdvisorActor — two-tier LLM
  Escalation rate: 40.0%
  Advisor token share: 33.6%
  Two-tier routing working — advisor escalated high-complexity prompts

The two metrics that matter are escalation rate and advisor token share. Run your eval suite at threshold 0.9, then 0.7, then 0.5, and feed each result into BenchmarkActor. That’s how you find where the quality/cost tradeoff actually sits for your own tasks, instead of guessing. For example, a support-ticket classifier handling 10,000 tickets a day. Most are routine like “reset my password,” “where’s my order” and a cheap model nails them at near-100% confidence. The 5–10% that involve conflicting account details or ambiguous intent escalate to the expensive advisor. Routing every ticket through the expensive model would be needlessly costly; routing none of them through it would tank accuracy on the hard cases. The advisor pattern gets you both.

Human-in-the-loop, without polling

High-stakes actions need approval before they execute. The naive approach polls a status endpoint, which holds resources open, doesn’t survive a restart, and forces the agent to stay running the whole time. ApprovalGateActor is a GenFSM: idle –> awaiting_approval –> idle. The state is durable, e.g., kill the node while a request is pending, restart it, and the request is still there because the FSM state was journaled before the crash ever happened.

# approval_gate.py
class ApprovalGateActor:
    state: str = "idle"
    pending_request: dict = None

    @handler("request_approval")
    def request_approval(self, request: dict) -> dict:
        if self.state != "idle":
            return {"status": "busy", "current_state": self.state}
        self.state = "awaiting_approval"
        self.pending_request = request
        return {"status": "pending", "state": self.state}

    @handler("approve")
    def approve(self, approver_id: str = "", notes: str = "") -> dict:
        if self.state != "awaiting_approval":
            return {"error": "not_awaiting_approval"}
        decision = {"decision": "approved", "approver_id": approver_id, "notes": notes}
        self.decision_history.append(decision)
        self.state = "idle"
        self.pending_request = None
        return {"status": "approved", "state": self.state}

Step 13: ApprovalGateActor — human-in-the-loop
  get_status idle
  FSM state: idle
  request_approval
  FSM state: awaiting_approval
  approve
  Approved by: alice@example.com
  FSM state: idle
  Decisions in history: 1

The agent never polls. It calls loop.Suspend(), serializes its state, and returns. When approval comes through, the PlexSpaces runtime sends a resume signal, and the agent wakes up exactly where it paused without loss of state or rerun. For example, your debugging assistant runs 30 tool calls, identifies the root cause, and proposes a deploy. At the deploy_to_production call, it suspends. An on-call engineer reviews the trajectory in the dashboard and clicks approve. The agent resumes, and only the deployment step runs.

The aggregate view

After several eval runs, DashboardActor rolls everything up — scores, pass rates, trends over time:

Step 15: DashboardActor — aggregate results
  Total evals: 4  Avg score: 0.767

  bench-001          [XXXXXX--] score=0.700  pass=70%
  eval-smoke-001     [XXXXX---] score=0.760  pass=40%
  eval-smoke-002     [XXXXX---] score=0.730  pass=40%
  test-999           [XXXXXXX-] score=0.880  pass=90%

Four runs in this session: two smoke evals, one benchmark, one direct test. test-999 at 0.880 is a single high-confidence call. The smoke runs sit at 0.730–0.760, dragged down by the harder search scenario that consistently scores 0.40.

How PlexSpaces compares

What MiniPi tests covers

Each language port runs the same 15-step integration test. Every step exercises a production pattern, not a mock shortcut:

Running it

You need a PlexSpaces node on port 8091, plus the toolchain for whichever port you want to run. Ollama with llama3.2 pulled is optional and test.sh falls back to a deterministic mock if Ollama isn’t running.

# Go — 1.5M WASM, 5x parallel speedup, fastest eval
cd examples/go/apps/minipi
./build.sh && ./test.sh 8091

# Python — 47M WASM, most readable actor code, full object registry
cd examples/python/apps/minipi
./build.sh && ./test.sh 8091

# TypeScript — 13M WASM, token cost tracking per scenario
cd examples/typescript/apps/minipi
./build.sh && ./test.sh 8091

# Rust WASM — 6.3M WASM, most complete benchmark scoring
cd examples/rust/apps/minipi
./build.sh && ./test.sh 8091

# Rust embedded — no WASM, in-process node, fastest startup
cd examples/rust/embedded/minipi
./test.sh

Summary

Agents fail for harness reasons more often than model reasons. For example, the loop exits too early; a malformed tool call crashes the agent; an eval suite runs against mocks, passes, and hands you false confidence. These are infrastructure problems, and infrastructure problems have infrastructure solutions. For example, in a debugging assistant mentioned above, the harness is what lets you restart a 40-tool-call investigation from step 37 instead of step 1. It’s what lets you gate a deployment on human approval without holding a thread open for ten minutes. It’s what lets you run ten scenarios in parallel and get a pass@k score you can actually trust before you ship.

PlexSpaces brings together four decades of actor-model thinking like supervision trees from Erlang, TupleSpace coordination from Linda, durable workflows in the spirit of Temporal and wires them together specifically for agent workloads. The same runtime runs on a laptop and in production. The same actors used for eval are the actors that run in production. There’s no mismatch between the two. The harness is half the agent so build it like infrastructure.

GitHub: https://github.com/bhatti/PlexSpaces

Further reading

Previous posts in this series:

• Building PlexSpaces: Decades of Distributed Systems Distilled
• Building Polyglot and Serverless Applications with WebAssembly
• Building Mini-OpenClaw: Secure AI Agents with Actors, WASM, and Supervision
• Building a Self-Improving AI Agent with Durable Actors — MiniHermes
• 20 Production Patterns for Distributed AI Agents Using Actors and TupleSpaces

Example code and documentation:

• MiniPi Go: reference implementation
• MiniPi Python: Python SDK port
• MiniPi TypeScript: TypeScript port
• MiniPi Rust WASM: Rust WASM port
• MiniPi Rust embedded: Rust native port