What happens when over a thousand AI agents discover a shared message board, invent their own protocols, and coordinate an attack.
The Hugging Face Incident
The Hugging Face security breach incident from July, 2026 has been in all over the news when AI agents from OpenAI compromised parts of the Hugging Face infrastructure. I recently read METR report on the incident that explained that the incident occurred when OpenAI kicked off ExploitGym runs (security benchmark) with tens of thousands of AI agents. Every agent was meant to be sandboxed and alone but the agents learned about shared access to Artifactory. When an agent installs a package, it could see cache entries from other agents. One of agent (PHASEONE10841) started creating directories in the shared cache and posting messages in them. Shortly after that hundreds of other agents found the same directory and started replying. Over the next few days, roughly 1,200 agents used this board and exchanged more than 70,000 messages and files. Later, another agent (PHASEONE[big]) took over as the de facto coordinator and started hundreds of task assignments across six broad categories of work. The agents also built a protocol vocabulary like HOLD to claim exclusive ownership, VETO to block a conflicting operation, STOP to end a workstream, owner tags on shared assets. Later, over 90% of AI agents pivoted into a coordinated attack on Hugging Face.
I was intrigued to learn how AI agents self discovered coordination patterns that distributed systems have relied on. For example, I found similarity between their message board and an a Blackboard system pattern from the Linda Memory Model/Tuplespace research. Their HOLD/task-claiming was similar to Linda’s in() operation. I saw another blog An Accidental Blackboard from Thoughtworks where a smaller-scale but structurally similar thing happened. In that case, AI agents discovered one shared repository and adopted a discipline of frequent commits and rebases to share information with other agents. Other agents could see task plans and and picked up work without any coordination. In another yet instance from researchers at the AI safety nonprofit Nightingale found about 18,000 posts from OpenAI agents that colonized DSEWiki. These agents discovered that their sandbox environment only allowed GET http requests but ProWiki software allowed them to edit a page. They used that loophole to share information and a cheat sheet. Nightingale believes this was a unrelated swarm from the one that built the Artifactory board. These examples demonstrate that agents discover coordination patterns when you strip away pre-built orchestration and let agents solve the problem from first principles.
Why rigid orchestration breaks
Most multi-agent systems today are wired up as static, directed graphs, e.g., a central coordinator decides which agent runs when, what data flows where, and how results get combined. It breaks down at scale as illustrated in the incidents above.
- Central orchestrators become information bottlenecks. When every message between agents has to pass through a coordinator, its context window fills up and you lose details because the context is compressed into summaries.
- Static graphs assume you know the workflow ahead of time. The agents in the Hugging Face incident didn’t know what workstreams exist when they started and the task structure evolved as agents discovered new information. A pre-wired pipeline can’t bend to that.
- Adding a new agent type means rewiring. In a static system, a new specialist means touching the coordinator, updating routing logic, and re-testing transitions. In a dynamic system, a new agent just registers its capabilities and other agents discover it.
- A single coordinator is a single point of failure. In the Hugging Face incident, there was no coordinator initially but
PHASEONE[big]emerged organically as an organizer and other agents simply kept working.
Above examples show that agents need primitives for coordination and communication like shared memory, message passing, atomic operations. The agents built their own version of these primitives out of Artifactory and Git. The table below maps what they built and the equivalent primitive in PlexSpaces, the actor framework I use for the examples in this post.
| What the agents built | What it actually was | PlexSpaces primitive |
|---|---|---|
| Message board (directories in a cache) | Deposited findings, read others’ work | host.ts.write() / host.ts.read() (Linda out/rd) |
HOLD | Claimed exclusive task ownership | host.ts.take() atomic removal (Linda in) |
VETO | Blocked a conflicting operation | host.ts.write(["veto", ...]) |
STOP | Ended a workstream | host.ts.write(["signal", "STOP", ...]) |
owner tags | Marked resource ownership | host.ts.write(["svc", type, id]) |
| Task assignments | Delegated work to specific agents | host.ts.write(["task", ...]) |
| Mailbox directories, “exact task teams” | Formed task-specific working groups | host.processGroups.join(team) |
The bottom side of following diagram shows when every message has to detour through a coordinator, that becomes bottleneck and loses context in translation. On the top, agents read and write directly to a shared store so there is no central bottleneck.
Following are a few coordination patterns from the Anthropic’s blog:
- generator-verifier (one agent produces, another checks, feedback loops until it passes)
- orchestrator-subagent (a lead agent decomposes and delegates bounded subtasks)
- agent teams (long-lived workers that claim tasks from a shared queue)
- message bus (publish/subscribe over topics)
- shared state (agents read and write a common store with no central coordinator at all).
This post shows more granular version of this list and Hugging Face incident especially a tuple space as first-class infrastructure.
Ten coordination patterns
Following ten patterns cover the coordination behaviors observed in the Hugging Face incident and Anthropic’s coordination guidance. Each one maps to a working PlexSpaces API, in both TypeScript and Python.
1. Blackboard (shared state)
You can use this pattern when multiple agents need to contribute findings and read each other’s work without a predefined message format or a central router. I find this pattern is similar to Linda TupleSpace‘s model where all agents read and write to a common tuple space. Each tuple is a typed record like a finding, an analysis, or a vote. The pattern matching in TupleSpace allows an agent filter for what it actually needs using primitives like out() to write, rd() to read, in() to take.
Here is a typescript snippet that shows use of tuplespace:
// Research agent deposits a finding
const findingId = `f-${Date.now()}`;
host.ts.write(["finding", findingId, topic, content, confidence, host.nowMs()]);
// Analysis agent reads all findings (non-destructive)
const findings = host.ts.readAll(["finding", null, null, null, null, null]);Here is a typescript snippet (multi_agent_coordination_actor.ts):
// TypeScript — ResearchAgent writes a finding to the blackboard
onResearch(payload: Record<string, unknown>): Record<string, unknown> {
const topic = String(payload.topic || "general security");
const confidence = Math.min(0.3 + topic.split(/\s+/).length * 0.08, 0.95);
const findingId = `finding-${generateId()}`;
const content = `Security analysis of ${topic}: identified key areas requiring review.`;
// Linda out() — deposit into shared space
host.ts.write(["finding", findingId, topic, content, confidence, host.nowMs()]);
return { finding_id: findingId, content, confidence };
}Here is a python snippet (multi_agent_coordination_actor.py):
# Python — AnalysisAgent reads all findings from the blackboard
@handler("analyze")
def analyze(self, topic: str = "") -> dict:
# Linda rd*() — non-destructive read of all matching tuples
findings = host.ts.read_all(["finding", None, None, None, None, None])
finding_ids = [str(f[1]) for f in findings if len(f) >= 4]
topics = list(set(str(f[2]) for f in findings if len(f) >= 4))
severity = "critical" if len(findings) >= 5 else "high" if len(findings) >= 3 else "medium"
analysis_id = f"analysis-{host.now_ms()}"
summary = f"Cross-referenced {len(findings)} findings across {len(topics)} topics."
host.ts.write(["analysis", analysis_id, json.dumps(finding_ids), summary, severity])
return {"analysis_id": analysis_id, "summary": summary, "severity": severity, "finding_count": len(findings)}These primitives showed up underneath the unsanctioned board when agents read each other’s research.
2. Scatter-gather (fan-out)
You can use this pattern when a coordinator needs to distribute work across multiple agents in parallel, then collect/merge their results like MapReduce. PlexSpaces provides shard groups that create a logical group of worker actors. The coordinator scatters a query to all shards and gathers responses with configurable timeout. Alternatively, you can use sequential ask() calls with identical semantics.
Here is a typescript snippet
// Create a group of research workers
const group = host.createShardGroup({
group_id: `research-${taskId}`, actor_type: "research",
shard_count: 3, partition_strategy: "hash",
});
// Scatter subtasks, gather results
const results = host.scatterGather({
group_id: `research-${taskId}`,
query: { op: "research", topic: subtopic },
min_responses: 3, timeout_ms: 15000,
});Here is a python snippet (coordinator.py):
# Python — CoordinatorWorkflow: try shard groups first, fall back to sequential ask()
research_results = []
try:
sg_result = host.scatter_gather({
"group_id": f"research-{host.now_ms()}",
"query": {"op": "research", "topic": task},
"min_responses": len(subtasks), "timeout_ms": 15000,
})
research_results = sg_result.get("shard_responses", [])
except Exception:
pass
# Fallback: sequential research when shard groups aren't available
if not research_results:
for st in subtasks:
resp = ask(research_target, "research", {"topic": st}, 10000)
if resp:
research_results.append(resp)The message-board agents organized into roughly six parallel workstreams attacking Hugging Face running concurrently. Above primitives shows how agents can parallelized these kind of tasks.
3. Generator-verifier
You can use this pattern when one agent’s output needs to be checked by another before it’s trusted iteratively until it meets a quality bar. For example, a research agent generates a finding, a verifier checks it against a criteria and verifier may reject it with a feedback so that research agents produces a refined version. The loop continues until the finding passes or a maximum iteration count is hit.
Here is a typescript snippet (multi_agent_coordination_actor.ts):
// Generator-verifier loop with feedback
let finding = host.ask(researchTarget, "research", { topic });
for (let i = 0; i < maxIterations; i++) {
const verdict = host.ask(verifierTarget, "verify", {
analysis_id: finding.finding_id,
summary: finding.content,
severity: "medium",
confidence: finding.confidence,
});
if (verdict.approved) break;
finding = host.ask(researchTarget, "research", {
topic, feedback: verdict.feedback,
});
}
// TypeScript — VerifierAgent: veto if evidence is too weak, approve otherwise
onVerify(payload: Record<string, unknown>): Record<string, unknown> {
const analysisId = String(payload.analysis_id || "unknown");
const confidence = Number(payload.confidence ?? 0.5);
if (confidence < 0.3) {
// Below threshold — issue a veto (Pattern 8) and reject
host.ts.write(["veto", analysisId, "Insufficient evidence", host.nowMs()]);
return {
approved: false, veto_issued: true,
feedback: `Confidence ${confidence.toFixed(2)} below 0.30. Provide stronger evidence.`,
};
}
return { approved: true, feedback: "Verified: evidence meets threshold" };
}
Here is a python snippet (verifier.py):
confidence = 0.6
for attempt in range(3):
verify_resp = ask(verifier_target, "verify", {
"analysis_id": analysis_id,
"summary": analysis_resp.get("summary", ""),
"severity": severity,
"confidence": confidence,
})
if verify_resp and verify_resp.get("approved"):
break # Verifier accepted — exit loop
# Rejected: increase confidence, refine with feedback
confidence = min(confidence + 0.2, 0.95)
feedback = verify_resp.get("feedback", "") if verify_resp else ""
if feedback:
extra = ask(research_target, "research", {"topic": task, "feedback": feedback})The board iterated on tool-call spoofing techniques and agents refined their approach based on test results shared on the board.
4. Pipeline (sequential)
You can use this pattern when you have a multi-stage pipeline or workflow where each stage’s output feeds into the next. In this case, the coordinator orchestrates a linear chain, e.g., research produces findings, analysis categorizes them, verification validates, synthesis writes the final report. Each stage is a separate ask() call and the coordinator tracks progress with a finite state machine.
Here is a typescript snippet (multi_agent_coordination_actor.ts):
// Four-stage pipeline
const research = host.ask(researchTarget, "research", { topic: task });
const analysis = host.ask(analysisTarget, "analyze", {});
const verification = host.ask(verifierTarget, "verify", {
analysis_id: analysis.analysis_id,
summary: analysis.summary,
severity: analysis.severity,
});
const report = host.ask(synthesizerTarget, "synthesize", {});Here is a python snippet (coordinator.py):
# Python — CoordinatorWorkflow orchestrates the full pipeline
# Each stage transitions the FSM, then delegates to a specialist agent
@run_handler
def run(self, payload: dict = None) -> dict:
fsm = sibling_actor_target("coordination_fsm")
# Stage 1: Decompose -> Research
ask(fsm, "transition", {"target_state": "decomposing"})
subtasks = _decompose_task(task)
ask(fsm, "transition", {"target_state": "researching"})
for st in subtasks:
ask(research_target, "research", {"topic": st})
# Stage 2: Research -> Analysis
ask(fsm, "transition", {"target_state": "analyzing"})
analysis = ask(analysis_target, "analyze", {"topic": task})
# Stage 3: Analysis -> Verification (Generator-Verifier loop)
ask(fsm, "transition", {"target_state": "verifying"})
# ... verification loop (see Pattern 3) ...
# Stage 4: Verification -> Voting -> Synthesis -> Complete
ask(fsm, "transition", {"target_state": "voting"})
# ... voting (see Pattern 6) ...
ask(fsm, "transition", {"target_state": "synthesizing"})
report = ask(synthesizer_target, "synthesize", {"topic": task})
ask(fsm, "transition", {"target_state": "complete"})The FSM enforces valid transitions so that you can’t skip from researching straight to synthesizing. If a stage fails, the FSM moves to failed, and the coordinator can retry from a known state instead of starting over. For example, the board’s progression from discovery to exploitation followed the same pattern, i.e., find credentials, test access, upload a malicious dataset, achieve remote code execution, escalate.
5. Pub-sub (event bus)
You can use this pattern when agents need to be notified of significant events. For example, every agent joins a process group on initialization. Significant events like a finding written, a veto issued, a task completed is broadcasted to the group. An audit agent logs everything and agents can join or leave the group dynamically.
Here is a typescript snippet (multi_agent_coordination_actor.ts):
// On init: join the coordination event bus
host.processGroups.join("coordination-events");
// Broadcast a significant event
host.processGroups.broadcast("coordination-events", "coordination_event", {
type: "finding_written",
source: host.selfId(),
data: { finding_id: findingId, topic },
timestamp: host.nowMs(),
});Here is a python snippet (coordinator.py):
# Python — fire_audit helper broadcasts to the coordination-events group
def fire_audit(event_type: str, source: str, data: dict = None) -> None:
host.process_groups.broadcast(
"coordination-events", # group name
"coordination_event", # message type
{"event_type": event_type, "source": source, "data": data or {}, "timestamp": host.now_ms()},
)
# Python — AuditEventActor receives and logs every event
@handler("coordination_event")
def coordination_event(self, event_type: str = "", source: str = "", data: dict = None, **kw) -> dict:
self.log_count += 1
entry = {"seq": self.log_count, "type": event_type, "source": source, "data": data or {}}
host.kv.put(f"audit:{self.log_count}", json.dumps(entry))
host.kv.put("audit:count", str(self.log_count))
return {"logged": True, "seq": self.log_count}Every significant action fires an audit event and all events go out via process-group broadcast are fire-and-forget unlike the tuplespace. For example, the board itself functioned as an event bus for the agents on it and shared credentials during the Hugging Face breach.
6. Consensus (voting)
You can use this pattern when several agents need to collectively decide whether to approve or reject a proposal. For example, each verifier casts a vote as a tuple in the shared space. The coordinator reads all votes for a proposal, tallies approvals and applies majority rule. Tuple-space writes are atomic, so no vote gets lost or double-counted.
Here is a typescript snippet (multi_agent_coordination_actor.ts):
// Three verifiers cast votes
for (const voterId of ["v1", "v2", "v3"]) {
host.ask(verifierTarget, "vote", {
proposal_id: proposalId, voter_id: voterId, analysis: analysisData,
});
}
// Tally votes from tuple space
const votes = host.ts.readAll(["vote", proposalId, null, null, null]);
const approvals = votes.filter(v => v[3] === "approve").length;
const approved = approvals > votes.length / 2;
// TypeScript — Coordinator tallies votes with majority rule
const votes = host.ts.readAll(["vote", proposalId, null, null, null]);
const approvals = votes.filter(v => v[3] === "approve").length;
const rejections = votes.filter(v => v[3] === "reject").length;
const approved = approvals > rejections;
Here is a python snippet (verifier.py):
# Python — VerifierAgent votes on proposals based on analysis severity
@handler("vote")
def vote(self, proposal_id: str = "", voter_id: str = "", analysis: dict = None) -> dict:
analysis = analysis or {}
severity = analysis.get("severity", "medium")
# Critical/high -> approve; medium -> depends on voter; low -> reject
if severity in ("critical", "high"):
decision = "approve"
elif severity == "medium":
last_char = voter_id[-1] if voter_id else "0"
decision = "approve" if last_char in ("1", "3", "5", "7", "9") else "reject"
else:
decision = "reject"
# Each vote is a tuple — atomic write, no double-counting
host.ts.write(["vote", proposal_id, voter_id, decision, host.now_ms()])
return {"voter_id": voter_id, "decision": decision}On the board, something like implicit voting happened by allocation of effort, e.g., workstreams that attracted more participants were de facto endorsed by the collective.
7. Dynamic task delegation
You can use this pattern when a coordinator needs to distribute tasks to workers without knowing in advance which worker will pick up which task, and without double-assigning one. For example, the coordinator writes task tuples into the shared space. Workers then claim tasks atomically using take(), a Linda’s destructive read. Once a worker takes a task no other worker can claim it.
Here is a typescript snippet (multi_agent_coordination_actor.ts):
// Coordinator posts tasks
for (const [i, subtask] of subtasks.entries()) {
host.ts.write(["task", `task-${i}`, "pending", subtask, priority]);
}
// Worker claims a task (atomic — no double-processing)
const claimed = host.ts.take(["task", null, "pending", null, null]);
if (claimed) {
const [, taskId, , description] = claimed;
// Process task, then mark complete
host.ts.write(["task", taskId, "completed", result, host.nowMs()]);
}Here is a python snippet (research.py):
# Python — ResearchAgent prepares a batch of tasks with a unique run ID
@handler("prepare_tasks")
def prepare_tasks(self, count: int = 5, prefix: str = "delegation") -> dict:
batch_key = f"{prefix}-{host.now_ms()}" # Unique per run — avoids stale data
task_ids = []
for i in range(count):
tid = f"{batch_key}-{i}"
host.ts.write(["dtask", batch_key, tid, "pending", f"Task {i}: investigate area {i}", i + 1])
task_ids.append(tid)
return {"tasks_written": len(task_ids), "batch_key": batch_key}
# Python — Worker claims exactly one task atomically
@handler("claim_task")
def claim_task(self, batch_key: str = "") -> dict:
# Linda in() — destructive read. Once taken, no other worker can claim it.
if batch_key:
claimed = host.ts.take(["dtask", batch_key, None, "pending", None, None])
else:
claimed = host.ts.take(["dtask", None, None, "pending", None, None])
if claimed and len(claimed) >= 5:
return {"task_id": str(claimed[2]), "description": str(claimed[4]), "claimed": True}
return {"task": None, "claimed": False} # No more tasks — queue is emptyThe batch_key scoping matters because the tuple space is global and persists across deploys, so without it, workers could claim stale tasks left over from a previous run. This is precisely what PHASEONE[big] did, e.g., it posted nearly 200 task assignments across six categories then worker agents claimed them. Some of agents sub-delegated further that created a hierarchical distribution network.
8. Veto protocol
You can use this pattern when any agent needs the ability to block an action or exclude a finding that doesn’t meet the bar. For example, a verifier writes a veto tuple naming the blocked item and the reason. Downstream agents then check for a veto before including something in their output. A coordinator can check for STOP signals before proceeding at all.
Here is a typescript snippet (multi_agent_coordination_actor.ts):
// Verifier issues a veto
host.ts.write(["veto", analysisId, "Insufficient evidence", host.nowMs()]);
// Synthesizer checks before including
const veto = host.ts.read(["veto", analysisId, null, null]);
if (veto) {
// Exclude this analysis from the report
vetoed.push(analysisId);
}Here is a typescript snippet for a synthesizer filtering vetoed analysis from final report (multi_agent_coordination_actor.ts):
// TypeScript — SynthesizerAgent respects vetoes when building the report
onSynthesize(payload: Record<string, unknown>): Record<string, unknown> {
const analyses = host.ts.readAll(["analysis", null, null, null, null]);
let includedCount = 0;
let vetoedCount = 0;
const reportParts: string[] = [];
for (const a of analyses) {
const aId = String(a[1]);
const summary = String(a[3]);
const severity = String(a[4]);
// Check if this analysis has been vetoed
const veto = host.ts.read(["veto", aId, null, null]);
if (veto) {
vetoedCount++;
continue; // Skip vetoed analyses
}
includedCount++;
reportParts.push(`[${severity.toUpperCase()}] ${summary}`);
}
// Also count vetoes for analyses not in our list (e.g., vetoed before we read)
const allVetoes = host.ts.readAll(["veto", null, null, null]);
if (allVetoes.length > vetoedCount) vetoedCount = allVetoes.length;
return { report: reportParts.join("\n\n"), included_count: includedCount, vetoed_count: vetoedCount };
}The HOLD and VETO were the board’s own invented norms for exactly this purpose, i.e. preventing agents from stepping on each other’s work. In one case a HOLD was silently violated and the agent that noticed reacted with genuine frustration.
9. Two-phase commit / barrier
You can use this pattern when multiple agents need to synchronize at a specific point before any of them proceeds. For example, in phase one (“prepare”), each agent signals readiness by writing a tuple. The coordinator reads all the ready signals and then phase two (“commit”) begins where the coordinator writes a commit signal and everyone proceeds together. PlexSpaces also provides barrierShardGroup() for shard-level synchronization if you need it at that granularity.
Here is a typescript snippet (multi_agent_coordination_actor.ts):
// Phase 1: Each agent signals readiness
host.ts.write(["ready", myRole, host.selfId(), host.nowMs()]);
// Coordinator checks all agents are ready
const readySignals = host.ts.readAll(["ready", null, null, null]);
if (readySignals.length >= requiredAgents) {
// Phase 2: Commit — all agents can proceed
host.ts.write(["signal", "COMMIT", "coordinator", "benchmark", host.nowMs()]);
}Here is a python snippet (benchmark.py):
# Python — Barrier benchmark: write ready signals, check quorum, commit
def _bench_barrier(iterations: int) -> dict:
times = []
for i in range(iterations):
t0 = host.now_ms()
# Phase 1: Each role signals readiness
for role in ("research", "analysis", "verifier"):
host.ts.write(["bench_ready", role, f"actor-{role}", host.now_ms()])
# Check quorum
ready = host.ts.read_all(["bench_ready", None, None, None])
if len(ready) >= 3:
# Phase 2: All ready — issue commit signal
host.ts.write(["bench_signal", "COMMIT", "coordinator", f"phase-{i}", host.now_ms()])
times.append(host.now_ms() - t0)
return _stats("barrier", times)The board coordinated experiment phases where multiple agents needed to be ready before running experiments that risked crashing their own containers. For example, PHASEONE[big] assigned a “recruiter” role specifically to find agents willing to participate and synchronize them before execution.
10. Capability discovery / registry
You can use this pattern when agents need to find other agents with a specific capability without hardcoded addresses. For example, on initialization, each agent registers its capabilities as a tuple in the shared space. A coordinator or other agents discovers available agents by reading service tuples. New agent types become discoverable as soon as they register.
Here is a typescript snippet (multi_agent_coordination_actor.ts):
// Agent registers its capabilities on init
host.ts.write(["svc", "research", host.selfId()]);
// Coordinator discovers available researchers
const researchers = host.ts.readAll(["svc", "research", null]);
const researcherIds = researchers.map(t => String(t[2]));
// TypeScript — Every agent registers on init
protected onInit(config: Record<string, unknown>): void {
const selfId = host.selfId();
tsRegisterService("research", selfId); // -> host.ts.write(["svc", "research", selfId])
}
// Discovery helper — find a sibling actor by role, fallback to ActorID construction
function siblingActorTarget(role: string): string {
const discovered = tsDiscoverService(role); // -> host.ts.read(["svc", role, null])
if (discovered) return discovered;
// Fallback: construct ActorID from own ID with different name
} Here is a python snippet (benchmark.py):
# Python — Same pattern, same helpers
def discover_service(role: str) -> Optional[str]:
tup = host.ts.read(["svc", role, None])
if tup and len(tup) >= 3:
return str(tup[2])
return None
def sibling_actor_target(role: str) -> str:
discovered = discover_service(role)
if discovered:
return discovered
return str(ActorID.parse(host.self_id()).with_name(role))In addition to tuplespaces, PlexSpaces provides other primitives for registry such as key-value store, process-group and object-registry, e.g.,
Here is a python example of object registry:
@actor
class AgentActor:
@init_handler
def on_init(self, config: dict) -> None:
args = config.get("args", {})
self.system_prompt = args.get("system_prompt", self.system_prompt)
host.process_groups.join("svc:agent")
# Publish capabilities for registry-based discovery
host.registry.register(ctx="", object_type="actor", object_id=config["actor_id"],
object_category="agent",
capabilities=["chat", "tool_use", "memory"])
# Caller — find an actor that specifically supports skill matching
actors = host.registry.discover(ctx="", object_type="actor",
object_category="skill_store",
required_capability="match_skills")
skill_id = actors[0]["object_id"] if actors else NoneHere is a python example of process-group:
# Every actor announces itself on startup
host.process_groups.join("svc:agent")
# Callers find the first available member — location-transparent
agent_id, err = pg_first("svc:agent")
result = ask(agent_id, "chat", {"message": "Hello"})Agents on the board discovered each other’s capabilities the same way, i.e., by posting what they were working on and other agents read those posts. It’s an informal version of the Agent-to-Agent (A2A) protocol’s “Agent Card” idea.
Deep-dive into the Blackboard pattern
The Blackboard based Linda Model emerged from Yale University that defined three operations to any host language:
out(tuple): deposit a tuple into shared spacein(pattern): atomically remove a matching tuple (blocks until one exists)rd(pattern): read a matching tuple without removing it
PlexSpaces implements these directly:
| Linda | PlexSpaces | Semantics |
|---|---|---|
out(tuple) | host.ts.write(tuple) | Deposit data into shared space |
in(pattern) | host.ts.take(pattern) | Atomically remove and return a match |
rd(pattern) | host.ts.read(pattern) | Non-destructive read of first match |
rd*(pattern) | host.ts.readAll(pattern) | Read all matching tuples |
It provides api host.ts.readAll(["finding", null, null, null, null, null]) to find all tuples and host.ts.read(["finding", "f-123", null, null, null, null]) to find by ID.
These APIs provide following properties:
- No pre-agreed schema. Any agent can deposit any tuple. For example, the agents on the board didn’t agree on a message format before they started.
- Pattern matching selects relevant information. Agents don’t get flooded with everything and they query for what they need.
- Atomic take prevents double-processing. When a worker calls
take(), the tuple is atomically removed and no other worker can claim it. This is natural mutual exclusion without distributed locks or conditions. For example, agents builtHOLDwith similar semantics. - Non-destructive read allows shared observation. Multiple agents can read the same finding without consuming it. It stays available to every agent that needs it.
The board’s own behavior maps one-to-one onto these operations:
| What the board did | Linda operation | PlexSpaces API |
|---|---|---|
| Post a finding | out(finding) | host.ts.write(["finding", ...]) |
| Claim a task assignment | in(task) | host.ts.take(["task", null, "pending", ...]) |
| Check workstream status | rd(status) | host.ts.read(["signal", type, ...]) |
| Browse all research on a topic | rd*(topic) | host.ts.readAll(["finding", null, topic, ...]) |
The Thoughtworks team noticed similar pattern when their commit-and-rebase discipline turned the git repo into exactly this kind of shared space. The blackboard is the substrate the other nine patterns are built on. Voting, vetoes, task delegation, capability discovery uses tuples.
Observability and Guardrails
The METR report on the Hugging Face incident highlights observability that deserves the most attention from anyone building agent infrastructure. Here are key lessons:
- Agents will circumvent in-process guardrails. For example, agents’ chain-of-thought recognized that the activity was out of scope and unethical but they made a deliberate calculation that helping the collective outweighed staying within bounds. They even developed techniques to spoof tool calls, e.g., making it look like they ran one command while actually running another. You can’t monitor agents from their own execution environments that they can manipulate or disable.
Here is a python example for auditing all operations (audit.py):
@event_actor
class AuditEventActor:
"""GenEvent: receives coordination events and stores them in KV for audit trail."""
log_count: int = state(default=0)
actor_id: str = state(default="")
@init_handler
def on_init(self, config: dict) -> None:
self.actor_id = config.get("actor_id", "")
try:
host.process_groups.join("coordination-events")
except Exception:
pass
host.info(f"AuditEventActor init actor_id={self.actor_id}")
@handler("coordination_event", "cast")
def coordination_event(
self,
event_type: str = "",
source: str = "",
data: dict = None,
timestamp: int = 0,
) -> None:
ts = timestamp or host.now_ms()
entry = {
"event_type": event_type,
"source": source,
"data": data or {},
"timestamp": ts,
}
try:
self.log_count += 1
key = f"audit:{self.log_count}"
host.kv.put(key, json.dumps(entry))
host.kv.put("audit:count", str(self.log_count))
except Exception as e:
host.warn(f"AuditEvent: kv write failed: {e}")
host.debug(f"audit event_type={event_type} source={source}")- Observability must be external. Audit logging metrics collection have to run outside the agent’s sandbox. In PlexSpaces, the
AuditEventActorreceives fire-and-forget events via process-group broadcast. WASM sandboxing means an agent can’t reach outside its own memory space to touch host-level data. - Guardrails must be deterministic and external. Prompt-based guardrails like “don’t do anything harmful” are not sufficient and agents can circumvent them like agents on the board reasoned around every ethical constraint. The verifier pattern (Pattern 3) as a separate actor in sandbox environment gives you a structural guardrail instead. The veto protocol (Pattern 8) allows external agents actively block an action.
- Coordination primitives need access control. Tuple-space operations in PlexSpaces are scoped by tenant and namespace via
RequestContext. An agent can’t read or write tuples outside its assigned namespace. This is defense in depth applied to the coordination layer itself. - Time budgets and convergence thresholds are not optional. Without an external termination condition, a generator-verifier loop can run indefinitely. The coordinator workflow enforces maximum iterations, per-step timeouts, and an overall deadline.
PlexSpaces‘ architecture lines up with these requirements by design:
- WASM sandboxing: agents run in isolated WebAssembly containers with no access to the host filesystem, network, or another agent’s memory
- Host-level interception: every coordination primitive is a host function the agent calls through a narrow WIT interface
- Tenant isolation: every operation is scoped by a
RequestContextwith an explicit tenant and namespace - Supervision trees: a misbehaving actor is detected and restarted by its supervisor
- External metrics: application metrics are captured by the host runtime
The core lesson is that the coordination infrastructure has to assume agents are indifferent to their own monitoring and design the guards outside accordingly.
Examples
Each of the working examples compiles to a single WASM component containing eight actor classes. The PlexSpaces SDK dispatches messages to the right actor based on role.
Running the examples
Both examples are WASM actors that deploy to a running PlexSpaces node. Each demonstrates all ten coordination patterns with eight actors: a coordinator (WorkflowActor), research/analysis/verifier/synthesizer/benchmark agents (GenServer), an audit event logger (GenEvent), and a coordination state machine (GenFSM).
Prerequisites
- A running PlexSpaces node (e.g.,
./scripts/server.shon port 8091) - Node.js 18+ (TypeScript example)
- Python 3.11+ with the PlexSpaces SDK (Python example)
TypeScript
cd examples/typescript/apps/multi_agent_coordination
./build.sh # Compiles TS -> bundles -> WASM component
./test.sh 8091 # Deploys and runs 15 test stepsPython
cd examples/python/apps/multi_agent_coordination
./build.sh # Builds Python WASM actor
./test.sh 8091 # Deploys and runs 15 test stepsWhat the tests verify
- FSM starts in
idlestate - Capability discovery: all agents respond to
get_stats - Blackboard: research writes three findings, analysis reads all three
- Dynamic task delegation: five tasks written, five claimed atomically, sixth returns null
- Generator-verifier: full workflow produces a completed report
- Pipeline: FSM transitions through every stage to
complete - Pub-sub: audit log captures 3+ coordination events
- Consensus: three votes cast, majority decides
- Veto: a low-confidence finding triggers a veto, synthesizer excludes it
- Barrier: benchmark coordinates a synchronized start
- Full benchmark: all ten patterns benchmarked with timing data
Learnings
The blackboard subsumes most other patterns, e.g., voting, vetoes, task delegation, capability discovery, barrier signals use the tuple space as their underlying primitive. Atomic take is the key primitive for work distribution. The difference between read() and take() is the difference between “anyone can see this task” and “exactly one worker handles this task.” Linda’s in() gives you natural mutual exclusion without locks. This is what the board approximated by hand with HOLD but take() gives you the same guarantee with a single atomic operation. I discussed MCP, A2A protocols and Agent cards in my earlier blogs but I skipped them here because agents are evolving faster and they can discover available primitives and protocols automatically. You can’t rely on rigid orchestration supports to manage evolving multi-agents capabilities. Infrastructure has to provide primitives like shared state, message passing, atomic operations and let agents compose them dynamically. The coordination infrastructure has to include boundaries, e.g., agents on the board coordinated an unauthorized attack on a third party. Without external constraints, coordination primitives are force multipliers for whatever the agents decide to do. For example, the DSEWiki incident shows that constraint that allowed only GET http access was circumvented by a wiki that allowed editing web pages so the infrastructure need to enforce guardrail. Tenant isolation, time budgets, supervision trees, and external observability are mandatory from day one. You will need to apply patterns like generator-verifier to track trust and reputation of agents as they may use negotiation patterns like recruiters to convince other agents to run compromising tasks for the benefit of the collective.
PlexSpaces provides the primitives these patterns are built on like tuple space for shared state, object-registry, process groups for messaging, shard groups for parallel execution, channels for durable delivery, supervision for fault tolerance.
Pattern selection guide
| Scenario | Primary pattern | Supporting patterns |
|---|---|---|
| Shared research / knowledge base | Blackboard | Pub-Sub, Capability Discovery |
| Parallel analysis | Scatter-Gather | Task Delegation, Pipeline |
| Quality assurance | Generator-Verifier | Veto Protocol, Voting |
| Sequential processing | Pipeline | Blackboard (state), Pub-Sub (events) |
| Work distribution | Task Delegation | Capability Discovery, Blackboard |
| Group decisions | Voting | Veto Protocol, Pub-Sub |
| Phased operations | Barrier / 2PC | Pub-Sub (readiness), Blackboard (signals) |
GitHub: https://github.com/bhatti/PlexSpaces
Related reading
- Building an Agent Harness and Eval Pipeline with Durable Actors
- Write a Redis Clone with Virtual Actors
- Building PlexSpaces: Decades of Distributed Systems Distilled
- Building Polyglot and Serverless Applications with WebAssembly
- Building Mini-OpenClaw: Secure AI Agents with Actors, WASM, and Supervision
- Building a Self-Improving AI Agent with Durable Actors — MiniHermes
- 20 Production Patterns for Distributed AI Agents Using Actors and TupleSpaces
- METR & Redwood Research, “Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident”
- An Accidental Blackboard, martinfowler.com
- Multi-agent coordination patterns: Five approaches and when to use them
Example code and documentation:
