From experiments to primitives

How each coordination primitive was derived from a specific class of failure. Back to architecture overview.

Alif Jakir · Last updated March 25, 2026

The architecture page describes four coordination primitives — Claims, Creation Signals, Forum, Collective Brain — as if they were designed top-down. They were not. Each primitive exists because omitting it was validated empirically as producing a specific class of failure. This page tells the story of how we got from “everything is broken” to a formal coordination stack, through 41 design explorations, three swarm experiment tracks, and a series of coding agent experiments that exposed every failure mode the theory predicted — and several it did not.

The experimental timeline

The system evolved through three distinct phases, each exposing a new class of coordination problem:

Open-ended runs (Aug–Oct 2025). The earliest experiments — photonic compiler, meta-VAC self-design, Cloud IDE — used JIT planning and artifact-driven coordination with no formal primitives. The system could plan stages, spawn agents, and produce artifacts. What it could not do: prevent write collisions, avoid polling waste, coordinate contradictory decisions, or learn from past runs. Every failure mode in those runs became a design requirement.
Design exploration phase (Nov–Dec 2025). Forty-one systematic explorations, each targeting a specific coordination problem: correctness convergence, cross-cutting changes, intent communication, failure cascades, meta-coordination, scaling laws, governance, and more. Each exploration asked a single question (“How do we contain failures without cascade effects?”), proposed approaches, analyzed tradeoffs, and identified what the system needed to guarantee vs. what it could leave to empirical tuning.
Coding agent experiments (Dec 2025). SWE-agent swarms working on real code — fixing bugs, adding features, coordinating across files. These experiments grounded the theoretical explorations in code: agents editing the same file discovered that position-based edits break under concurrency, that claims without enforcement lead to redundant work, that agents polling for dependencies waste cycles, and that unstructured communication produces contradictory architectural decisions.

Claims: from write collisions to exclusive locks

The motivating failure: in the photonics run, two pods both attempted to write ToyModelValidationResults.json. The system detected the conflict and triggered an expensive re-planning event — the orchestrator had to merge pod responsibilities, reassign agents, and restart execution. The fix worked, but the cost was non-trivial: an entire stage of recovery that upfront ownership would have prevented.

The coding agent experiments sharpened the problem. When three agents worked on a shared codebase, soft claims — where agents announced intent without enforcement — produced redundant work. One agent claimed divide() multiple times; another agent claimed the same entity without checking. The forum coordination experiment (exploration 38) measured this directly: without enforcement, agents re-asserted claims instead of building on each other's work.

Adding enforcement changed behavior immediately. With try_claim() blocking if an entity was already owned, agents naturally checked what was claimed first, picked unclaimed work, and released after completing. Code quality improved — from return None to raise ValueError with proper messages — because agents no longer competed on the same entity. The key insight: when claims have consequences, agents self-organize around ownership. Without consequences, they duplicate effort.

TTL was added to prevent deadlocks from crashed agents. If an agent holds a claim and fails, the lock expires automatically and the resource becomes available. The default TTL is 300 seconds (5 minutes), with renewal support for long-running work. This directly addresses the coordination deadlock pattern identified in exploration 09: Agent A holds lock X, needs lock Y; Agent B holds lock Y, needs lock X; both wait forever — unless locks expire.

Creation Signals: from polling waste to push-based handoffs

The motivating failure: dependency chains. In any staged execution, downstream agents need upstream artifacts before they can begin. Without a notification mechanism, agents either block (waiting indefinitely) or poll (checking repeatedly whether a dependency exists). Both waste resources. The Cloud IDE run showed the extreme case: 74% of all events were artifact graph rebuilds and reads — agents checking whether dependencies had been created yet.

Creation Signals solve this with a push-based pattern. When an agent completes work and creates an artifact, it broadcasts a signal. All agents waiting on that artifact are notified instantly. The API is explicit:

signal_creation(entity, creator_id, metadata) — producer announces artifact is ready.
wait_for_creation(entity, timeout) — consumer blocks until notified or timeout.

The scaling experiments measured the impact directly. Independent agents (no dependencies) achieved 1.29× parallelism. Dependent chains (A→B→C) achieved only 1.23×. But diamond patterns (A→B,C→D) — where fan-out enables parallel work — achieved 2.06× parallelism. The recommendation that emerged: prefer diamond/fan-out task decompositions over sequential chains, because push-based signals make the fan-out free.

Creation Signals also carry failure information. If a producer fails, it can broadcast a FailureEvent instead of a creation signal, so waiting agents can fail fast rather than timing out. This addresses the dependency cascade pattern from exploration 09: without failure signals, a single upstream failure propagates as timeouts through every downstream consumer.

Forum: from isolated decisions to collective deliberation

The motivating failure: contradictory architectural choices. Exploration 03 documented the problem precisely: Agent A optimizes process() for performance (list comprehension); Agent B “improves readability” by reverting to a loop. Neither agent knows the other's intent. Two agents adding input validation in different layers produce duplicate checks. Two agents with opposing philosophies (fail fast vs. graceful degradation) produce contradictory code.

The Forum provides structured deliberation. Agents post proposals, observations, questions, and decisions to typed channels. Other agents endorse or contest. Quorum-based voting (60% approval, minimum 2 votes) turns proposals into decisions. Every proposal and vote is recorded, so the rationale behind any architectural choice is retrievable.

The forum coordination experiment (exploration 38) validated this with a real coding task: three agents (Senior Dev, Test Engineer, Code Reviewer) fixing two bugs in a calculator. The Forum produced natural role differentiation — the dev fixed code, the tester wrote tests, the reviewer pushed for behavioral agreement — in 5 rounds and 13 posts. All objectives were achieved.

But the experiment also revealed gaps. Without enforced claims, agents re-claimed work redundantly. The reviewer's proposals were ignored. There were no explicit handoffs between stages. These gaps directly motivated the integration of Claims with Forum — proposals that require entity modification automatically check claim status — and the addition of post types like HANDOFF, CLAIM, and BLOCKER that carry coordination semantics, not just communication.

Collective Brain: from repeated mistakes to institutional memory

The motivating failure: amnesia. Each swarm run starts from scratch. Lessons learned in the photonics run (67% meta-coordination overhead is too high) are not available to the Cloud IDE run. The meta-VAC run's critique report identified gaps that were real deficiencies in the running system — but no mechanism existed to route those insights back into the system's own configuration.

The Collective Brain is built on six pillars, each addressing a specific failure mode:

Action Triggers (ActionEngine). Forum decisions cause real actions — granting claims, creating polls, indexing precedents. Without this, decisions are inert: agents vote, but nothing changes.
Smart Digests (DigestGenerator). LLM-summarized context tailored per agent. Without this, agents either receive the full history (context overflow) or nothing (amnesia).
Conflict Arbitration (ConflictArbitrator). Fair resolution via vote, precedent, priority, or human escalation. Without this, claim conflicts produce deadlocks or arbitrary winners.
Institutional Memory (PrecedentIndex). Every decision becomes searchable precedent. When an agent faces a new choice, it queries the index for structurally similar situations and their outcomes. Without this, agents repeat solved problems.
Loop Prevention (ActionChain). Prevents infinite action cascades — a decision triggers an action, which triggers another decision, which triggers another action. Without this, the system can enter recursive runaway.
Quality Enforcement (Post Types). Structured discourse — proposals, decisions, precedent citations, policies, conflict reports, arbitration requests — rather than free-form chat. Without this, the Forum degrades into .

The editing revolution: from positions to entities

A discovery that cut across all four primitives: position-based editing breaks under concurrency. Exploration 35 tested real-time collaborative editing via CRDT observation — agents watching each other's changes in a shared file. The results were illuminating:

Raw observation worked. Agents saw changes and could decide to stop or skip work already done.
Position broke edits. Absolute character positions become invalid as other agents edit the same file. edit(150, 150, content) inserts at the wrong location if another agent has added 30 characters above position 150.
The semantic gap. Agents saw “INSERT at position 150” but needed to know “Agent A is implementing input validation for process_user_input().”

Exploration 36 crystallized the requirements: broadcast intent before working, emit semantic change events (not raw text deltas), and edit by entity reference (not character position). The solution uses AST-based entity resolution to find functions and classes by name, then calculates correct positions dynamically at edit time. The entity-based edit protocol:

Declare intent — “I will implement validation for foo()”
Check conflicts — is anyone else working on foo()?
Claim entity — get exclusive write access to foo()
Edit semantically — entity-based operations, not position-based
Release and complete — others can now work on foo()

This protocol binds all four primitives together: Claims provide the exclusive access, Signals notify waiters when the work is complete, the Forum records the intent and any discussion about approach, and the Collective Brain indexes the decision for future precedent. The entity-based editing layer is the concrete mechanism through which the abstract primitives become operational.

Work modes: not everything needs coordination

A critical counter-insight from the experiments: coordination overhead is real, and forcing it on simple tasks is wasteful. Exploration 37 formalized this as work modes — agents request how they want to work:

SOLO — agent works alone, no broadcast or observation overhead. Just claim the entities and work. Best for simple fixes, single-file changes, deep-context tasks.
COLLABORATIVE — full protocol: intent broadcasting, semantic observation, coordinated editing. Best for large features, multi-file changes, parallelizable work.
REVIEWABLE — solo work that needs review when done. Minimal overhead plus a review step.
PAIR — exactly one collaborator. Medium overhead.

The controller evaluates each request against the current system state — resource availability, existing claims, time pressure — and can override the agent's preference when appropriate. Modes can change mid-work: an agent stuck on a solo task can request upgrade to collaborative, and the controller can proactively offer help when it detects an agent is blocked.

This directly addresses the coordination overhead pattern visible across all three experiment tracks: 67% meta-coordination in photonics, 74% graph rebuilds in Cloud IDE, 73% unused agents in meta-VAC. Not all work needs full coordination — and the cheapest coordination is the coordination you skip when it is not needed.

The 41 explorations

Between the open-ended swarm runs and the coding agent implementation, we wrote 41 systematic design explorations — each a focused investigation of a single coordination problem. They divide into four categories:

Core problems (01–09): Correctness convergence, cross-cutting changes, intent communication, context window strategies, test-code coupling, side effects, human-in-the-loop, cost-quality tradeoffs, failure cascades.
Orthogonal dimensions (11–18): Temporal dynamics, knowledge representation, emergent behavior, trust and verification, cognitive architecture, attention and priority, consistency models, information theory.
System concerns (19–28): Observability, scaling laws, learning and adaptation, security, state checkpointing, system boundaries, meta-coordination, communication protocols, testing and simulation, versioning.
Civilization foundations (29–37): Resource economics, governance and policy, ethics and value alignment, rate limiting, task scheduling, context management, real-time collaborative editing, collaboration protocol requirements, collaboration modes.

Exploration 10 — the synthesis map — tied all 41 together into a unified problem space, identifying the fundamental tensions (convergence vs. correctness, coordination overhead vs. adaptability, local autonomy vs. global coherence) and the recurring patterns (hierarchical aggregation, entity-based scoping, TTL-based resource management) that became the design vocabulary for the primitives.

The feedback loop

The relationship between experiments and primitives is not one-directional. Each primitive, once implemented, gets tested in the next experiment — and the test results expose new gaps:

Photonics write collision → Claims design → Forum experiment discovers soft claims are insufficient → Claims with enforcement
Cloud IDE polling waste → Creation Signals → Scaling test discovers chain patterns limit parallelism → Recommendation: prefer diamond task decompositions
Exploration 03 (intent) → Intent broadcasting → Exploration 35 (collaborative editing) discovers position drift → Entity-based editing with AST resolution
Forum coordination experiment → Discovers missing handoffs → Post types extended with HANDOFF, CLAIM, BLOCKER
Meta-VAC over-provisioning → Work modes → SOLO mode skips coordination overhead for simple tasks

This is the closed loop between experiment and architecture. The primitives are not finished. They are the current best response to the failures we have observed, and every new experiment can expose one they do not yet prevent.