status: in_progress ## OBJECTIVE 1-3 sentences: what "done" looks like ## CURRENT STATE verified state (<= ~800 tokens), e.g. "28 tests pass" ## NEXT STEPS ordered list of what happens next ## FILE MAP one line per file + status (added / tested / done) ## HANDOFF what you are about to do, exact commands, expected outcome ## LOG append-only run log, one line per checkpoint ``` The two tools: - **`checkpoint`** — the *only* writer. After every unit of work the agent calls `checkpoint` with the sections that changed (a new `current_state` reflecting a verified test result, an updated `file_map`, or a `handoff` block before heavy work). It appends one line to `LOG`, stamps `updated`, and rewrites the file. Because it takes an **exclusive lock** and writes via a temp file + atomic `os.replace`, two sub-agents — or a sub-agent and the main agent — can call it concurrently without interleaving corrupting the file. - **`load_state`** — the cheap reader. At the *start* of every turn and *after every compaction*, the agent calls `load_state` to pull back just the hot sections (~1k tokens). This is the compaction-survival mechanism: no matter how much context was summarised away, `load_state` restores exactly what's needed to resume from `NEXT STEPS` / `HANDOFF`. Rules that make it work: - **Bootstrap first, resume second.** First turn of a task: `checkpoint` with `task`, `status`, `objective`, `current_state`, `next_steps`. Every later turn: `load_state` first; if `status` isn't `done`, resume from `NEXT STEPS` — do not re-plan. - **Record verified state, not intent.** `CURRENT STATE` states what is actually true (test results, committed refs), never what you plan to do. - **Sub-agents keep their hands off the shared sections.** A sub-agent may only append to `LOG` (via `checkpoint` with `actor = sub_agent_1` / `sub_agent_2`); it must not write `CURRENT STATE`, `NEXT STEPS`, `OBJECTIVE`, `HANDOFF`, or `FILE MAP`. The orchestrator is the sole writer of those and merges them at the Primary Sync — the same split as the mailbox rule above. `file_map` in particular is owned by the main agent and consolidated from both mailboxes. - **LOG auto-archives.** `LOG` is capped (a few KB); when it overflows, the oldest lines are rolled into a sibling `notes-archive.md` (append-only), so the hot file stays small and re-readable. - **Set `HANDOFF` before context-heavy work.** Before a step you expect to burn a lot of context (a long build, a big refactor), write a `HANDOFF` block with the exact commands and expected outcome, so a later compaction can resume precisely. Status is a small fixed set — `pending`, `in_progress`, `blocked`, `done` — which keeps the frontmatter greppable and lets other tools or agents reason about progress at a glance. The tools are plain Python (`task-harness-tools.py`, a small module on `pydantic`) and are covered by a pytest suite (`tests/test_task_harness.py`) that specifically checks the concurrency guarantee (many concurrent `checkpoint` calls must not corrupt the file) and the LOG-rolling behaviour. Treat it as a tested part of the stack, not a script you paste in. ## Related - [Sub-agents](/ai-docs/04-sub-agents/) — the delegation pattern this page orchestrates - [Open Terminal & Tooling](/ai-docs/06-open-terminal-and-tooling/) — the shell and the `uv` tooling the orchestrator runs - [Skills](/ai-docs/07-skills/) — packaged procedure a sub-agent can be handed as its brief --- ## Source Disclaimer - [x] AI Generated - [ ] Human Generated - [ ] AI Edited

--- title: "05 — Orchestration & Task State" description: "This page covers the two operational patterns that make multi-agent work reliable in a shared git workspace: **parallel orchestration** (slots and mailboxes) and the **task-state harness** (a shared `…" section: ai-docs raw: "05-orchestration-and-task-state.md" source: ai-generated tags: ai, llama.cpp, docker last-updated: 2026-10-13 --- # 05 — Orchestration & Task State > How a main agent (the "master orchestrator") fans work out to multiple parallel sub-agents without write conflicts, and how task state survives long runs and context compaction. This page covers the two operational patterns that make multi-agent work reliable in a shared git workspace: **parallel orchestration** (slots and mailboxes) and the **task-state harness** (a shared `notes.md` file driven by `load_state` / `checkpoint`). They sit on top of the sub-agent pattern in [04](/ai-docs/04-sub-agents/) and the tooling in [06](/ai-docs/06-open-terminal-and-tooling/). ## Parallel orchestration: slots and mailboxes When the orchestrator delegates to *several* sub-agents at once, two problems appear: they may try to write to the same files (git merge / write conflicts in the shared workspace), and the orchestrator needs a clean way to collect what each one did. The pattern used here resolves both with **fixed slots** and **one mailbox per slot**. - **Two slots, max.** The first concurrent worker takes **Slot 1**, the second takes **Slot 2**. Keep it to two: beyond that the orchestrator's own context and the shared workspace get hard to track, and the throughput gain tapers off. - **A dedicated mailbox per slot.** Slot 1 writes its results *only* to `.mailbox/sub_agent_1.md`; Slot 2 *only* to `.mailbox/sub_agent_2.md`. Each worker is told, in its activation prompt, to read a shared conventions file (`.agent-rules`) and to write its completion status and step-by-step outcomes to *its own* mailbox — and never to open or write the other slot's file. Giving each parallel worker a private write target is what stops two agents from clobbering each other in the shared git workspace. - **The activation prompt is prescriptive.** The delegation message must name the slot and the mailbox explicitly, e.g. *"You are assigned to Slot 2. Read `.agent-rules` for the project conventions. Write your completion status and step outcomes to `.mailbox/sub_agent_2.md`. Do not read `AGENTS.md`."* The last clause matters: workers get their rules from the small shared `.agent-rules` file, not from the orchestrator's longer `AGENTS.md`, which keeps the worker's context lean. - **Primary Sync.** The orchestrator waits for *both* workers to return, reads and consolidates the two mailboxes, then runs the verification pass (`uv run pytest` — see [06](/ai-docs/06-open-terminal-and-tooling/)). Consolidation is a single step owned by the orchestrator, so there is no cross-worker write contention at sync time. This is the multi-file version of the self-contained-prompt rule in [04](/ai-docs/04-sub-agents/): the *prompt* carries the task, the *mailbox* carries the return, and the *slot* isolates the write surface. ## Local tooling: `uv` as the default When agents — the orchestrator or its parallel sub-agents — run local Python tooling (scripts, linters, tests), the convention is to **always go through `uv`**, so the interpreter and dependency set are pinned by the workspace instead of drifting across whatever system Python happens to be installed: - Run scripts with `uv run `. - Run the test suite with `uv run pytest` — this is the verification pass the orchestrator runs at the Primary Sync after consolidating sub-agent work. - Fix style/type anomalies mechanically first: `uv run ruff check --fix` (strict `PEP 484` type hints are the standard for new code). The point is reproducibility: the *same* environment whether a human, the main agent, or a parallel sub-agent runs the command, and a verifiable git workspace before anything is committed. ## Keeping task state across a long run: the `notes.md` harness A long, tool-heavy task has two failure modes a plain chat transcript doesn't solve: **compaction** (the model's context is summarised mid-task, losing detail) and **coordination** (the orchestrator must remember which files are done, what each sub-agent did, and what comes next). The pattern used here is a small **task-state harness** — two tools, `load_state` and `checkpoint`, operating on one shared `notes.md` file. `notes.md` is a small markdown state doc: frontmatter (`task`, `status`, `created`, `updated`) plus a fixed set of **hot sections** kept near the top so they are cheap to re-read. ```text # TASK STATE task: status: in_progress ## OBJECTIVE 1-3 sentences: what "done" looks like ## CURRENT STATE verified state (<= ~800 tokens), e.g. "28 tests pass" ## NEXT STEPS ordered list of what happens next ## FILE MAP one line per file + status (added / tested / done) ## HANDOFF what you are about to do, exact commands, expected outcome ## LOG append-only run log, one line per checkpoint ``` The two tools: - **`checkpoint`** — the *only* writer. After every unit of work the agent calls `checkpoint` with the sections that changed (a new `current_state` reflecting a verified test result, an updated `file_map`, or a `handoff` block before heavy work). It appends one line to `LOG`, stamps `updated`, and rewrites the file. Because it takes an **exclusive lock** and writes via a temp file + atomic `os.replace`, two sub-agents — or a sub-agent and the main agent — can call it concurrently without interleaving corrupting the file. - **`load_state`** — the cheap reader. At the *start* of every turn and *after every compaction*, the agent calls `load_state` to pull back just the hot sections (~1k tokens). This is the compaction-survival mechanism: no matter how much context was summarised away, `load_state` restores exactly what's needed to resume from `NEXT STEPS` / `HANDOFF`. Rules that make it work: - **Bootstrap first, resume second.** First turn of a task: `checkpoint` with `task`, `status`, `objective`, `current_state`, `next_steps`. Every later turn: `load_state` first; if `status` isn't `done`, resume from `NEXT STEPS` — do not re-plan. - **Record verified state, not intent.** `CURRENT STATE` states what is actually true (test results, committed refs), never what you plan to do. - **Sub-agents keep their hands off the shared sections.** A sub-agent may only append to `LOG` (via `checkpoint` with `actor = sub_agent_1` / `sub_agent_2`); it must not write `CURRENT STATE`, `NEXT STEPS`, `OBJECTIVE`, `HANDOFF`, or `FILE MAP`. The orchestrator is the sole writer of those and merges them at the Primary Sync — the same split as the mailbox rule above. `file_map` in particular is owned by the main agent and consolidated from both mailboxes. - **LOG auto-archives.** `LOG` is capped (a few KB); when it overflows, the oldest lines are rolled into a sibling `notes-archive.md` (append-only), so the hot file stays small and re-readable. - **Set `HANDOFF` before context-heavy work.** Before a step you expect to burn a lot of context (a long build, a big refactor), write a `HANDOFF` block with the exact commands and expected outcome, so a later compaction can resume precisely. Status is a small fixed set — `pending`, `in_progress`, `blocked`, `done` — which keeps the frontmatter greppable and lets other tools or agents reason about progress at a glance. The tools are plain Python (`task-harness-tools.py`, a small module on `pydantic`) and are covered by a pytest suite (`tests/test_task_harness.py`) that specifically checks the concurrency guarantee (many concurrent `checkpoint` calls must not corrupt the file) and the LOG-rolling behaviour. Treat it as a tested part of the stack, not a script you paste in. ## Related - [Sub-agents](/ai-docs/04-sub-agents/) — the delegation pattern this page orchestrates - [Open Terminal & Tooling](/ai-docs/06-open-terminal-and-tooling/) — the shell and the `uv` tooling the orchestrator runs - [Skills](/ai-docs/07-skills/) — packaged procedure a sub-agent can be handed as its brief --- ## Source Disclaimer - [x] AI Generated - [ ] Human Generated - [ ] AI Edited - [ ] Human Edited