padi — the per-host terminal-workspace daemon
The architecture that replaced the R9/R10 finale plan for remote terminals (#951). One workspace daemon per host — padi, atop kaval — knows everything about that host's terminals and serves it as one surface (padiSurface); kolu-server thins to a web shell that connects each browser view to one padi. The canvas shows one host at a time (instant switch; other hosts reach you via badges and OS notifications). The host owns its state — layout, memory, clock. Phases W0–W12 all shipped (W10 — hosts persist as a settings field in the conf store; W12 — restore survives an unclean kaval death), with the surface consolidation in parallel.
The ground-up re-architecture for #951 — remote terminals — and the extraction that untangles kolu-server and the client along the way. The old plan (the pre-padi record) tagged every terminal with a HostLocation and checked that tag all over kolu-server, because it assumed one canvas could mix tiles from several hosts. Drop that assumption and the problem gets much simpler: “which host” is a property of the connection, not of each terminal. One daemon per host — padi (படி, the stepped stand a koḷu is arranged on) — knows everything about that host’s terminals and serves it as one complete surface. kolu-server becomes a thin web shell that connects each browser view to one padi. The four prep PRs (#1637 #1638 #1639 #1640) were closed; pulam and pulam-web dissolve into padi. Before committing, the design was stress-tested hard — seven independent reviewers each trying to break it a different way, every finding re-checked by a skeptic, plus a UX study of two real usage patterns; everything they caught is baked in below.
What padi is
Shipped with W4: a host switcher — kinda like tmux sessions (under the command palette’s Labs group until it stabilizes; the ChromeBar picker graduates later). A canvas shows one host’s terminals; switching hosts swaps the whole canvas instantly (bindings stay warm server-side). A host’s arrangement — tile layout, names, themes, remembered agent commands — lives on that host, so your desktop, laptop, and phone all see the same canvas for nix@prod, and a teammate-free second kolu (or a future kolu-tui) joins the same arrangement. A remote canvas is a full peer of the local one: Code tab, binary preview, paste/upload, transcript export, sleep/wake, session restore — all against the terminal’s own host, nothing silently wrong-host.
While you’re parked on one host, the others reach you through the channels an installed PWA already owns: the app badge aggregates awaiting-agents across all bound hosts, the switcher chips carry per-host counts, and an OS notification’s click is one action — switch binding + focus the tile. (The per-terminal plumbing — setAppBadge, service-worker notifications — predated it; W5 aggregated it across bindings.)
Two personas anchor the design, and both get the same shape: an ephemeral cloud box (a pu-style instance per project — teardown removes the host from the picker as one clean unit, never wreckage on a shared canvas) and a permanent headless server (days-long agents; hop local⇄server many times a day — instant switch + 1-action notification response is the tmux-grade loop that persona lives in).
The architecture
The gap this fixes. Today, no single thing can tell you everything about one host’s terminals. kaval has the PTYs. pulam has the awareness and fs/git — which it gets by dialing kaval itself. Everything else — layout, memory, restore targets, saved sessions, sleep/wake, boot adoption — lives only inside kolu-server. Because there was no one place to point at, the old plan had to tag every terminal with its host and check the tag at ~11 different places, behind two different lookup helpers. Meanwhile the client never needed any of it: it talks to one server over one socket, and no client code reads a host field. padi fixes this at the right level: below, one daemon that knows everything about its host’s terminals; above, one connection per view that picks the host; nothing per-terminal anywhere.
The three layers, and what each owns
| layer | owns | explicitly does NOT own |
|---|---|---|
| kaval (per padi) | PTYs, screen mirror, VT taps, inventory — unchanged role, byte-minimal; the overflow frame shipped (#1591, contract 5.0) | awareness, git, sleep, persistence (as today) |
| padi (per host × state-root) | the registry (terminals = authored ⋈ snapshot, composed server-side); the one fold + AgentMemory on the host’s clock; restoreTarget; lifecycle (create · kill · sleep/wake · boot adoption · inventory reconcile); spawn-policy composition against its kaval’s system.info; fs/git endpoint + watcher pulses; byte procedures (previewRead · scratch.write); transcript.exportHtml (the agent loaders run host-side — codex/opencode transcripts are SQLite queries, not file reads); worktree create/remove; the MRU activity feed (recentRepos/recentAgents — host-fs facts written inside the fold’s emit path); session persistence in its state-root; kaval supervision; status cells (kaval daemonStatus, expectedKaval); the urgency projection (awaiting counts + ids — no recency); the frozen control core (hello · version · drain · clock.now) |
serving browsers; user preferences; which client looks at it |
| kolu-server | HTTP · static · PWA · websocket; user-scoped preferences; buildInfo (minus expectedKaval, which moves to padi); the binding pool (dial + supervise-or-front padis; per-connection scope; re-serve); binding/link status on its own shell surface; notification fan-in across bindings |
terminals — it writes nothing about them. No registry, no fold, no adoption, no terminal-derived cells — with one read-only exception: the cross-binding urgency sum for badges, sourced solely from each padi’s urgency-projection member |
| client | one padiSurface consumer per view + the shell surface; ages recency against the bind-time clock offset (never the browser clock vs a foreign clock) | per-terminal host routing — there is nothing left to route with; HostLocation is deleted from records and create input |
padi’s identity is its state-root — the folder on the host that holds its data — and clients never kill a running padi. (kaval today is namespaced per kolu-server port precisely because its supervisor kills-and-respawns on version mismatch, and a shared daemon once let a dev server kill prod’s — the #1313 incident.) padi flips the relationship: one padi per (host, state-root). The deployed kolu binds the host’s default state-root; a dev kolu and each e2e worker pass their own private one, which also gives them a private kaval. A dev instance therefore can’t touch prod’s daemon, and a second kolu binding the same state-root does see the same canvas — sharing vs isolation is now an explicit choice, not a port accident. The mechanics (each exists because a reviewer constructed the failure without it):
- The data folder is permanent; the socket is not. The state-root must survive reboots — restore depends on it. But the socket and the lock file live in the runtime dir, named by a hash of the state-root path (
$XDG_RUNTIME_DIR/padi-<digest>/; same scheme for padi’s kaval). The runtime dir is wiped at every boot, so a stale lock can never make a dead padi look alive, and socket paths stay short no matter how deep the state-root sits. A small file in each runtime dir maps hash → state-root, sokaval-tui’s no-flags discovery keeps working and can label what it finds — that migration ships in the same PR, not a release note. - The padi binary decides the default state-root, on the host. One fixed rule in one place — never computed by a client, never sent over the wire. Otherwise two ways of reaching the host (login shell vs ssh, with different env) could compute two different “defaults” and silently split the host’s terminals across two padis. Clients pass nothing (default) or an explicit path (dev/e2e).
- Clients may start or join a padi — never restart one. This is a new, third supervisor policy, named so nobody reaches for the existing one (which kills-and-respawns on version mismatch — exactly what must not happen here). Two clients racing to start the same padi is safe: taking the lock is atomic, and the loser just connects to the winner.
- Upgrades converge on the newest version, through a small frozen “control core”. padiSurface will grow, so version mismatches will happen — and if “restart” only existed inside the versioned contract, the one moment you need it is the moment you can’t call it. So padi also serves a tiny never-changing side channel: hello · version · drain ·
clock.now. The rule: a client older than the running padi is refused, loudly (upgrade your kolu). A client newer may ask padi to drain — save everything and exit; the PTYs stay alive in kaval — and then start the newer version. Newest always wins, so two clients at different versions converge instead of fighting, and nothing ever force-kills a padi.
The state model — what survives from the awareness design, and what moves
The observe-vs-remember split survives unchanged inside padi: the sensors still cannot write memory (the types forbid it), the fold stays the one and only writer, and the shipped types are reused as-is. What moves is where remembering lives: sensors and fold now sit in the same process, so nothing folded ever has to be rebuilt across a wire — the whole planned event-stream apparatus (#1638’s framer, sequence numbers, gap frames) is simply never built, and the old fold-overwrite bug class (#1614) can’t happen because there’s no second writer left. Two consequences, decided out loud rather than slipped in:
- The host’s clock stamps memory now. (The old rule said the consumer’s clock — but that rule existed for a mixed-host canvas, which no longer exists. Within one host, its own clock is the only one; two hosts’ times are never compared, because no view ever shows two hosts.) For display (“2m ago”), the client measures the clock difference once per connection — one
clock.now()round-trip, halved — and ages everything locally against that offset, so even a badly-skewed host shows sane ages. We deliberately do NOT serve a ticking “now” value: it would either be stale at connect or push a useless frame to every connection forever. Anything that does cross hosts (badges, the future cross-host dock) carries counts only, never times. - The client-side join retires. R8 split a terminal’s data into two collections because two processes wrote the two halves. In padi one process writes both, so it serves one composed
terminalscollection (the same compose function, now run server-side; disk saving keeps using it too). The two-halves split lives on as internal types — the fence that keeps sensors from writing memory — it just stops being a wire format.
Session state and restore — the host restores itself
Today session state lives in kolu-server’s config store, and restore is client-driven: after a restart the browser re-creates saved terminals one create call at a time, with a known race against PTYs that survived. Under padi, session state lives with the process that can actually guarantee it, and the client stops participating:
| state | lives | survives |
|---|---|---|
| live PTYs + scrollback | kaval (RAM) | padi restarts, kolu-server restarts, ssh blips |
terminal session — authored records (layout · chrome · active tile · panel sizes · memory · restoreTarget) + the restore-relevant awareness projection |
padi’s state-root, on the host (persistent storage — pinned) | host reboots included — at a reboot only the PTYs die (kaval’s row); the records park for restore |
| user preferences · binding pool (which hosts, last binding per view) | kolu-server conf-store | kolu-server has no terminal state to lose |
| per-tab view posture (canvas-maximized, minimap-expanded, …) | browser localStorage, namespaced per binding | host switches don’t collide two hosts’ view-state. (Active tile + panel sizes are NOT here — they ride padi’s authored session via the chrome procedures, as today’s setActive/setSubPanel persistence does) |
The four restore flows, each owned by the layer that knows:
- padi restarts (deploy, crash): on boot it adopts surviving kaval PTYs by id, reconciles them against its persisted session, re-seeds fold memory from its store, and re-runs sensors fresh (the #1031 posture). By the time any client binds, the canvas is already whole — no client participates.
- Host reboots (the PTYs are gone): padi’s boot finds saved records with no live PTYs and marks them
parked— a distinct state. Deliberately-slept terminals staysleeping, untouched: waking a slept terminal resumes its agent, so parking must not blur the two (they restore differently, exactly as today). padi never auto-respawns agents with nobody watching — and a warm server-side connection is not a watcher — so restore is never a side effect of connecting. The browser shows today’s restore card over the parked records, and your click callssession.restore({resumeIds})(per-terminal opt-out preserved). The respawn then runs inside padi, the single writer, and calling it twice is harmless (each record flips parked→active exactly once). The old client-loop race is gone by construction. “Import session” becomes a padi procedure too. - kolu-server restarts: nothing happens to terminals at all — padi’s registry stays warm (strictly better than today, where every restart re-derives all metadata). The browser reconnects and rebinds; the canvas is byte-identical.
- Browser reload / device switch: pure re-subscribe. The client no longer contains respawn logic of any kind; a reload against a rebooted remote host lands in flow 2 on that host’s padi, same as local.
Two safety rules came out of W1’s shipping rounds, each pinned by a test that fails if its guard is removed (reattach.test.ts, restartLocal.test.ts): nothing but the user may ever empty or shrink a non-empty saved session — parked records freeze the autosave, and the restart’s capture→drain→park window freezes it entirely (that transiently-empty moment is exactly where a session got eaten on a real host); and a daemon we adopt that isn’t the one we recorded pairing with (compared by startedAt) is treated as a fresh boot — preserve and park, never overwrite the saved session with the new daemon’s emptiness.
padiSurface — one contract, with per-member forwarding semantics
padi serves a new surface, padiSurface 1.0. (The old terminalWorkspaceSurface is frozen at 3.0 and dies with the pulam daemons at W2.3 — growing it instead would have forced every new member through three packages we’re deleting, because a served surface must implement every member or refuse to start.) Members: version + status cells; the activityFeed (MRU) + session cells (padi-owned on the wire from W1; conf-store storage moves to the state-root at W2.2); the composed terminals collection (record states active | sleeping | parked; the dock projection carries the optional host axis, reserved); the urgency projection (awaiting counts + terminal ids, recency-free — the sole thing kolu-server reads from every warm binding, for badge fan-in); activity stream; repo/file {seq} pulses; fs/git procedures (incl. worktree ops); byte procedures; transcript.exportHtml; lifecycle + chrome procedures; session.restore / session.import; terminalExit event; terminalAttach. Beside the surface, the frozen control core (hello · version · drain · clock.now) that never versions.
Per-connection scope is new machinery, and we say so plainly. Today the framework wires every member to one store at startup, and every websocket shares one router — nothing is per-connection. W2/W3 add it: one mirror store per bound host (shared by all connections watching that host), a router chosen by the connection’s declared host, and the client’s global wire objects rebuilt per binding, so an in-app switch is a clean rebuild (v1 of the switch may simply be a page reload). Whatever part of this lands in @kolu/surface* goes through the usual drishti check.
The contract’s one novelty, forced by review: every member declares how it may be relayed. Value members (cells, collections, pulses) can be held open across a hiccup and replayed — replaying a value is harmless. Byte streams (terminalAttach, activity) must fail through instead: if the padi↔kolu-server leg drops, the browser’s stream must end too, so the existing retry reconnects end-to-end and a scrollback snapshot only ever arrives as the first frame of a fresh stream. (Holding a byte stream open and splicing a replayed snapshot into a live terminal would corrupt the screen — the relay helper makes that a type error, not a convention to remember.) Attach keeps its special path — one dedicated stream per subscriber, straight through every hop — and the shipped overflow signal (#1591) rides it unchanged. The relay helpers grow out of pulam-web’s reserve into the shared surface stack, behind the usual drishti check.
Rejected shapes (each killed by a specific review finding)
- Per-terminal
HostLocationtagging (the finale plan): checked the host at a dozen places when one place (the connection) suffices — and the client never needed it at all. Closed with #1637–#1640. - Merging kaval into padi: padi bundles the system’s highest-churn code (agent sensors, fs/git serving); kaval earns “restarts only on contract change” by containing nothing that churns. Fusing them means sensor churn kills PTYs. padi restarts are instead cheap by design: re-seed from the state-root + fresh sensors (the #1031 posture — warm across web-shell churn, honestly re-derived across padi churn).
- Server-global binding with a “rebind + client reset” switch (the brief’s own first draft): behind a constant origin, in-flight id/repoPath-keyed calls and every other device silently straddle a rebind — a two-actor temporal convention with a race window. Replaced by per-connection binding scope: a call minted under one binding cannot reach another, and other devices are untouched by your switch.
- Local padi as in-process library, process only for remote: keeps two assemblies (the drift the one-library-two-homes era already suffered), keeps kolu-server fat, and forfeits warm-across-restart metadata and multi-client (kolu-tui) for the local host.
Where this stands — and what’s next
As of 2026-07-12, in order:
- W10 — hosts survive a restart shipped (#1772) 2026-07-12 in the ratified conf-store shape, clearing the numbered ladder.
- W12 — restore survives an unclean kaval death shipped (#1784) — independent of W10 in code (it lives in padi’s agent sensor, not the host pool). The seedling’s exit-flush guess was falsified by the incident forensics + a live kill-9; the real defect and cure are recorded in its section.
- The consolidation runs in parallel — its plan of record is A Complete Surface Runtime (the kernel + bridge PR sequence); the remaining parked padi-area cleanups are listed below.
- Parked, not forgotten: W3.4 (remote e2e parity in CI) waits on the mock-agent burn-down (#1689); W4’s honest-clocks piece 5 is a named fast-follow; W7’s field-acceptance leg (srid confirms the camera symptom gone on a real deploy) stays open until observed; W5’s drishti pair (srid/drishti#93 — the
alertscell) is still in review.
The law of every PR in this plan (srid, promoted note-wide 2026-07-04): a PR is self-sufficient and end-to-end — it ships NOTHING that isn’t consumed within it. No framework halves awaiting their consumer, no pools nobody switches, no contracts nobody serves: a capability with zero consumers is a commit stage inside the PR that consumes it, never a PR of its own (W2.2’s “a daemon with zero consumers is a commit stage” ruling; the #1651 complete-but-unconsumed adapter is the canonical tell). The one sanctioned exception is a framework PR whose consumer-proof is a paired second consumer landing with it (the W2.1/drishti precedent).
W12 — restore survives an unclean kaval death (shipped #1784 — independent of W10)
In simple words: before W12, if kaval died cleanly your agents came back after restore — but if it was killed or crashed, you got your terminals back empty: the agents were gone, and you restarted each one by hand.The 2026-07-12 production incident: prod’s kaval was killed un-gracefully; session restore brought back the terminal list, but every agent came back as a bare shell and srid restored them by hand. On zest, a graceful kaval restart restores agents fine — the two outcomes differ only in how kaval died. After W12 the two cases look the same: restore brings the agents back either way. The only thing an unclean death may still lose is the last few moments of screen output.
The defect, precisely (corrected 2026-07-12 after the incident forensics + a live kill-9 — the seedling’s exit-flush guess was falsified): the facts are not written at exit — padi already journals restoreTarget on every change (the always-on autosave). The real defect: an unclean kaval death makes each terminal’s agent sensor emit an authoritative agent-ended — because it can no longer observe the terminal through the dead pty-host — and the fold turns that into restoreTarget: none, which the always-on autosave journals over the good exact targets ~0.5s later, before any restore. On 2026-07-12 the false ended fired at 09:38:39.923, ~3ms before the endpoint was even marked degraded, so no freeze/park-on-degraded guard could win that race — the emit itself is the defect. The cure (structural, not a race-win — hardened after an independent perfection review): tell the two facts apart by the triggering foreground sample’s OWN content — pure data, so there is no cross-stream ordering to lose. A resolved-null whose foreground is a defined non-shell process (the agent’s own pid, its session file gone — exactly what an unclean death leaves) is ambiguous: emit unknown, and the fold keeps the last agent and its resume id — for a pre-death buffered burst of any length. A resolved-null whose foreground is shell-idle (the shell is back — the post-quit state) is a genuine end and still clears the target, so restore never resurrects a dead agent. Safety leans on one pinned invariant: blindness never presents an undefined foreground while an agent lives (the tap’s error handler doesn’t reset the last sample), so a dead observer always shows the stale defined agent pid. The resume id survives an unclean death by construction. (A companion fix freezes the autosave across session.restore and re-parks a respawn that fails mid-restore, so a kaval death during restore re-offers the card instead of deleting the terminals.)
Shipped #1784. The grounding overturned the seedling’s two guesses: the resume facts live in padi, not kaval (kaval persists no session file at all), and they were already journaled on change — so the “move exit-flush to change-journaling” framing was moot. The fix lives entirely in @kolu/padi: the agent sensor’s sample-content discriminant (a resolved-null with a defined non-shell foreground emits unknown, not an authoritative end) plus a restore-respawn seed so restore re-persists the surviving exact, never none, and a freeze + re-park so a kaval death mid-restore never deletes terminals. Witnessed on a real kill-9 — restoreTarget stayed exact across the kill (100/100 polls) and restore brought the agent back on its exact prior session (same sessionId + transcript). Independent of W10: that lives in kolu-server’s host pool; this lives in the per-host padi.
Consolidation and parked cleanups (unnumbered — a parallel, ongoing track)
W2 and W3 shipped at pace (a deliberate call); the consolidation pays the pace’s debts — refactors, decomplecting, framework-level electricity upstreaming, deletions. Nothing user-visible ships here; that is the point. The active plan of record is A Complete Surface Runtime — the kernel + bridge PR sequence, byte-identical on screen, each PR proven by what it deletes. A shipped phase’s deferred review items land in the note that owns the area — or below, as a parked cleanup with a named gate; an item that never ripens is deleted with a reason, never silently kept.
Parked, each with its gate (none is schedulable until its gate opens):
- Upstream padi-tui’s wait-until-predicate helper into
@kolu/surface— gated on kolu-tui existing (the second consumer that proves the boundary; the helper carries a ledger tag comment inpackages/padi-tui/src/). If kolu-tui never ships, delete this item with that reason — never upstream on one consumer. - Delete the legacy-kaval migration arm — at a version srid names. The W2.2 adopter (binder legacy-socket hint, padi’s legacy-adopt branch, port-keyed discovery in
socketPath.ts, theadoption-upgradeVM tests) exists solely for pre-W2.2 upgrades; done when no code path can dial a port-keyed kaval and the release note names the required upgrade path. - Split the sensors out of padi’s restart hash — only on recorded deploy-pain evidence (a stretch where sensor-only changes forced padi restarts users noticed). The restart is cheap by design; until the evidence exists this is deliberately a no-op, and never serve stale detection (#1031) governs any eventual split.
- Retire the near-unreachable
PADI_MEMORY_READ_ERRORcontract arm — #1699 moved the memory read onto the in-process mirror, which cannot fail like the remote dial the arm was built for; retiring it is a contract change and ships as its own deliberate wire-version decision, not a rider. - The fine-grained CI token — srid mints a public-repo read-only PAT (its only job: authenticated rate limits for flake-input fetches); an agent wires it via the pool provisioning path (stdin, never in the store or logs); done when the personal token is rotated and absent from every box.
- Flaky-test debt — a standing root-cause pass dispatched off the flaky tracker (fix or quarantine-with-reason, never re-tuned retries); done when a full both-platform e2e run passes without retries twice consecutively, or every tracked flake carries a root-cause note and a quarantine decision.
W3.4 — remote e2e parity in CI (parked — machinery done + proven; the gate waits on the parity tail: the remote-bind parity note, srid 2026-07-05)
Today the full e2e suite (~493 scenarios) runs in CI per-PR locally only; the KOLU_PADI_HOST path has zero CI coverage (the sandbox claim is honest but the exposure is real — the two live bugs of 2026-07-04 were both found by hand). This phase closes it: every PR’s CI runs the existing e2e suite AS-IS, twice — local (as today) and with kolu-server bound remotely (KOLU_PADI_HOST=<a second pu box leased from the pool>). No tagged subsets, no reduced tier: the same scenarios, a real ssh hop.
Mechanics (we do NOT control the pool boxes’ config — everything rides ssh we already have): the CI recipe leases a pair of pool boxes — box A runs the lane as today, box B is the bind target; box B’s per-run hygiene (state wipe before, destructive-ack, provisioning via the baked drv map — which also exercises the cross-arch arm when the darwin lane binds a linux box) is done over the ssh session, never via box config. The just e2e-ssh transport lane folds into the same node.
The real work is the burn-down so “as-is” is honest: (a) the ~70 scenarios that failed the one manual full-suite ssh run need their tight Playwright waits widened to remote-tolerant timeouts — behaviour was correct, the waits were not; (b) the agent-state scenarios get an honest mock — decided 2026-07-04 (srid), replacing an earlier copy-files-over-ssh idea: a small pretend-agent program runs INSIDE the kolu terminal like a real agent, driven through the terminal itself (“act busy”, “act done”), writing its session files on its own machine at the real default paths. Wherever the terminal is, the files land on the right machine by construction — the tests contain no remote-vs-local branches at all, and the pretending gets MORE realistic (a foreground process writing its own files is exactly what production senses — including the file-watching layer where #1680 lived); (c) lane wall-time budgeted (remote suite ran ~2× local on the manual run) — it runs as a parallel odu node on the paired box, so the wall grows only if it becomes the slowest node.
- Done: a PR that breaks any feature only-under-remote-binding goes RED in normal CI on both platform lanes; the remote node’s scenario count equals the local node’s (no silent exclusions — an excluded scenario is a named, counted skip with a reason); the pair-lease returns both boxes clean.
Future work — demand-driven
(The researched, tiered menu for everything below — ecosystem survey × shipped primitives, plus the promoted port-preview and shared-canvas notes — lives in remote-terminals-future.)
Cross-host dock (B-lite: foreign hosts’ rows, urgency-ranked, click = switch + focus — the host axis is already in the contract) is the first escalation if switching proves insufficient. kolu-tui — the graduation proof: consume padiSurface over the socket/ssh without kolu-server; done only when that named path renders a live canvas (the path, not the feature, is the unit of done). Hybrid canvas (model B) only if one-window side-by-side proves a recurring demand — it stays an aggregation layer over N bindings, zero daemon changes.
Risks, named, with mitigations
- Attach latency gains a local hop (kaval→padi→kolu-server vs kaval→kolu-server). Measure typing-echo p99 before/after W2 with a budget (< 5ms added); if breached, the carve-out permits a raw-byte relay on the padi→kolu-server leg (frontDaemonOverStdio-style, no decode) without touching the contract.
- Sensor code changes often, and any padi change restarts padi (the restart-hash covers the whole package). Accepted: a padi restart re-reads its store and re-runs sensors — seconds, PTYs untouched. If deploys make this hurt in practice, the sensors can later be split out of the hash; we’ll decide that on evidence, not up front.
- Multi-client writes (two kolu-servers, one padi): padi serializes all chrome/layout procedures — one writer, last-write-wins, concurrent editors see each other’s moves live. Accepted semantics (same as any shared session).
- drishti gate: the graduated forwarding helpers (per-member policy, input-keyed forward) and any per-connection scope machinery landing in
@kolu/surface*— paired drishti PR, pinned to final kolu HEAD, per.claude/rules/surface.md. - N warm bindings have a standing cost (one ssh session + one urgency subscription per host, even while parked elsewhere). Bounded: the pool holds only user-added hosts, the subscription is the count-sized urgency member (never
terminals), and an unreachable host degrades its chip, not the app. Costed explicitly rather than discovered in production. - Session-file migration: one-shot import at padi’s first boot, then hard cutover — a failed import crashes loudly (fail-fast), never silently starts empty.
- Testing W2.2 on zest: that box runs TWO instances sharing
~/.config/kolu/config.json— the deployed one and afaint-bottomdev worktree carrying an older, pre-fix pairing implementation that can null the shared file and poison any repro. Stop or reconcilefaint-bottombefore testing there. (The state-root move being tested is exactly what ends this collision class.)
The shipped record
Each phase, compacted to what shipped and the facts that still govern; the full build narratives live in git history and the PRs. When packages are born — graduation is a scheduled event, not a side effect:
| package | born | why then, not earlier/later |
|---|---|---|
@kolu/padi |
W1.C | location is structure: code destined for the daemon must not camp in packages/server. Package at W1, process at W2.2. The dependency arrow: padi imports no kolu app package (libraries only) — the app imports padi, never the reverse. Otherwise app churn would pollute padi’s restart hash, and kolu-tui (which exists to prove padi needs no kolu) would drag kolu back in |
| re-serve machinery | W2.1 ✓ #1661, into existing @kolu/surface / surface-nix-host |
it generalizes proven pulam-web code for a second consumer (kolu-server) — that is the graduation moment; no new package, it extends the receptacle that already owns mirroring (drishti-gated) |
| the control core | lives in @kolu/padi |
extraction is gated on electricity test ② (a real axis of variation), and with one daemon the axis doesn’t vary yet — nothing to encapsulate. If a second daemon adopts it, that’s test ③’s after-the-fact proof and it graduates to @kolu/surface-daemon. (Corrected 2026-07-03: an earlier wording here inverted the doctrine into ‘proof before extraction’ — the second consumer is the proof extraction was right, never the gate for doing it) |
padi’s wait (the agent done-signal) |
W2.3, a subcommand of the padi binary | no new package: the kaval-tui precedent (separate TUI package) is for a full client; a done-signal probe rides the daemon’s own CLI |
kolu-tui |
future work, new package | a new consumer app, not an extraction — it exists to prove padiSurface serves a second frontend (test ③’s proof). Demand-driven, not scheduled — nothing forbids it earlier |
W0 — decks cleared (all three landed)
All three landed: the freeze of terminalWorkspaceSurface at 3.0 + pulam-web retired (#1650); the overflow frame (#1591, kaval contract 5.0 — typed overflow → re-attach, the recovery loop later migrated into padi at W2); and the four in-flight PRs closed (#1637 #1638 #1639 #1640, 2026-07-01), the pre-padi plan marked superseded.
W1 — the padi seam, in place — shipped #1652
Shipped 2026-07-02 — one PR (contract → motion → rewiring commit stages): the whole root terminal.*/git.* RPC namespace deleted member-by-member into padiSurface procedures; @kolu/padi exited complete as a library (W2.2 gave it the process). Still standing from this phase: the seal — tests pinning the boundary both ways (packages/server contains no terminal code and imports only padi’s public entries; padi imports nothing from kolu’s packages, transitively); permanent trace logging on session writes (every destructive write logs a stack — the residue of three session-loss fixes, each pinned by a failing-first test); and the typing baseline (#1660: p50 2.14 ms · p99 4.36 ms — the reference W2.2’s ≈ 9.3 ms ceiling was set against; the baseline note).
W2 — padi the process (local) — shipped, three PRs
W1 sealed the domain behind the contract as a library; W2 gave it a process. The note-wide self-sufficiency law was first articulated here.
W2.1 — the re-serve machinery in @kolu/surface* — shipped #1661
Shipped 2026-07-03 — the policy-driven re-serve machinery (streams + events re-serve; cells always fold; procedures always forward), landed first to freeze the surface API before the big PR. drishti adopting the new initialKeys reconcile hook (srid/drishti#84, fixing its own ghost-PID bug with it) was the second-consumer proof.
W2.2 — padi the process: the binary AND the cutover — shipped #1664
Shipped 2026-07-03 — the padi binary born and kolu-server bound to it, one PR (a daemon with zero consumers is a commit stage, not a PR). Standing doctrine born here: deploy-adoption — a deploy must ADOPT a compatible running daemon and its live PTYs; restore is the path for a DEAD daemon only. Also still governing: newest-wins convergence (a newer binder drains a skewed padi via the control core and respawns its own closure; an older binder is refused, never recycles anything), padiLink on the shell surface (padi cannot serve its own unreachability), parked records immutable (the no-non-user-writer invariant), and the measured hop — +1.3 ms p99 against the +5 budget.
W2.3 — padi-tui is born, and the burial — shipped #1665
Shipped 2026-07-03 — the dial carve (@kolu/padi/dial; the seal’s allowed set is exactly {assembly, surface, dial, log}); padi-tui born (wait/status off real agent state · create --parent / --worktree · the live activity stream · $PADI_SOCKET stamped on every padi-spawned terminal, so the agent-drives-agent loop is flagless); and the burial — packages/pulam + pulam-tui deleted, terminalWorkspaceSurface + its assemblers deleted, every legacy-conf strip/import backing the file up first.
W3 — remote (the binding · the full-peer gaps — the CI-parity leg W3.4 is parked, under what’s next)
W3.1 — the remote binding — shipped #1675
Shipped 2026-07-04 — provision + front a remote padi over ssh: KOLU_PADI_HOST with RemotePadiSession as the ssh arm through the same re-serve seam; the arch-keyed drv map ships kaval inside padi’s closure (one dial provisions both, cross-arch verified); padi --stdio fronts the durable daemon; convergence hardened for the transport (the instance-keyed drain fence, adopt-loudly on budget exhaustion); every degraded bind a standing dialog state naming the bound host; deterministic bounded daemon logs on all spawn paths; and the ssh e2e lanes (just e2e-ssh + e2e-ssh-2box, destructive-ack-guarded). Standing permanently: the ssh-user caveat — the remote padi runs as the SSH user; 0700 sockets make the SSH identity the daemon owner.
W3.2 — the full-peer gaps — shipped #1685 #1686
Shipped 2026-07-05, two self-sufficient PRs. Preview (#1685): a file on a remote machine now previews for real — the route asks padi for the bytes in bounded 8 MiB chunks, pinned by a strong ETag so a file changing mid-read aborts loudly instead of serving spliced bytes. Inventory (#1686): the “Running daemons” list shows the connected machine’s daemons, each group labeled by machine. Together they met the W3 “full peer — nothing silently wrong-host” promise for the single-host binding.
W4 — the switch — shipped #1714
Shipped 2026-07-07 as @kolu/surface-map + the KOLU_PADI_HOST-gated multi-host canvas: kolu-server holds a warm pool of padi connections (buildHostRegistry, one PadiSession per machine, opened on demand), each browser tab names its machine, and switching swaps the client’s wiring in place — no reload — like switching tmux sessions. Your laptop tab can sit on zest while your phone sits on sincereintent, against the same kolu, neither disturbing the other. (The machinery primer: the surface framework’s hosting side, taught — serve → mirror → sessions → the registry.) Deliberately, the only ways in until the feature stabilizes are the picker under the command palette’s Labs group and KOLU_PADI_HOST (which sets the default host only — the picker switches freely away from it); the ChromeBar picker graduates when it’s stable.
Still live from the ship: honest clocks (piece 5) is a named fast-follow — the serving half exists (a wall-clock probe every daemon answers), and the offset half now ships in the surface framework: makeSession measures the far-end offset off the framework-reserved system.clockNow at admit and carries it on the session’s connected state, which a keyed SurfaceMap folds into EntryStatus.connected (PR3 — the offset MEASUREMENT graduated to the framework-reserved system.clockNow, a member every surface answers; the padi-specific control.core.clockNow is kept forever — a frozen-core member never versions, so it stays as the cross-version skew channel old binders cross on, living beside the new path, not replaced by it). The remaining consuming piece is the client formatters reprojecting host-stamped times through that offset; until they do, remote hosts’ ages mislabel (a pre-existing W3.1-era gap, not a switch regression). The per-host-state bugs the deploy surfaced (focus, splits, camera) got instance fixes in-PR; their class fix became W7. The ledger’s L11 sweep (the client’s interim wire-shape scaffolding) landed with cleanup campaign 3 (#1747).
W7 — per-host state by ownership, not enumeration — shipped #1723 (+ the srid/drishti#91 framework-gate pair)
Shipped 2026-07-08 — the class fix for W4’s “forgotten field” bug family (focus, splits, camera each quietly living at app lifetime while the tiles became per-host): per-host-ness flipped from enumeration to ownership. scopedByEntry lives in @kolu/surface-map; kolu’s hostScope/hostScopes dissolves the HostView record and its kin into one per-host reactive owner, so everything born inside the canvas subtree is per-host by construction, and a boundary-guard test (canvas/canvasBoundaryGuard.test.ts) fences state born outside it. The design story lives in surface-map-101.
Two live edges: K1’s completion shipped as W9 — the per-host wire subscriptions the owner deliberately did not retain. Field acceptance still open: srid confirms the live camera symptom gone on a real deploy (the timing a unit pin structurally can’t see) before this phase closes.
W8 — remote terminals, documented honestly — shipped #1722 #1727 (a docs phase)
Shipped 2026-07-08, two PRs — a docs phase seeded by a live incident (a kolu remote terminal is an ssh session, not the machine’s GUI login session, so tools like gh that lean on the macOS Keychain silently break over the bind): the atlas analysis of the ssh-session environment gaps (#1722) and the user-facing website page (#1727 — remote hosts as a feature: enable · add · switch, with the gaps as one compact limitations/FAQ section, marked alpha). The living rule: the catalogue grows by incident — every future “works locally, breaks in kolu” report lands a row (the flaky-tracker discipline, applied to environment gaps).
W6 — the honest connect — shipped #1730 (+ the srid/drishti#92 pair — both merged 2026-07-09)
Shipped 2026-07-09 (merge e4acb5b2f) — a first connect to a fresh remote host can spend minutes in a Nix copy+build, and the canvas used to show one mute “Connecting…” for the whole window; now the connect overlay names the phase, streams a live tail of the actual build output, and shows a server-stamped elapsed — and the connection type was made perfect along the way (the server’s state sum reaches the browser as a discriminated mirror, not a lying 4-nullable-field product; seven names collapsed into the family below; remoteProgressLines and the in-band [local]/[remote] prefixes deleted). Live testing hardened the batch: the coarse-chip/fine-cell divergence trap is a type error (connectPhase exists only on the not-yet-connected facts arms); the cell is floored on transport liveness; and the connect copy has one authority — CanvasMode arms carry no strings and no per-phase flags, display derives from the frame’s data, so a silent or flickering connect window is unwritable. ConnectPhase graduated to a @kolu/surface-remote export beside ConnectionInfo, its owner; kolu’s overlay is the consumer (drishti’s color map keys the full phase union directly, so it imports nothing extra).
The type family as shipped — the reference for the connection vocabulary:
// @kolu/surface-remote — ONE framework type; the connector declares its own phases
type SessionState<Prov extends string = never> = {
log: readonly { source: "local" | "remote"; line: string }[]; // last 20; per-EPISODE (reset on a down→up crossing)
sinceMs: number; // current episode's elapsed, stamped on the SERVER's clock — a duration, never a foreign epoch
} & (
| { phase: "connecting" | "connected" | Prov } // up — error fields don't exist
| { phase: "disconnected"; error: string; cause: "network" | "remote" }
| { phase: "failed"; error: string; cause: "remote" } // network is never terminal — by type
);
type SshProv = "probing" | "copying" | "building";
// probing = the warm realise-probe (is padi already there?) — presented calmly, no build UI
// copying = nix copy --derivation … (the .drv push — fast)
// building = ssh $host nix-store --realise … (the remote compile — the minutes)
// sshConnector: SessionState<SshProv> · local endpoint: SessionState<never>
// browser cell: ConnectionInfo = SessionState<SshProv>, LITERALLY — the zod schema's
// inferred type is pinned identical by a test-d file, so schema/type drift is a compile error.
// ConnectPhase = Exclude<ConnectionInfo["phase"], "connected"|"disconnected"|"failed">
// — the one narratable-phase vocabulary, exported beside its owner; consumers
// derive from it, never hand-list phase names.
W5 — cross-host attention — shipped #1759 (the drishti pair srid/drishti#93 — the alerts cell, the first reactor consumer — is still in review)
Shipped 2026-07-11 (merge 52d60bf1b) — an agent flipping to awaiting you on a background host now reaches you: an OS notification whose click is one action (switch host + focus the tile), the PWA app badge summed over live hosts, beside the switcher chips’ per-host counts (W4). The architecture as ratified: attention is level state carried in an ordinary per-entry cell — kolu minted no wire member (it consumes padi’s existing cells.urgency, the tiny projection W7’s K1 ruling keeps hot per host) — and the PR’s framework slice is phase 0 of the reactive bridge: reactor.ts (source · scan · derived.cell, the engine lint-banned behind it) in @kolu/surface, Subscription.updated() change pairs, mirrors-never-fabricate in @kolu/surface-remote, the eager watchByEntry watcher in @kolu/surface-map, and notify delivery in @kolu/surface-app. The design is taught end-to-end in the attention primer; the bridge’s later phases, sequenced after, live in the bridge note.
W9 — instant host switch-back — shipped #1764
Shipped 2026-07-11 (merge 10070a662, two-platform CI green) — host switch-back is now the instant, tmux-grade move it was designed to be: the per-host wire subscriptions (terminal keys, terminals, saved session, daemon status, activity feed) live in the retained app-lifetime per-host owner (createHostWire inside scopedByEntry), so switching back has no resubscribe and no pending window — the dock, tiles, and RightPanel stay mounted. The Code tab’s queries are scoped per host by ownership in a parallel per-host owner (right-panel/hostCodeTab.ts): paused while backgrounded, resumed from the held value on switch-back, disposed on membership exit. Review shaped the ship — the first keep-last/LRU query cache and the component-owned retention were both replaced by ownership, and the old perHostPolledQuery/createRepoPolledQuery constructors were retired outright so component-owned retention is unspellable. Deliberately excluded, still: xterm/WebGL stay active-host-only, and a grounded follow-up investigation pinned what that exclusion concretely costs — a full-scrollback replay on every cross-host switch: the old host’s <Terminal>s unmount and the new host’s mount, and each mount re-attaches from scratch — kaval serializes its whole headless mirror with no viewport bound and the client writes the entire snapshot back into xterm, so the “sub-second re-attach paint” is a full serialize + full re-write, bounded only by the 10,000-line mirror cap.The mechanism, file-cited: same-host terminal switches replay nothing — inactive tiles stay mounted and CSS-hidden (Terminal.tsx classList={{ hidden }}, #988), so switching activeId is a CSS reshuffle. Cross-host switches swap the tile set; on mount, snapshotOf calls serialize.serialize() with no viewport bound (packages/kaval/src/ptyHost.ts:705-708), emitting full scrollback + screen up to DEFAULT_MIRROR_SCROLLBACK = 10_000 lines (ptyHost.ts:52 — the cap exists because an unbounded serialize once caused heap OOMs), which the client writes wholesale into xterm before live deltas resume. The ratified direction is now scrollback backfill: a bounded attach snapshot kills the per-switch replay, and older history backfills into the terminal’s own scrollback on scroll-up. #1577’s separate copy-mode-pager shape was rejected for this purpose — the user only cares about the actual terminal scrollback — and its memory goal (the 10k-line RAM mirrors) remains open, untouched by that plan. No server or padi changes.
W11 — one verb per fact: user-remove vs system-retire — shipped #1775
Shipped 2026-07-12 (squash 7f2811539) — surface-remote’s pool now speaks two verbs for its two facts: remove(host) means the user removed it (fires the persist hook), retire(host) is the system shedding a faulted session (same teardown, no persist); kolu’s #1708 pump-death path uses retire, so an internal fault can never masquerade as a user decision — the prerequisite W10’s persistence needed. The gauntlet’s structural insight: the remembered fleet is its own ordered set (persistedMembership), a fact in its own right — never derived from the live pool’s entries.keys(), because retire deliberately desyncs live from remembered. Codex caught two edges, both fixed: remove-after-retire was a no-op, and a throwing listener could make retire reject.
W10 — hosts survive a restart — shipped #1772
Shipped 2026-07-12 (merge 9f516987c) — the hosts you added through the selector strip come back after a kolu restart; the only way a host leaves the strip is your explicit remove. Hosts are settings: the remembered fleet is one field in PersistedStateSchema, a value in the conf store beside every other preference — same store, same migrations ladder, same throw-on-corrupt behavior (the fail-fast doctrine inherited, not re-implemented) — written through the pool’s transactional persist-before-commit hook, which fires only on user-intent mutations (W11’s split). The unremovable local default never enters the field (it’s seeded in code — persisting it would mint a second authority for “local always exists”); an env seed (KOLU_PADI_HOST) stays declarative provenance, never captured into persistence; validation is loud — a persisted value naming local or carrying duplicates is rejected, not repaired. Boot re-enters through the front door: each persisted host flows the ordinary pool.add and the W6 connect pipeline (warming → probing → connected, or failed with its honest cause — no lazy-dial knob). The shape itself was a pre-merge overturn: the design originally persisted membership as its own separate hosts.json, killed when fact-checking dismantled its justifications — hosts are no different from settings.The separate hosts.json was zod-schemed with hand-rolled atomic writes (tmp + rename) and its own crash-on-corrupt rule. It was overturned by srid’s 2026-07-12 ruling when its justifications failed fact-checking: conf does not reset-to-defaults on corruption (it throws — clearInvalidConfig is false — the same behavior the separate file was built to guarantee), and no concrete incident distinguished the two designs. Hosts are no different from settings, so they ride the settings store.
Superseded by this note
The pre-padi R9/R10 decomposition (recorded in the portal; whole plan: PR-1/2/3, F-REMOTE, R10) — closed PRs #1637, #1638, #1639, #1640. Partially superseded: awareness-derive-store (the fold’s home moves to padi; owner-clock replaces consumer-clock; producer, fold, and types survive verbatim) and the terminal model (the reader-join collapses into padi’s composed terminals collection; the authored/snapshot split stays as padi-internal types). Unaffected: kaval’s design (pty-daemon) apart from the shipped overflow frame (contract 5.0, #1591); #1577.