Chat threads as terminals, over ACP

One conversation ↔ one kolu terminal — but the terminal runs an ACP proxy, not an interactive TUI. DM the bot in Xyne Spaces; pesu creates a tile whose process speaks the Agent Client Protocol to a headless agent behind it. Your message is a session/prompt; replies and tool calls stream back as session/update frames; permission requests arrive as data you can answer from the thread. The tile stays an ordinary kolu terminal: the proxy renders the frame stream in its PTY, so clicking through from XS gives full depth.

The wire

ACP is the boundary on both sides of the proxy — pesu speaks it over the socket, the adapter speaks it over stdio — so there is no invented protocol anywhere on the wire.ACP is NDJSON JSON-RPC 2.0; the spec’s usual transport is the agent’s stdio, and the proxy re-serves the identical byte protocol on a unix socket. The proxy adds only what a harness must own: spawn/respawn, cancel-with-grace, idle timeouts, logging. The proxy is the volatility receptacle — process lifecycle, transport, per-vendor quirks live there and nowhere else — which is what makes it a @kolu/* package rather than pesu internals. Note the coupling runs one way only: acp-proxy imports nothing from kaval, padi, or @kolu/surface — both of its faces are standard ACP — and kolu never learns about it either. Running it in a tile is a deployment choice (that’s what puts the session on the canvas); the same binary runs identically under tmux or systemd, and the package (packages/acp in this repo) could move out to its own repo later as a file move, not surgery.

Driving the agent through ACP instead of typing into its TUI deletes the whole observation problem for the bot path: no bracketed-paste races, no screen-settle detection, no turn-end heuristics, no transcript-file watching — session/prompt returns when the turn ends, and everything in between is a typed frame.Buzz (github.com/block/buzz) runs this exact stack in production — its harness spawns claude-agent-acp and consumes the same frames. Its scar tissue is our checklist: some agents keep streaming after session/cancel, so cancel gets a grace window then a respawn; idle timeouts must reset on any stdout byte because agents go silent during tool execution; an in-flight permission request must be answered cancelled before cancelling the turn or the agent deadlocks. Buzz’s client itself is hand-rolled and welded to its Nostr relay — a reference to read, not a dependency to take.

What the PTY path could never offer, this one gets for free: tool-call frames become live agentProgress labels in the thread, and permission requests become flow-JSON cards with buttons — approve a gated tool right from the thread, without opening kolu at all.

Decisions

Roadmap — HX

HX0 — an ACP agent in a tile

The @kolu/acp package and its two bins. The adapter to spawn is the proxy’s argv, so agent-agnosticism is structural, not a feature to add.

Nothing about the tile is special. An “ACP tile” is an ordinary kolu terminal whose child happens to be acp-proxy — kaval sees a process in a PTY, padi sees a terminal, and no kolu code changes anywhere. Create it like any terminal (lifecycle.create with acp-proxy as the argv), or just run the bin by hand in a shell.

acp-proxy is one session, end to end. It spawns claude-agent-acp over stdio, re-serves the same ACP byte protocol on the unix socket for clients, and renders every frame human-readably to its own stdout. So the tile shows the session — messages, tool calls, permission decisions scrolling by — but is never the thing you type into; all input arrives as ACP calls on the socket.

acp-chat is the stand-in for pesu until pesu exists: a REPL that connects to a proxy’s socket, sends each line as session/prompt, and streams the reply. It makes this phase usable without XS in the picture while proving the exact client path HX1’s pesu will reuse — same library, same socket, same frames — and it stays useful afterwards as a debugging client.

Test flows (this PR):

1. Create a kolu terminal running acp-proxy (or run it by hand in any shell) — it prints the socket path and the spawned adapter, then idles.
2. From another terminal, acp-chat <socket>; ask a question — the reply streams in the REPL while the tile scrolls the same turn as frames.
3. Ask something that needs a tool (“what does git status say here?”) — watch tool_call / tool_call_update frames appear in the tile, with the auto-answered permission line between them.
4. Ctrl+C in acp-chat mid-turn — the tile shows session/cancel and the turn stops (or the grace window expires and the adapter respawns).
5. kill the adapter’s PID mid-turn — the proxy respawns it; your next prompt works.

Verification, autonomous (how the implementing agent proves this without a human):

• The fake adapter is the test double — packages/acp/test/fake-adapter.ts, a scripted stdio ACP agent (the fake-XS trick, applied to the other side): it answers initialize / session/new, and on session/prompt replays canned session/update frames — message chunks, a tool_call + tool_call_update, a permission request. The prompt text is the directive: crash exits mid-turn, hang streams then never finishes (the cancel case), anything else echoes. Deterministic, offline, no auth, no tokens.
• One vitest per done-criterion, all driving the real bins (spawned processes, real socket — not in-process shims): round trip through the official client; proxy stdout captured and asserted to contain only frame-derived rendering; crash → respawn → next prompt succeeds; hang → session/cancel → grace expiry → respawn. The suite takes the adapter argv as a parameter, which is also how the second-adapter criterion is met structurally. This suite runs in CI (just ci) — it needs no agent credentials.
• Real-adapter smoke, run locally before the PR is declared done: acp-chat against claude-agent-acp and codex-acp with a one-line prompt (“reply with exactly: pong”). Both adapters inherit the dev box’s existing claude / codex logins — no interactive auth step. Capture the tile + REPL side by side as the PR’s ## Evidence.

Done when (the phase exists to prove the socket path, so the criteria pin it):

• A prompt travels acp-chat → socket → proxy → adapter through the official client library on both hops; the reply streams back the same way.
• The tile’s transcript is rendered from session/update frames only — the proxy never reads ~/.claude/projects or any session file.
• Kill the adapter process mid-turn: the proxy respawns it and the next prompt works.
• session/cancel is exercised, including the grace-then-respawn fallback.
• The same flows pass against a second adapter (codex-acp) changing only the proxy’s argv — proving nothing in the package is claude-shaped.

HX1 — the XS round trip

pesu: signed webhook (APP_MENTIONED / DIRECT_MESSAGE) → allowlist gate → lifecycle.create a proxy tile via padi → store the bind → session/prompt → post the final assistant text to the thread. Later messages in the thread resolve the bind and prompt the same session. The XS leaf — webhook.ts + HMAC tests, xyneApi.ts + fake-XS tests, fail-fast env config, logging, allowlist, nix run .#pesu — ports from #1810.

The thread’s ceremony-free home is a DM (every message is delivered); in a channel each message must @mention the app — the apps platform has no follow-the-thread.Follow-the-thread (replying in a bot’s thread without re-mentioning) exists only in XS’s separate bot-catalog system, not for installed apps. Two recorded options if the per-message @mention grates: an XS-side feature request for app thread-follow, or a pesu-side poller over one dedicated channel’s channelHistory — not built until wanted. pesu answers the webhook 200 immediately and works async — delivery is fire-and-forget, so a slow handler loses events. Deployment shape: pesu binds 127.0.0.1 as a systemd user service fronted by Tailscale Funnel for the stable public webhook URL; secrets reach it only as environment variables — never the repo, logs, or an agent transcript.

Test flows (this PR):

1. DM the bot a question — ack arrives, a new tile appears on the kolu canvas, the reply lands in the thread.
2. Send a follow-up in the same DM — no new terminal; the same session answers with the earlier context.
3. Keep the tile open in kolu while asking a third question — watch the turn scroll by as frames while the thread gets the reply.
4. Have someone not on the allowlist DM the bot — they get the one-line decline; no terminal is created.
5. curl the webhook with a bad X-Xyne-Signature — rejected, nothing posted, nothing created.
6. In a channel, @kolu <ask> works; a plain message in the same thread without the @mention does nothing (the documented apps-platform gap).

Done when: DM the bot → tile exists on the host → reply lands in the thread; a second message continues the same session; opening the tile shows the live frame log; a forged-signature request is rejected (pinned by a test); a non-operator gets a one-line visible decline, never silence.

HX2 — approve from the thread

A session/request_permission frame becomes a flow-JSON card in the thread — one button per option the request carries. The /flow/action callback answers the ACP request; the card updates to show the decision. The request stays open until answered or the turn is cancelled. Proxy-side auto-answer is switched off for bound threads.

Test flows (this PR):

1. Ask the bot for something gated (“push this branch”) — the permission card appears in the thread and the tile shows the pending request; the agent is visibly paused.
2. Click Allow once — the run unblocks; the card flips to the resolved “allowed by you” state.
3. Repeat and click Reject — the agent declines and says so in its reply.
4. Leave a card unanswered and cancel the turn — the request resolves as cancelled; no deadlock, no orphaned card.

Done when: an agent hits a gated tool, the card appears in the thread, clicking Allow unblocks the run — end to end, with auto-answer off.

HX3 — turn polish

The growing reply (updateMessage ≤1/s, cap-aware split); tool_call frames → agentProgress labels while the turn runs; deep link to the tile in the ack; acpSessionId in the bind so a pesu restart resumes the conversation via session/load.claude-agent-acp and codex-acp advertise loadSession; resume replays the session’s history as the same session/update stream, so the proxy’s tile rendering and pesu’s cursor logic need no special path. For an adapter that doesn’t advertise it, pesu starts a fresh session and says so in the thread — visibly, never silently.

Test flows (this PR):

1. Ask for a long task — one message grows in place (edited ≤1/s) instead of a stream of separate messages.
2. Watch the progress pill change labels as the agent moves between tools; it clears when the turn ends.
3. Click the deep link in the ack — kolu opens focused on that tile.
4. systemctl --user restart pesu mid-conversation, then send the next DM — the conversation continues with prior context and no earlier reply is double-posted.
5. Provoke a reply longer than 40,000 chars — it splits into consecutive messages instead of failing.

Done when: restart pesu mid-conversation; the next DM continues the same conversation with prior context, without double-posting earlier replies.

Later, on demand (small bounded additions, not phases): per-thread agent selection UX in pesu (the wire already takes any adapter); a small integration so proxy tiles light up dock state / awareness from the proxy’s own stream.

Background

Why chat at all. Anthropic shipped Claude Tag — one shared Claude per channel, pursuing tasks “over hours or days” — and Karpathy called the shape “the 3rd major redesign of LLM UIUX”: the agent as a persistent coworker that lives where the team already works.The proof-point Anthropic leads with is blunt: “65% of our product team’s code is created by our internal version.” The human’s job shifts from driving keystrokes to managing agents. In that world chat owns the breadth and the terminal owns the depth — which is this note’s spine: the XS thread is the conversation, the kolu tile is the live process you can step into.

Why kolu stays the substrate. kolu runs claude or codex under any chat app, on infrastructure you control — the model swappable, the shell and files real, the host yours. Claude Tag is one vendor’s model with the shell and files hidden in a cloud sandbox you don’t own; that difference is the moat, and the ACP wire preserves it (any adapter, any chat front end).Marc Andreessen, on what agent research converged on: “an agent is … a language model, and above that a bash shell … and then a file system, and the state is stored in files” — and “your agent is now actually independent of the model … you can swap out a different LLM underneath.” That is the architecture here, line for line; the headless session still acts through a real shell on files you own.An earlier revision of this note designed the bridge as a mirror of the coordinator — observe flowing out raw, act flowing in through one orchestrator terminal, with a fleet status board and a campaign feed as phases. Thread-per-agent over ACP supersedes it; fleet-level chat (board, feed) is campaign surface territory.

The XS apps platform, as grounded

The contract HX1 builds against — read from the xyne-spaces source (2026-07-13), not its docs, because XS has three separate integration systems that are easy to conflate: the bot catalog (backend-native bots), incoming webhooks (URL-embedded-secret message drops), and the apps platform — installed apps with a webhook URL, a bearer token, and a signing secret. pesu is an installed app; only the third system applies.

Auth, both directions, one secret. Deliveries carry X-Xyne-Signature — HMAC-SHA256 of the raw body with the app’s signing secret; no timestamp or replay scheme, so verification is a constant-time compare and replay is an accepted, recorded risk. The bearer token is a JWT signed HS256 with that same secret, no expiry — it lives until regenerated.

The app API (/api/apps, bearer): chat/postMessage (text, markdown, or structured content; conversationId targets a thread) · chat/updateMessage · chat/agentProgress · chat/channelHistory / chat/conversationReplies (catch-up after downtime) · channel/openDm · user/info.No webhook payload carries a sender email (the DM payload omits even the display name) — pesu resolves people through user/info and caches them. Event delivery is fire-and-forget with no retries: a delivery missed while pesu is down is lost; the read APIs recover.

Out of scope

• PTY-agent observation — session watchers, transcript export, and every interactive tile a human drives stay exactly as they are; this note adds a kind of process to run in a tile, not a change to kolu.
• kolu entity-model changes — no new session kind, no dock/awareness work in the phase tree (see Later).
• Bot secrets in a web UI — pesu config is env/systemd beside the daemon.

Done criterion (full tree)

DM the bot → a tile with a headless agent exists → replies stream into the thread → the agent hits a gated tool → you click Allow in the thread → the run finishes and the reply grows in place → restart pesu and the thread resumes → at any point, click through and watch the session in the tile.