tutorial
Make it a daemon.
In Your first surface, fleet-top died when you closed the terminal. Here we make it durable: one long-lived process that holds a single-instance gate, serves the same surface over a unix socket, and can be replaced by a fresh build while a client stays connected. The surface itself does not change at all — only how it is reached and kept alive.
Wrap the engine in a daemon
daemonMain is the entire gate → serve → teardown lifecycle. It claims an
atomic single-instance gate, serves the surface’s { group, handlers } over a
socket, waits for the
lifetime to end, then closes the socket and releases the gate — returning a
DaemonExit rather than calling process.exit, so the whole thing is drivable
from a test. The bin wraps it in daemonProcessMain, which owns the process
exit: when the daemon’s tenure ends, the process ends — top’s live sampler
interval can’t keep a finished daemon alive.
/**
* The daemon — part 1's `top`, now single-instance and durable.
*
* `daemonMain` is the whole `gate → serve → teardown` skeleton:
*
* acquirePidGate(GATE_PATH) — atomic single-instance claim (link(2)).
* │ held? → exit 0 (a live daemon already serves this scope)
* ▼
* serve { group, handlers } — the @kolu/surface unix-socket listener
* ▼
* wait for lifetime to end — { kind: "forever" }: only a signal / abort
* ▼
* close socket · release gate · return DaemonExit
*
* We pick `forever` — an idle `top` still watches your machine, so an idle
* timeout would wrongly kill it. `daemonMain` never calls `process.exit`; it
* RETURNS a `DaemonExit`, which is what makes the whole lifecycle drivable
* in-process from a test. The BIN half — mapping that value to an exit code
* and actually ending the process — belongs to `daemonProcessMain`: without
* it, `top`'s live sampler interval would keep this process alive forever
* after the daemon shut down (the lingering-daemon class).
*
* The same `createTop()` surface from part 1 is served verbatim — the daemon
* changes how it's *reached* (a durable socket instead of a fresh
* per-connection process), not what it serves. It hands the spine the SAME two
* fields every transport takes: the flat `group` and the tag-keyed `handlers`.
*/
import {
daemonMain,
daemonProcessMain,
stderrLogger,
} from "@kolu/surface-daemon";
import { HOME } from "../common/paths";
import {
readProcessIdentity,
selfProcessIdentity,
} from "../common/processIdentity";
import { createTop } from "./top";
daemonProcessMain({
name: "fleet-top daemon",
run: async () => {
const top = createTop();
// `finally`, not fulfilled-only: the release stage must run on the crash
// arm too (a `top.start()` or `daemonMain` throw reaches
// daemonProcessMain AFTER the sampler is disposed) — the same
// wrapper-finally teardown ordering kaval and padi use. Everything after
// `createTop()` sits inside the try so `top.dispose()` is structural.
try {
top.start();
return await daemonMain({
// gate, socket, anchor — all derived from home inside the spine
home: HOME,
processIdentity: selfProcessIdentity(),
readProcessIdentity,
group: top.runtime.group,
handlers: top.runtime.handlers,
lifetime: { kind: "forever" },
log: stderrLogger(),
onReady: ({ socketPath, pid }) =>
process.stderr.write(
`fleet-top daemon listening on ${socketPath} (pid ${pid})\n`,
),
});
} finally {
top.dispose();
}
},
});
We choose { kind: "forever" }: an idle top still watches your machine, so an
idle timeout would wrongly kill it. Note the daemon hands the spine the same
two fields every transport takes — the flat group and the tag-keyed handlers
— from the same createTop() runtime as part 1, served verbatim. (forever is
one of three lifetimes — the
Reference lists idleTimeout for a quiet
coordinator and boundToPid for a daemon a test/smoke run wants to die with it.)
anchor is the one required field with no default: the directory whose deletion
makes the daemon garbage, polled by the spine so the daemon
reaps itself instead of
outliving a deleted workspace. Once daemonHome supplies the rendezvous home,
that home is the on-disk identity — fleet-top anchors to it
(anchor: () => HOME_DIR) so a deleted home cannot leave a zombie daemon
holding unlinked gate/socket inodes.
-
Start the daemon.
pnpm run daemonfleet-top daemon listening on /run/user/1000/fleet-top/fleet-top.sock (pid 48213)That friendly line comes from
onReady;stderrLoggeralso writes a structured JSON line per event to stderr, so you will see a{…"msg":"daemon listening"…}line alongside it. Leave this running. -
In a second terminal, start it again.
pnpm run daemonIt exits immediately, logging one line that a live daemon already holds the gate (
daemon already running; yielding to the live instance), and steps aside. That is the single-instance guarantee:acquirePidGateis an atomiclink(2)claim.
Supervise it
A daemon you cannot spawn, watch, and replace is not much use. createEndpoint
is the supervisor half of the spine — it runs in the client, never the daemon.
converge(endpoint) is the only boot verb — an always-recycle boot: kill any
live survivor, wait for its pid to be reaped, spawn a fresh daemon so it outlives
us, connect, and handshake. Every transition is reported.
/**
* The supervisor — spawn, watch, and LIVE-recycle the daemon.
*
* `createEndpoint` is the supervisor half of the spine (it runs in the CLIENT,
* never the daemon). It takes the daemon from nothing to a live, handshaken
* connection and reports every transition:
*
* connecting → connected (recycled, socket up, handshake passed)
* connecting → dead (couldn't spawn / connect)
* connected → degraded (the daemon died mid-session)
*
* `converge(endpoint)` is the only boot verb: a live survivor is killed, then a
* fresh daemon is spawned — every boot exercises kill → `waitForPidGone` →
* spawn → connect (composed from `@kolu/surface-daemon`'s gate primitives).
* `survivableSpawnDriver` launches the daemon so it OUTLIVES us (systemd-run
* --user under a service; detached + unref otherwise).
*
* The finale is the LIVE recycle under a connected client: `recycle` runs the
* fixed `capture → drain → recycle → reattach` sequence. This demo makes no
* survival promise, so it supplies the degenerate steps (B2's boot recycle);
* part 3's remote fan-out is where the same sequence carries real per-host
* session state. The client we hold reconnects on the far side of the recycle.
*/
import { fileURLToPath } from "node:url";
import type { StreamingProcedure } from "@kolu/surface/client";
import { stderrLogger } from "@kolu/surface-daemon";
import {
converge,
createEndpoint,
recycle,
survivableSpawnDriver,
} from "@kolu/surface-daemon-supervisor";
import { Effect, Option, Stream } from "effect";
import {
bakedOsFactsBin,
osfactsSocketHolders,
processIdentityAsync,
} from "osfacts-client";
import { GATE_PATH, HOME, SOCKET_PATH } from "../common/paths";
import type { Memory } from "../common/surface";
import { connectTop, type TopClient, type TopIdentity } from "./connect";
/** The first frame of a snapshot-then-deltas member. */
function snapshot<T>(
stream: Stream.Stream<T, unknown>,
what: string,
): Effect.Effect<T, Error> {
// A member stream fails with `unknown`; the endpoint's `connect` contract is
// "a plain Error unless it is the branded skew", so the narrowing happens HERE,
// once, rather than at each caller.
const head = Effect.mapError(Stream.runHead(stream), (err) =>
err instanceof Error ? err : new Error(String(err)),
);
return Effect.flatMap(head, (head) =>
Option.isNone(head)
? Effect.fail(
new Error(`${what}: stream closed before its snapshot frame`),
)
: Effect.succeed(head.value),
);
}
const main = Effect.gen(function* () {
const log = stderrLogger();
// The daemon binary the driver spawns. In a Nix build this is the realised
// executable; from source we point `node` at the daemon entry through tsx.
const daemonEntry = fileURLToPath(
new URL("../daemon/main.ts", import.meta.url),
);
// ONE axis — where this program's osfacts binary lives — resolved ONCE, here,
// and bound to BOTH OS-fact injects below: a missing bake is a loud boot
// failure rather than a surprise mid-recovery, and there is only one place to
// change when the bake moves.
const osfactsBin = bakedOsFactsBin("KOLU_OSFACTS_BIN");
const endpoint = createEndpoint<TopClient, TopIdentity>({
hostId: "local",
home: HOME, // SAME home declaration as the daemon — disagreement impossible
// The Effect twin on a supervisor path, so the osfacts spawn never blocks
// the loop — and so a supervisor that gave up mid-read KILLS the child
// rather than leaving it to run out its deadline. The daemon half uses the
// SYNC reader instead (`common/processIdentity.ts`), because its gate claim
// must not reorder against the boot side effects it guards.
readProcessIdentity: (pid) => processIdentityAsync(osfactsBin, pid),
// The second OS-fact inject: who holds the rendezvous socket, for the
// recovery that runs when the gate no longer names the daemon.
readSocketHolders: osfactsSocketHolders(osfactsBin),
policy: {
capability: "not-drainable",
baked: {
contractVersion: "1.0",
build: { kind: "known", id: "fleet-top" },
},
onContractSkew: { kind: "recycle" },
onBuildMismatch: { kind: "nudge-human" },
},
probe: () => Effect.succeed(null),
driver: survivableSpawnDriver({
binPath: process.execPath, // node
args: ["--import", "tsx/esm", daemonEntry],
env: {
FLEET_TOP_GATE: GATE_PATH,
FLEET_TOP_SOCKET: SOCKET_PATH,
},
unitPrefix: "fleet-top",
// Launched from source (tsx), not a built binary — so the driver forces
// the detached branch even under a systemd session, and (as an actual from-source
// launch) must inherit our ambient env (node/PATH/HOME) to run from source. A built
// binary would omit `fromSource` and pass a complete env.
fromSource: { inheritParentEnv: true },
}),
// the framework hands you the path
connect: (socketPath) => connectTop(socketPath),
log,
onStatus: (hostId, status) =>
process.stderr.write(`[supervisor] ${hostId}: ${status.state}\n`),
});
// Boot: always-recycle → spawn → connect. Fails (after reporting `dead`) if
// it cannot bring the daemon up.
yield* converge(endpoint);
const conn = endpoint.current();
if (conn === undefined)
throw new Error("endpoint connected but current() is undefined");
const mem = yield* snapshot(
(conn.client.surface.memory?.get as StreamingProcedure<undefined, Memory>)(
undefined,
),
"memory",
);
process.stderr.write(
`[supervisor] connected — daemon reports ${conn.identity.cores} cores, ` +
`${(mem.used / 1e9).toFixed(1)} GB used\n`,
);
// The LIVE recycle: kill the daemon under us and stand a fresh one up, with
// the status held at one honest "restarting". Degenerate steps — nothing to
// preserve in this part.
yield* recycle(endpoint, {
capture: Effect.succeed(undefined),
drain: () => Effect.void,
reattach: () => Effect.void,
});
process.stderr.write("[supervisor] live recycle complete\n");
endpoint.current()?.dispose();
});
Effect.runPromise(main).catch((err) => {
process.stderr.write(`supervisor fatal: ${(err as Error).message}\n`);
process.exit(1);
});
The finale is the payoff: recycle runs the fixed
capture → drain → recycle → reattach sequence — a live recycle. The daemon
is killed and a fresh one stood up underneath, while the connection we hold
reconnects on the far side. This demo has nothing to preserve, so it passes the
degenerate steps; part 3 is where the same sequence
carries real per-host state.
-
With a daemon still running from before, start the supervisor.
pnpm run supervisor[supervisor] local: connecting [supervisor] local: connected [supervisor] connected — daemon reports 8 cores, 9.2 GB used [supervisor] local: connecting [supervisor] local: connected [supervisor] live recycle completeThe second
connecting/connectedpair is the live recycle itself:recyclere-runs the endpoint’s boot sequence, so the endpoint reconnects to the fresh daemon and re-fires its status. (stderrLoggeralso emits a structured JSON line per event to stderr; the[supervisor]-prefixed lines above are the ones to read.)
Watch the order: the supervisor recycled the daemon you had running (the always-recycle boot), connected to the fresh one, read a cell straight off it, then recycled it again live — and the client it held simply reconnected. No dropped session, no lost socket.
Next
fleet-top now survives on one machine. In Across the hosts you will reach a fleet-top running on another box over ssh — and watch the same surface arrive in your browser from a machine you never deployed to by hand.