osfacts — scoped, honest OS process & socket facts

osfacts is a product of its own — its own README, its own CI, and since OSF5 its own repo — not a kolu module. One static Rust binary that answers what does the OS say about these processes and their sockets, scoped to exactly what you asked, honest about what it could not read. On a Linux host with about 450 processes, its one-process process+port shape takes 6.5 ms and its host-wide all-process-facets shape takes 24.3 ms. The flags are part of the number. The binary and kolu’s port-sensor adoption shipped in #1996; drishti’s adoption (srid/drishti#122) is what wrote the rest of the phase tree below. Everything in that tree has shipped, including the closing phase — the graduation to its own repo (OSF5’s second half), described under what shipped last; the shipped work is compacted to its durable lessons.The candidate measurements come from the #1982 sweep. The current osfacts numbers come from 31 interleaved warm runs for #2011 on naiveintent, with 450–466 live processes and stdout captured as a real client captures it.

The creed, one line per word — also the repo README’s opening:

• Scoped — cost tracks the ask (--roots walks a subtree), never the host.
• Attributed — every snapshot carries the full pid→ppid table, so a grandchild listener walks back to its root.
• Honest — what it could not read is reported, never silently dropped.
• Versioned — schema version first; a mismatched consumer fails loudly instead of parsing a half-understood shape into zero rows.
• Fast — cost tracks the ask: 6.5 ms for the one-process port shape, 24.3 ms for every process facet across about 450 Linux processes. Interactive callers poll at seconds scale, and the live lane guards the large shape with a process-count-scaled child-CPU budget that ignores scheduler contention.

The contract

One verb, composable facets, first-class scoping — every user story is the same verb with different facets, never a consumer-shaped flag or mode:

who	wants to know	runs	comes back
kolu’s port sensor, every ~5 s	which ports this terminal’s subtree listens on	osfacts snapshot --roots 4242 --procs --ports	pid table + listener rows with raw addresses; unreadable pids listed, so a blind terminal renders blind, not empty
drishti’s inspector	every process, with identity, launch details, memory, age, and CPU%	snapshotHost(bin, { procs, uid, cwd, status, argv, mem, startTime, cpuTime })	the real host process set — not pid 1’s incomplete subtree — with raw uid, cwd, state/nice/threads, full argv, RSS, start time, and cumulative CPU microseconds
kolu’s daemon supervisor	is this pid still mine, and since when	osfacts snapshot --pids 991 --start-time	a start time to compare against the recorded one
padi’s socket takeover	which pids hold this unix socket	osfacts socket-holders /run/user/1000/padi.sock	the holder pids, or none
drishti’s host dashboard	how the machine itself is doing	osfacts host --cpu --mem --load --net --disk	cumulative counters + gauges, one versioned schema
a human at any shell	all of it, readable	osfacts snapshot --json | jq	the same facts as JSON

• Scoping is the headline. --roots walks the given pids’ subtrees, --pids takes an exact set, no flag means host-wide. Cost then tracks the ask, not the host — the property no surveyed tool has and the one the measurements say matters most.
• Facets so a consumer pays only for what it reads: --procs, --ports, --mem, --start-time, --cpu-time, --uid, --cwd, --status, and --argv. socket-holders and host are separate verbs because they answer different questions (a path, not a pid set; the machine, not a process) — and each owns its own E facet vocabulary, so a consumer cannot match one verb’s blindness against another’s by accident.
• One versioned schema. TSV on the hot path (V<TAB>2 first, tagged rows), --json for humans and one-offs.
• Raw address bytes, never a wildcard boolean. Consumers keep exactly one classifier. Two predicates that must agree about ::ffff:0.0.0.0 is precisely how they come to disagree.
• Cumulative counters, never rates. CPU% and bytes/s are consumer-side diffs between two snapshots; a one-shot sampler never sleeps.This is why procs costs ~30 ms even at its lowest setting: it sleeps to compute a rate nobody asked it for.
• A MANDATORY unreadable section. Every pid that could not be inspected is reported, with its errno — the policy of what is fatal stays in the consumer, the only layer that knows which pids it asked about.The sudo lesson made structural: when an unreadable pid was silently dropped, one sudo password prompt emptied the Ports section for every terminal on the host; when it threw indiscriminately, one foreign-uid descendant did the same.
• One home for the facet vocabulary. A facet name is a Facet enum variant in src/schema.rs, never a string literal at an emit site, and facets.json carries the same list to the TypeScript client — pinned from both sides by tests in the fast unit lane. The facet set is this contract’s live axis of change (2 → 18 across V2), so adding one must be a single edit that fails loudly if only half of it lands, not a typo that reaches a consumer as a parse error at runtime.
• A blind source names every facet it costs, at the right scope. One source can feed several facets — darwin’s kern.proc.all supplies process identity, start time, uid, and state — so its silence emits one E row per facet the ask actually loses. A host-global constant is the mirror image: CLK_TCK, the linux page size, and the darwin mach timebase each cost their facet ONCE, as one E row, never as one U row per process.

process facet	tagged TSV row	JSON field	unreadable facet	semantics
--procs	P pid ppid name	procs	proc	short display name and parent identity
--mem	M pid rss_bytes	memory	mem	resident bytes at snapshot time
--start-time	S pid start_unix_us	startTimes	start_time	process identity instant in Unix microseconds
--cpu-time	C pid cpu_time_us	cpuTimes	cpu_time	cumulative user + system time; CPU% is a consumer-side diff
--uid	UID pid uid	uids	uid	real uid; username resolution belongs to the consumer
--cwd	CWD pid cwd_json	cwds	cwd	current directory; JSON in the final TSV field preserves delimiters
--status	STAT pid state nice threads	statuses	status	one-character state, signed nice, nullable thread count (- in TSV)
--argv	ARGV pid argv_json	argv	argv	the full vector, distinct from the short name; JSON preserves tabs, newlines, and NULs

Darwin’s process table is public even when its task accounting is not. kern.proc gives an ordinary caller pid, ppid, real uid, state, nice, start time, and a short command name for foreign processes; proc_pidpath adds the executable path, whose basename keeps long names from being cut to p_comm’s 16 bytes. RSS and cumulative CPU time still come from task APIs that return EPERM across uid boundaries./bin/ps looks unprivileged from the shell, but the binary is setuid root and Apple signs it with the private com.apple.system-task-ports.read entitlement. Its source calls task_read_for_pid for the protected half (reader, entitlement). So foreign U mem and U cpu_time rows are real blindness for an ordinary osfacts binary, while U proc, U uid, and U start_time rows would be false blindness.

Darwin’s listener rows carry no owning uid: neither net.inet.tcp.pcblist_n’s xinpcb_n records nor the libproc fd walk exposes one, where linux’s /proc/net/tcp does. So a --ports snapshot on darwin always emits E darwin_listeners ports_uid ENOTSUP and leaves every L row’s uid column -. Reporting it costs one row and buys the property the whole contract rests on: a consumer never has to know which OS it is running on to read a field.

The two darwin listener sources are always merged as a union, never switched between. A listener the fd walk positively observed is a fact the tool is holding, so a host table that omits it — the shape a partially gated pcblist_n would take — cannot delete it. Emptiness of the host table is evaluated once, by the caller, and is the sole trigger for the ports_unclaimed row. A record the walker cannot frame is a different failure: Apple layout drift, reported as E darwin_tcp_pcblist ports EINVAL rather than returned as a silently short listener set that looks healthy.

BLIND_OR_EMPTY is the single code for “this source cannot tell gated from genuinely empty”, on both platforms — the gated pcblist_n table, and an interface list that came back with nothing when lo/lo0 always exists. One condition, one code: a consumer that branches on the code must not have to know which kernel produced the row.

socket-holders answers a path, and its three answers are the reason it is a verb of its own rather than a facet. Nobody holds it is an empty document; something holds it that no readable pid claims is an H unclaimed - row; the search could not complete is an E … socket_holders … row. The reader it replaced spelled all three as an empty list, which is how a supervisor comes to spawn a second daemon onto a live rendezvous socket. Linux can genuinely prove the first — /proc/net/unix is world-readable and lists every bound unix socket, so absence from it is evidence. Darwin has no such table at all, so a descriptor walk that named nobody reports BLIND_OR_EMPTY rather than borrowing linux’s proof; that asymmetry is the platform’s, and reporting it is what keeps the contract’s meaning identical on both.The pid set a socket-holder search covers is the whole host — the ask names a path, not a pid set — so a descriptor directory this binary may not read is deliberately not one U row per process. On linux the blindness is reported where it is decision-relevant instead (the unclaimed row, exactly as OSF6 reports an unattributed listener); on darwin it is one source-level row. Hundreds of identical U rows would drown the answer on a boot path and say nothing a consumer could act on.

One more thing the verb made explicit, because adding a verb is when it bites: the version line is written on the usage path too, so a document’s shape cannot separate an answer from a refusal — only the exit status can. 2 means the binary refused the ask (an unknown verb, an unknown flag), 1 means it answered with nothing but blindness, 0 means it answered. A client that parsed an exit-2 document read “this binary has no such verb” as “nothing found” — which is precisely what an older binary produces, every time.

Host HCPU rows add a nonempty model name and nullable MHz to the cumulative per-core counters. The frequency is honestly absent on Apple Silicon rather than represented by zero. HDISK carries both freeBytes (bfree) and availableBytes (bavail) so the consumer chooses its used-space definition without inheriting a hidden semantic change.

Why not an existing tool

Every candidate was measured, not surveyed, and each fails on the contract above rather than on packaging — a packaging gap is an errand, not an architecture verdict.Earlier drafts listed “not packaged for darwin” against osquery and “not in nixpkgs” against portls. Both struck: anything merely absent from nixpkgs could be packaged in an afternoon, so packaging never counts as a disqualifier. The repo README carries this same table:

candidate	disqualifier
osquery	the wrong operational model: a resident fleet-telemetry agent with a SQL surface, measured at ~378 ms per query and ~158 MB — built for querying thousands of machines every few minutes, not an interactive poll measured in tens of milliseconds
procs	discards the bind address — loopback and wildcard collapse to one row
portls	no PPID, no process table — a listener cannot be walked back to its root
rustnet	interactive capture TUI, needs packet-capture privilege
portview	listener rows only, no process table — a shell → npm → node server cannot be walked back to its terminal. No subtree scoping either: its single-port query is a host-wide scan plus a filterportview is also the strongest evidence for building this: it measures 9.9 ms on darwin against our former C helper’s 10.6 ms on the same box by the same method, in Rust, in a single pass. Single-pass Rust matches C — the language was never the constraint, the double enumeration was. That measurement is what settled osfacts on Rust.
sysinfo+listeners	wrapping the two crates enumerates the process list twice: 23 ms darwin / ~100 ms linux, while osfacts’ current Linux host-wide process+listener pass is 17.8 ms — and listeners shipped the v4-mapped decode bug we fixed upstreamThe ~50-line darwin address decode from that fix is vendored into osfacts with its regression fixtures; upstream is pinned in CI as a drift oracle. The repo README’s Testing section owns the detail.Disqualified as the process/socket engine — not banned outright. OSF7’s darwin network counters use sysinfo’s network module (target-gated to macos, default-features = false, network feature only): one function, taken to delete an unsafe hand-computed if_data byte overlay after CodeQL flagged it — the same remove-a-defect-class clause that admitted serde. The measured disqualifier (a host-wide process double-walk) doesn’t apply to a single sysctl read, and the live-oracle lane’s monotonic-counter check is the drift detector for the dependency. Its one honesty cost is contained rather than accepted: sysinfo drops a failed NET_RT_IFLIST2 sysctl and returns an empty set, so darwin::host reads that emptiness as blindness and emits E sysinfo_networks net BLIND_OR_EMPTY — lo0 always exists, so an empty interface table is a gated read, never a host without interfaces. Keeping the crate therefore costs no E row, which is what disqualifies a dependency here.
lsof / netstat	lsof measured 93 ms; macOS netstat is intermittently blind — success and zero rows in one sampling window, 29 rows in the next, same boot

Adoption — kolu and drishti

Inside kolu the OS used to be inspected in five places, each with its own hand-rolled reader, its own error policy, and its own platform split — five answers to one question. All five now call the one binary, and a full-source sweep confirmed the list is exhaustive.The sweep covered every package including @kolu/surface. The one near-miss: surface-remote’s groupAlive probe (process.kill(-pid, 0) — process-group liveness inside a SIGTERM→SIGKILL escalation timer). Deliberately not a consumer: group liveness is not in the fact set, and forking a snapshot binary inside an escalation timer is the wrong operational model.

consumer	asks	path
padi port scan	a subtree’s processes + listening ports	shipped — osfacts-client (OSF2, #1996); V2 wire compatibility for the same path in #2011
padi memorySampler	per-pid RSS	shipped — one baked osfacts --mem snapshot for padi + connected kaval; kaval RPC deleted (#2067)
daemon supervisor	“is this pid still ours”, and since when	shipped — pid-first tolerant reader, ±2 s start-time identity, and injected osfacts-backed readers (#2058; archived upgrade-window plan)
socketHolder (in surface-daemon-supervisor)	which pids hold a unix socket path	shipped — the socket-holders verb; the /proc/net/unix parse, the fd readlinks, and the darwin lsof shell-out all deleted (#2071)
drishti	process inspection + host telemetry	shipped — osfacts V2 is its host-observation boundary (#122); native host-telemetry readers retired (#123)

Drishti’s adoption is what wrote the phase tree. Its first attempt hit two walls that named two phases: deployed as an ordinary SSH user, 41 system processes came back readable as P rows but U … EACCES for their fds, so each rendered listener-blind — honest, useless for a whole-host inspector (OSF6); and its native host-telemetry readers (/proc gauges, vm_stat, sysctl, netstat -ib, Node os, statfs) had no contract to move onto at all (OSF7). A dropped-info audit found the quieter third wall: pid 1 is not a portable synonym for the host process set, and the first contract omitted uid, cwd, state/nice/threads, argv, CPU model/frequency, and bfree (OSF3). All three closed tool-side in V2 (#2011), and drishti’s migration then landed whole on a fail-fast baked path — DRISHTI_OSFACTS_BIN or a loud boot refusal, no PATH fallback.One native reader survives on darwin, by design rather than by omission: Apple gates task accounting across uid boundaries, so U mem / U cpu_time rows are real blindness for an unprivileged binary. srid/drishti#127 adds a scoped recovery layer over Apple’s setuid ps that fills only the facets osfacts marked unreadable — readable osfacts values always win, an absent usage row stays honestly blind, and input rows are never mutated. The contract predicted exactly this blindness; the recovery layer is a consumer policy on top of it, not a second source of truth.

Kolu’s own adoption closed in three PRs. #2058 took daemon identity: a pid-first tolerant gate reader keeps legacy one-field files actionable while a ±2 s start-time match protects new two-field identities, with osfacts-backed reads injected at the daemon roots — the archived upgrade-window plan owns that mixed-version design. #2071 took the last hand-rolled reader — the daemon supervisor’s socket-holder lookup — and with it kolu’s last lsof dependency; its durable detail is the three-way answer above, injected on EndpointSpec beside readProcessIdentity so the shared spine never learns which env var a consumer bakes the binary into. #2067 took memory: one osfacts --mem snapshot replaces padi’s self-read and its kaval RPC hop. Its load-bearing detail is generation identity — the sampler captures the endpoint’s immutable {pid, startedAt} target and re-verifies that same endpoint-owned identity after the async read, so a recycle mid-flight publishes nothing rather than attributing the old daemon’s RSS to the new one; a raced-away kaval is honestly absent, an unreadable fact is a typed error. Deleting the procedure advances kaval’s wire to 6.0 while leaving system.version byte-identical, and the pid stays in padi-internal connection metadata, never projected onto DaemonStatus. The paired UI survey found nothing unpainted: kolu’s asks are deliberately narrow (procs + ports for the port sensor, startTime for daemon ownership, mem for the sampler), every RSS fact on the client wire already appears in host chips, daemon dialogs, and diagnostics, and the remaining V2 facets are not held by kolu’s UI stack at all — they are tool-side vocabulary for consumers like drishti.

Linux cost follows the ask again

Drishti measured the first full Linux migration at 55.8 ms per poll, against 5.6 ms for its old in-process reader. The second process was not the problem. The host call cost 2.6 ms. The process snapshot was reopening stat four times per pid, walking every fd in one long line, growing zero-length proc files through repeated reads, and issuing almost one stdout write per row. Plain waste.

warm Linux shape	before	after
host-wide, every process facet	52.56 ms	24.33 ms
drishti’s snapshot + host calls, serial	55.41 ms	26.48 ms
one-process --roots, process+ports	7.58 ms	6.49 ms
83-process subtree, process+ports	19.85 ms	17.27 ms

The fix opens shared proc files once, preallocates one page for virtual-file reads, reuses the cached stat RSS in combined snapshots, buffers stdout, and parallelizes only large fd walks — each cycle timed against the binary immediately before it in one interleaved run, every kept cycle clearing a 3% noise bar, the largest single win being shared stat/cmdline reads at −23.3%. Ordered collection keeps attribution stable; small scopes never spawn a worker, and the narrow --mem shape still reads statm unless another requested facet has already paid for stat.

The durable artifact is the guard, not the numbers. A live-lane smoke takes 11 interleaved warm --procs and all-facet samples, then budgets the difference at 75 µs of child CPU per live process plus 20 µs per readable descriptor — two terms, because the cost has two drivers: --ports scales with descriptor count while the other seven facets scale with process count. Measured on an idle 407-process, 2725-descriptor host at ~15 µs/process and 6.0 µs/descriptor, so each term carries about 3x headroom, and child CPU against a same-load baseline keeps it a generous smoke rather than a wall-clock benchmark.The first version budgeted per-process only. On a workstation the two counts track each other and it passed; on a CI container — 50 processes, a build daemon’s worth of descriptors — the fd walk alone spent 20 ms against a 3.75 ms allowance, and the smoke failed on every CI host it ever ran on. Widening the per-process constant enough to cover that would have made the budget larger than the regression it exists to catch, so the denominator changed instead. Fixed in #2011.

The phase tree — track prefix OSF

phase	delivers	needs
OSF1 — the binary	snapshot --roots --procs --ports on both platforms · versioned TSV + --json · the mandatory unreadable section · the scar-tissue suite — shipped in #1996	—
OSF2 — kolu’s port sensor	padi on osfacts-client, with the C helper, TS /proc, and @kolu/port-scan deleted — shipped in #1996; V2 wire compatibility for the same path — shipped in #2011	OSF1
OSF3 — process detail facets	tool-side: the seven detail facets, host-wide snapshotHost, and the Linux perf work above — shipped in #2011	OSF1
OSF4 — socket-holders	the second verb, its three-way answer, and the deletion of socketHolder’s /proc/net/unix parse, fd readlinks, and darwin lsof shell-out — shipped in #2071	OSF1
OSF6 — unattributed listeners	tool-side: a listener no readable pid claims is emitted with explicit unattributed status (and its owning uid on linux) instead of dropped — the fix for #122’s privilege regression — shipped in #2011Linux’s world-readable /proc/net/tcp{,6} table makes this honesty, not new privilege: under a narrow --roots, an unclaimed socket may simply belong to an out-of-scope process, so the row says unclaimed, never unowned. Darwin’s fd-walk-free source is net.inet.tcp.pcblist_n. On macOS 27, zest gave Apple’s platform-signed sysctl a 54,872-byte table whose paired netstat read contained 29 listeners, but gave the ad-hoc osfacts binary only 48 bytes of empty framing; netstat also carries the private com.apple.private.network.statistics entitlement. Both captures are fixtures: the full one decodes to 29 and proves the record walker is current, while the gated one pins BLIND_OR_EMPTY. Same-uid fd walks still emit claimed listeners; only the independent host-wide unattributed table is gated. Anything stronger would require Apple identity or privilege, deliberately outside OSF6.	OSF1
OSF7 — host facets	tool-side: the host verb — load, memory + swap, uptime, per-core cumulative CPU with model and nullable MHz, per-interface cumulative counters, root-fs total/free/available bytes — shipped in #2011. drishti deletes its native telemetry readers on adoption (srid/drishti#123).Semantics to preserve, per the issue: cache-aware available memory, swap used/total, cumulative per-interface byte counters, per-core CPU deltas, honest CPU metadata absence, and distinct bfree/bavail disk meanings. Cumulative per the creed — rates are consumer-side diffs between snapshots; the tool never sleeps.	OSF1
OSF8 — kolu V2 facet adoption	identity gates + the mixed-version gate law — shipped in #2058; one baked --mem snapshot for padi + connected kaval, kaval’s RPC procedure deleted, and the UI fact-consumer survey — shipped in #2067. Design detail in Adoption above; the parked osfacts-kolu-adoption branch was sampler archaeology, not design authority.	OSF3
OSF5 — drishti & extraction	drishti’s migration relands whole on a fail-fast baked path — shipped (#122, #123, #127). The extraction is shipped too (detail)The directory moved wholesale: osfacts/ already carried its own README, default.nix, a zero-input subflake, and client-ts/ (osfacts-client — no @kolu scope, Node builtins only) precisely so extraction would be a move, not a redesign. The subflake became the flake and the contract did not change.	OSF3 · OSF6 · OSF7

Every phase has shipped, OSF5’s extraction half last, now that both external adopters are on the contract and the verb set is complete. Each phase is independently revertible.

What shipped last

OSF5 — extraction. Drishti’s half was done first: osfacts V2 is its host-observation boundary, its native host-telemetry readers are retired, and its baked path fails loudly rather than falling back to PATH. Then the move itself — osfacts/ out of the kolu tree and into juspay/osfacts (#1). It was built for this from the start, so the work was repo plumbing, not redesign: the only functional line that changed anywhere was which directory the flake imports its each-system.nix from, and both repos build the identical store path. The pin travelled with the directory, so the extracted flake still has zero inputs.

The whole directory went, client included. The TypeScript client was the one part with no obvious exit — a pnpm workspace member has to be a path inside the repo, and five kolu packages import osfacts-client by name. Keeping a copy in kolu was never worth defending: two directories with the same source is a duplicate whatever gate you put around it.

Two package-manager answers were tried and rejected on their own terms. A registry or release-tarball dependency needs a publish step to own. And pnpm’s git-subdirectory dependency (github:juspay/osfacts#path:/client-ts) looks right — it resolves correctly on an ordinary pnpm install — but silently drops its #path: fragment under pnpm fetch + --offline, which is exactly how kolu’s hermetic Nix build installs. What lands in node_modules is the whole osfacts repo with no package.json at its root, so resolution fails.

The answer was already in the repo: npins pins osfacts, so Nix has the source already. nix/workspace.nix grafts client-ts/ from that pin into the build tree as the osfacts-client workspace member, and just install performs the same graft for a working tree. pnpm then sees a plain workspace:* link with nothing to download — offline by construction, no lockfile integrity entry, no registry. The directory is generated and .gitignored, so there is nothing to drift. Kolu excludes it from its own recursive typecheck/test:unit runs: those are osfacts’ tests, and they run in osfacts’ CI.

That makes the client a stable external in padi’s and kaval’s build-identity closures rather than a hashed root — the same treatment @kolu/surface and every pinned dep already get. Kolu’s port sensor, the code whose behaviour padi’s staleKey is actually about, still lives in padi’s hashed src.

Both of kolu’s osfacts CI leaves left with the tool — the live-oracle lane because its script did, and the hermetic one because it had nothing left to assert that ci::nix does not already build — so all four ci::osfacts…@… contexts must be dropped from branch protection.

Out of scope, deliberately: foreground process (a PTY-fd fact — tcgetpgrp on the pty, so it stays kaval’s), anything requiring root — OSF6 is the best honest answer without privilege, not an escalation — and any daemon/resident mode. This is a one-shot sampler; if startup cost ever becomes the problem, the answer is to revisit the operational model, not to hide it behind a resident process. The same restraint applies outward: no TUI, no kill, no docker verbs — the broad user-facing surface is what made portls and portview wrong for embedding, and osfacts stays embeddable by staying narrow.