Roundhouse — llama unit renderer & watcher, design

[why-roundhouse]

## 0 · The name, and the five positions

A roundhouse turns locomotives onto the stalls that fit; the signal box beside it enforces mechanical interlocking — you physically cannot set two conflicting routes. This document is the interlocking half, and it now carries the Roundhouse name outright. That is exactly the missing piece — two units both claiming :8085, a retired unit one `systemctl enable` away from a collision, an edit one `-c` away from an OOM. Named 2026-08-12: folded into Roundhouse.

Five positions, defended below:

1. Systemd unit files stay the single source of truth. The tool holds no config of its own. Ever.
2. "Renderer" means render to the screen, not from a template. A generator that emits units from a database would silently make the database the truth and your comments disposable. Rejected.
3. Push for process config, Pull for routing metadata. That is the answer to your question 2 — it splits cleanly, see §6.
4. The watcher watches; it does not knock. On a serialized single-user box, health is observed passively. Active probes only at explicit moments (a switch, a button).
5. The measurement loop proposes; the operator disposes. Answer to your question 3, see §6.

[position]

## 1 · Scope, and where it sits

This IS: a local web UI on boltzmann that (a) renders every llama-serving systemd user unit with its semantics legible — model, quant, context, KV, pinning, port, gate, sampling, and the operator's notes; (b) watches liveness beyond systemd's notion of "active"; (c) performs guarded operations: start, stop, switch, and surgical parameter edits; (d) keeps history in git instead of `.bak` files.

This is NOT: a process supervisor (systemd is), a request router/proxy (llm-proxy is), a config store (the unit files are), a batching/throughput dashboard (irrelevant at one request at a time), or a model downloader.

### vs. llama-swap and friends

llama-swap's whole premise is "give me a YAML, I am the supervisor." Migrating means transcribing 23 units into YAML and losing the thing that makes them valuable — `mixperten.service` is ~80 lines of measured decision record wrapped around 9 lines of command. Cockpit has the opposite problem: it speaks systemd fluently and knows nothing about `-ctk q8_0`. Roundhouse occupies the empty cell: systemd-native, llama-literate. It neither wraps nor replaces llama-swap; it makes it unnecessary for this fleet's shape (deliberate, operator-driven model residency — not request-driven auto-swap). If you ever want request-driven swapping, adopt it then as a separate decision; don't let a dashboard smuggle in a supervisor.

### Push vs. Pull (your question 2)

Position

Split the question by what the config is for. Process config must be Push: a serving host has to boot and start its model with nothing but its own disk — pulling ExecStart lines from a Pi cluster at boot is an availability dependency you'd curse exactly once, at the worst time. Files on disk, drift visible in git. Roundhouse is the Push horn, and that's correct. Routing metadata should be Pull: a consumer choosing a model already needs the gateway reachable to make the request, so asking `/v1/models` at request time adds zero new dependency and kills the hand-copies (pi-agents' 53 stale entries) permanently. Push what a host needs to exist; pull what a client needs to choose.

One consequence worth stating: Roundhouse is not the generator that renders configs for consumers. It manages the serving side only. The consumer-side fix is your own "smallest step" — enrich `/v1/models` — and it stays in llm-proxy.

[views]

## 2 · Information architecture

Four views. Navigation is flat — this is a tool you glance at, not an app you live in.

View | Job | Wireframe
--- | --- | ---
Platform (home) | One glance: what's on the turntable, what's always-on, what's standing by, memory, port board. | below
Stall (unit detail) | One unit, fully legible: parsed semantics + live status + operator's notes rendered as prose + git history + journal tail. | below
Tweak (edit flow) | Change parameters with a byte-exact diff preview, preflight checks, stale-prose warning, commit. | §3
Switch (turn the table) | Stop A, start B, with interlock, fit check, and the loading ladder made visible. | §4

### Platform — the home screen

Organized by role, not alphabetically: the big slot (one at a time — the layout itself says so by having exactly one seat), the always-on small units, kernel-gated standbys, retired units folded away. All data below is real, from today's recon.

     ROUNDHOUSE · boltzmann          kernel 6.12.x (mainline) │ mem ████████░░░░░ 17.2 / 30.6 GiB
     ═══════════════════════════════════════════════════════════════════════════════════════════
     BIG SLOT — one model at a time
     ┌─────────────────────────────────────────────────────────────────────────────────────────┐
     │ ● READY  qwen3.6-coding                                     :8085   alias qwen3.6-coding │
     │          Qwen3.6-27B-A3B-Coder · Q4_K_M · 14.94 GiB file · ctx 65536 · KV q8/q8 · fa on  │
     │          pinned A76 (4-7) · peak RSS 17.8 GiB measured · last req 13:41 (7.7 t/s decode) │
     │                                                       [ switch… ] [ restart ] [ stop ■ ] │
     └─────────────────────────────────────────────────────────────────────────────────────────┘
     in the roundhouse (12): qwen3-30b · gemma-a4b · deepseek-coder · llama-server-cpu-q4km · …

     ALWAYS-ON
     ● READY  llama-embed   :8082   nomic-embed-text-v1.5 (mneme)          peak 1.1 GiB
     ● READY  llama-task    :8086   Llama-3.2-1B (OWUI title/tag gen)      peak 1.9 GiB

     STANDBY — kernel-gated, normal on this kernel
     ◌ waiting llama-server-qwen35-npu    wants 6.1.75-npu-port    :8086  ⚠ shared with llama-task
     ◌ waiting llama-server-gemma4-npu    wants 6.1.75-npu-port    :8087
     ◌ waiting llama-server-qwen3-4b-npu  wants 6.1.75-npu-port    :8091

     RETIRED (1)  mixperten [2026-08-12, claims :8085]                              [ show ]
     ═══════════════════════════════════════════════════════════════════════════════════════════
     PORT BOARD   8082 embed ✓ │ 8085 big slot ✓ (+1 retired claim) │ 8086 ⚠ 2 claims │ 8091 free

### Stall — one unit, fully legible

The left column answers "what runs, exactly" without reading a shell command. The right column is live. Below the fold: the operator's notes — the comment block rendered, with measurement tables detected (aligned columns → real table) and never, ever rewritten. The core-pinning strip is the kind of "graphic" that earns its pixels: it shows `taskset -c 4-7` as a picture of the actual silicon.

     ◂ platform    qwen3.6-coding.service              ● READY since 09:14 · uptime 4 h 27 m
     ─────────────────────────────────────────────────────────────────────────────────────────
     WHAT RUNS                                          LIVE
     model     qwen36-27b-a3b-coder-Q4_K_M.gguf         unit      active (running)  enabled
               14.94 GiB · file exists ✓                port      :8085 listening ✓
     quant     Q4_K_M                                   health    200 ok
     ctx       65 536   (model max 262 144)            busy      idle (journal-observed)
     KV        q8_0 / q8_0 · flash-attn on              mem now   17.8 GiB RSS
     threads   4, pinned A76                            mem peak  17.8 GiB (cgroup, this boot)
               [ A55 · · · · ][ A76 ▓ ▓ ▓ ▓ ]           journal   ▸ tail −f
     sampling  temp 1.0 · top-p .95 · top-k 20          requests  13:41  499→426  7.67 t/s
               min-p 0 · presence 0                               12:05  1399→88   7.9 t/s
     alias     qwen3.6-coding      gate  none
     restart   on-failure / 5 s                         [ tweak parameters ]  [ raw file ]
     ─────────────────────────────────────────────────────────────────────────────────────────
     OPERATOR'S NOTES — rendered from the unit file's comments; the file is never rewritten
     │ MODEL SWAPPED 2026-08-12: Qwen3.6-35B-A3B UD-Q3_K_XL → 27B-A3B-Coder Q4_K_M.
     │ Measured end-to-end on a real bullpen prompt (499 in / 426 out), median of 3:
     │   ┌──────────────────┬─────────┬─────────┬──────────────┬───────────┐
     │   │ 27B coder Q4_K_M │  83.1 s │  ±1.8 % │ 7.67 t/s dec │ 14.94 GiB │
     │   │ 35B Q4_K_M       │ 102.8 s │  ±24 %  │ 7.83 t/s dec │ 19.70 GiB │
     │   └──────────────────┴─────────┴─────────┴──────────────┴───────────┘
     │ … MTP stays OFF. Re-measured 2026-08-12 on the MoE: −25 % at n_max=3 …
     ─────────────────────────────────────────────────────────────────────────────────────────
     HISTORY (git)  2026-08-12 model swap 35B→27B · 2026-08-12 sampling · 2026-08-11 imported .bak

[round-trip]

## 3 · The round-trip problem

This is the part every existing tool gets wrong, and the part that decides whether you trust Roundhouse. The recon numbers make the constraint vivid: `mixperten.service` is 103 lines, of which 9 are the command. The rest is a lab notebook — KLD tables, a McNemar test, a decision record signed "ENTSCHEIDUNG markus 2026-08-11", and a warning about which comment is now decoration. Any mechanism that regenerates the file from a parsed model will eventually normalize, reorder, or drop something. So:

### Mechanism: parse everything, rewrite nothing, splice bytes

1. Concrete-syntax parse. Read the file as lines. Locate `ExecStart=` and its backslash continuations; tokenize the command with systemd's exact quoting rules (single/double quotes, `\` continuations including the no-space form `llama-server\` in the NPU unit, `%%` specifiers). Every token keeps its byte range in the original file. Same for plain `Key=Value` lines.
2. Edits are splices. Changing `-c 65536` to `-c 131072` replaces exactly those bytes. Adding a flag inserts into the line the operator picks in the preview. Removing a flag deletes its tokens and nothing else. Comments, blank lines, ordering, indentation: untouched by construction, because they are never parsed into a model that gets serialized back.
3. Readback verification — refuse rather than corrupt. After splicing (to a temp file): re-parse and assert the argv equals the intended argv; then `systemd-analyze verify` the unit; then after `daemon-reload`, compare `systemctl --user show -p ExecStart` — systemd's own parser — against intent. Any mismatch aborts the write and shows both parses. This is the guard against the tokenizer being subtly wrong about an edge case.
4. Atomic write with compare-and-swap. Hash the file at read time; refuse to write if it changed underneath (you hand-edit these files, and will continue to — that must stay safe).
5. Git replaces `.bak`. `git init` in `~/.config/systemd/user/`; the seven existing `.bak*` files become the initial history (dated commits reconstructed from their timestamps, then the files removed). Before any Roundhouse write, uncommitted hand edits are auto-committed as `manual edit (imported)` — hand edits are first-class, never clobbered, and drift is visible exactly as your Push philosophy wants.

### The stale-prose problem, honestly

The comments reference the values ("65536 kept from the previous unit to change one thing at a time"). A byte-perfect splice preserves the prose and thereby makes it a liar. No tool can fix that; a good tool makes it impossible to do unknowingly: on every edit, grep the comment block for the flag name and the old literal value; hits appear in the review step and must be either edited or explicitly acknowledged. Never auto-edit prose — a tool that rewrites your measurement notes is a tool you'd rightly uninstall the same day.

     TWEAK qwen3.6-coding.service                                        step 2 of 3 — review
     ─────────────────────────────────────────────────────────────────────────────────────────
       -c   65 536 → 131 072     KV q8_0 on this arch: +~2 GiB (estimate — no measured peak
                                 at this ctx) · projected peak 19.9 / 30.6 GiB ✓ fits
     ─────────────────────────────────────────────────────────────────────────────────────────
     DIFF — only these bytes change
       −   -c 65536 -t 4 -fa on -ctk q8_0 -ctv q8_0 \
       +   -c 131072 -t 4 -fa on -ctk q8_0 -ctv q8_0 \

     ⚠ PROSE MAY NOW BE STALE — 1 comment line mentions the old value:
       # …supports 262144 ctx; 65536 kept from the previous unit to change one thing at a time
       [ edit this comment ]   [ acknowledge — leave as is ]

     PREFLIGHT   ✓ readback parse = intent     ✓ systemd-analyze verify clean
                 ✓ port unchanged              ✓ projected 19.9 < 28.0 warn line
     commit: [ ctx 65536→131072 — headroom from the 1.1 GiB smaller model            ]
                                        [ back ]   [ apply + restart ]   [ apply, restart later ]

### Drop-ins

You have none today (verified). Policy: read effective config via `systemctl --user cat`; if a drop-in ever overrides a key, that key renders read-only with "override lives in `foo.service.d/x.conf`" and Roundhouse edits the fragment only. It never creates drop-ins: for user-owned units in `~/.config`, drop-ins would split one file of truth into two files of truth for zero benefit. That's a fight with `systemctl edit` deliberately not picked.

### Named failure modes

- Tokenizer wrong on an edge case (quoted JSON like `'{"enable_thinking":false}'`, `\`-no-space, `%%`). Caught by readback-vs-intent; outcome is a refusal, not a corruption. Residual risk: refusing too often. Acceptable.
- Stale prose acknowledged and forgotten. Mitigation is the ack being recorded in the commit message trailer. Residual risk real; it exists today with hand edits too.
- Concurrent hand edit. Caught by compare-and-swap; the UI re-renders and asks again.
- Git repo confusion if you ever put that directory under another repo/stow scheme. Detect a pre-existing repo and join it instead of nesting.
- Semantic edits beyond splicing (e.g. restructuring a command). Out of scope by design: the escape hatch is the raw-file editor with the same verify + commit pipeline and none of the parameter UI.

[watcher]

## 4 · Live status — what "online" means

"Active" is nearly meaningless here: a unit is `active (running)` for the full 72 s a 15 GiB model takes to load, and `inactive` can be the correct, healthy state of a kernel-gated unit. Roundhouse's status is a ladder, and the UI always shows the rung, not a boolean:

Rung | Signal | Source
--- | --- | ---
OFF | unit inactive, not gated | `systemctl --user show`
STANDBY | inactive because `ExecCondition` failed — rendered neutral, with the unmet condition in words: "waiting for kernel 6.1.75-npu-port (running: 6.12.x)" | `ExecMainStatus`/condition result + parsed condition
STARTING | systemd activating, process spawning | unit state
LOADING | port bound but `/health` → 503 "loading model" — the 72-second gap, shown with elapsed vs. last-known load time | `ss` + one cheap local HTTP GET
READY | `/health` → 200 | HTTP
BUSY | a request is in flight | journal tail (passive), not probing
FAILED | systemd failed / restart-looping (count shown) | unit state + `NRestarts`
RETIRED | `[RETIRED]` in Description | parsed

Principle — the watcher does not knock

No periodic completion probes, ever. The workload is serialized; a probe can queue behind a real request (your own notes flag a runaway generation costing 94 minutes at 5.8 t/s), and synthetic tokens would pollute the honest per-request log. `/health` is a cheap status read, fine on a few-second cadence; BUSY and per-request stats come from tailing the journal, which llama-server already writes. A real completion runs only when you press "smoke test" during a switch. Corollary: `tg128`-style numbers never appear anywhere in this UI — every displayed t/s is from a real request's wall clock, your own honesty rule.

### Memory fit

Memory is the failure mode that matters, and your own data says formulas lie: the 35B Q4_K_M "fits" on paper, then ARM repack residency pushed 30.5/31.6 GiB and prefill halved while decode looked fine. And the KV formula needs per-architecture knowledge anyway (the hybrid Qwen has KV on 11 of 41 blocks only). So:

- Measured beats modeled. Roundhouse records cgroup `memory.peak` per (unit, model-file hash, ctx) into a small sqlite. Predictions come from the last measured peak when one exists — labeled "measured" — and from file-size + KV estimate only as a fallback, always labeled "estimate".
- Warn line at 28 GiB projected (the measured collapse zone starts ~30.5 of 31.6); hard confirm above it. Co-residency is summed: big slot + always-on peaks + a fixed system reserve.
- Prefill-collapse tripwire: if a unit's journal-observed prompt-processing rate drops well below its own history while RSS is near the ceiling, badge it "memory-pressure suspected" — that exact signature cost you a day once; the UI should name it.

[interlocking]

## 5 · Safety rails

Found during this design's recon

Port :8086 has two claimants right now: `llama-task.service` (enabled, active) and `llama-server-qwen35-npu.service` (kernel-gated, disabled). Harmless today — but enable the NPU unit before the next NPU-kernel boot and both start, one loses, and it happens at boot with nobody watching. This is precisely the class of trap the mixperten `DO NOT ENABLE` comment guards :8085 against, one port over, without the comment. A dashboard that surfaces this on day one has paid for itself.

- Port interlock, gate-aware. Claims are computed from parsed `--port` across all units, then evaluated per scenario: active conflict (two running — should be impossible, flagged red), armed conflict (two enabled whose conditions can be simultaneously true — the :8086 case on an NPU kernel — flagged on the port board and at enable time), latent (disabled claimant — a footnote, like mixperten's :8085). Enabling or starting into an armed conflict requires typing past a named warning, never a silent failure at boot.
- Alias interlock, same machinery. Clients route by `--alias`; two live units answering the same alias on different ports is a subtler lie than a port clash.
- The retired trap. `[RETIRED …]` in Description is the convention (already your practice). Retired units render folded and grey, keep their full notes (they're the best-documented decisions you have), and their start/enable actions are demoted behind an explicit "this unit is marked RETIRED 2026-08-12 and claims :8085, owned by qwen3.6-coding — start anyway?" Never deleted, never hidden.
- STANDBY is not broken. Kernel-gated units get their own section, neutral color, and the condition spelled out in words. No red, no "failed", no restart storm — the whole point of the `ExecCondition` pattern, finally reflected in a UI.
- OOM prevention = the §4 fit check, enforced at three doors: starting a unit, switching the big slot, and any edit that raises `-c` / changes model / changes KV quant. Above the warn line the action needs an explicit override; the override is recorded in the commit/journal so the 3-a.m. question "why is the box swapping" has an answer.
- Co-fit notes stay human. The "stop the small models first — they do not co-fit" knowledge lives in comments today. v0.1 does not invent a metadata language for it; the summed-peaks check covers the arithmetic. If it earns its keep later, a single opt-in comment directive (`# roundhouse-slot: big`) — in the file, so truth stays in the file — is the ceiling of cleverness allowed here.

### Switch flow — turning the table

     SWITCH big slot   qwen3.6-coding  →  qwen3-30b
     ─────────────────────────────────────────────────────────────────────────────────────────
     INTERLOCK   :8085 free after stop ✓ · alias overlap none ✓
                 fit: measured peak 21.3 + always-on 3.0 + reserve 1.5 = 25.8 / 30.6 GiB ✓
     ─────────────────────────────────────────────────────────────────────────────────────────
      1 ▣ stop qwen3.6-coding        done · 2 s
      2 ▣ start qwen3-30b            active · 1 s
      3 ▶ model loading              /health 503 · 48 s elapsed (last load on this box: 72 s)
      4 ▷ ready
      5 ▷ smoke test (optional)      one short real completion, wall-clock timed, logged
     ─────────────────────────────────────────────────────────────────────────────────────────
     [ abort — restores qwen3.6-coding ]        enable-on-boot follows the switch: [x] yes

[roundhouse]

## 6 · Roundhouse integration — your three questions

Q1
  Who owns the truth? Two truths, two owners, one boundary. llm-proxy on hossenfelder owns catalog truth: what exists, what it costs, what it can do. Roundhouse owns process truth on the serving host: what is actually up, on which port, with which ctx, in which state — because that truth is literally the systemd state it renders. The boundary is a one-way street: Roundhouse publishes process truth (a read-only `/status` JSON: alias, port, rung, ctx, last-measured t/s), and llm-proxy may consume it to mark its 4 [local] entries live or dead instead of guessing. Roundhouse never pushes routing decisions and never reads the catalog to decide anything. Consumers keep talking to the gateway, not to Roundhouse.
Q2
  Push or Pull? Answered in §1: Push for process config (boot must not depend on the network), Pull for selection metadata (a request already implies the gateway is reachable). Roundhouse is the Push horn done right — files in git, drift visible, and the "generator" temptation explicitly rejected because your units are documents, not build artifacts.
Q3
  May the measurement loop influence the ladder? Close the loop through the operator, not around him. bullpen-evals verdicts flow into a recommendation inbox: "rung 2 solved 14/15 tasks that escalated to rung 3 last month — demote the default?" One click applies it; the applied change is a git commit (or an llm-proxy config change) with the evidence linked. What makes self-reconfiguration undebuggable is not the feedback — it's unattributed change. Your own unit files show the standard: every decision in them has a date, a measurement, and a signature. Hold the loop to the same standard and it stays debuggable; let it write silently and in six months no one can say why the ladder looks the way it does. Fully automatic re-rating is a later, separate decision to make after a few months of watching how often you just click "apply".

Hard rail, structural

No paid offloading, ever. Roundhouse manages local units only, so it structurally cannot offload. If it ever displays catalog context next to a recommendation, [$] entries are inert text — never an action, never a fallback target. Worth writing into the code as an assertion, not just into this document.

[plan]

## 7 · Staged plan — smallest useful thing first

Shape: one Python file (matching the `/opt/llm-proxy.py` house style) + one static page, SSE for live updates, sqlite for measured peaks, talking to systemd via the user D-Bus/`systemctl --user`. Runs as — of course — a systemd user unit, `roundhouse.service`, on a free port (8090 is unclaimed today; the port board will confirm its own). No node toolchain, no build step, nothing to babysit.

- v0.1Read-only, one week, already useful. Scan + parse all units, Platform + Stall views, status ladder, operator's-notes rendering, port board with the :8086 finding visible, kernel-gate STANDBY treatment, memory gauge. Zero write path = zero risk while trust builds. This alone beats every existing tool for this fleet, because nothing else can render these units.
- v0.2Operations. Start/stop/restart/enable with port+alias interlocks; the Switch flow with the loading ladder; git init + `.bak` absorption; journal tail in the Stall view.
- v0.3The round-trip. Parameter tweak with splice, diff preview, readback verification, stale-prose flagging, commit messages. The hard part, shipped only after v0.1/v0.2 have proven the parser against all 23 real files.
- v0.4Memory intelligence. Peak-RSS sqlite, measured-vs-estimate fit predictions, prefill-collapse tripwire, per-request t/s history from the journal.
- v0.5Roundhouse hooks. `/status` JSON for llm-proxy; bullpen-evals recommendation inbox. Gated on the `/v1/models` enrichment landing first — that remains the true smallest Roundhouse step, and it happens in llm-proxy, not here.

Fleet scope: v0.1–v0.4 are boltzmann-only on purpose. The design generalizes (a host column, N unit dirs), but multi-host before the round-trip is trustworthy would multiply risk, not value.

[open]

## 8 · Open questions for you

1. Auth: LAN-only bind like the llama-servers, or tailnet/basic-auth? It can stop and start your models, so "whatever :8085 does" may not be enough.
2. Git location: repo directly in `~/.config/systemd/user/`, or a mirror repo with the unit dir symlinked into it? Direct is simpler; say if that directory has other plans.
3. Naming: Roundhouse, or something else in the railway family? The metaphor is load-bearing enough in the UI copy ("turn the table", "standing by") that the name should be settled before implementation.