# Roundhouse Warm-up Routing Configuration

## 1. What Roundhouse Generates

Roundhouse serves two unauthenticated endpoints for external systems to discover and warm up models:

- **`/api/routing-config`** (text/yaml)
  - Complete YAML document: header comment block + `model_list:` array
  - Header includes: `generated-by:`, `generated-at:`, `warm-hook:` (the cold-path entry point)
  - One entry per model: `model_name`, `litellm_params`, `model_info` (Roundhouse-standard shape)

- **`/api/routing-config.json`** (application/json)
  - JSON twin of the YAML: same entries, same null-omission policy, same metadata

Both are **live reads** — pull any time to get current fleet state. No authentication required. Generation incurs no file writes, no subprocess, no git operations.

### Inclusion Rules

A model appears in `model_list` when:

- **Hot always:** `rung ∈ {READY, BUSY}` — the unit is actively serving
- **Cold only if marked on-demand:** `on_demand == true` AND `rung ∈ {OFF, STARTING, LOADING}` — waiting for the warm hook
- **Never:** `rung ∈ {STANDBY, FAILED}` or `retired == true`

An unmarked unit that happens to be OFF does not appear, even if it is capable of serving. This enforces the contract: only units explicitly marked may be warm-started by external systems.

### Namespacing (`model_name`)

Each entry's `model_name` is `{host}-{logical_alias}`, e.g., `boltzmann-qwen3.6-coding`. The bare logical name is in `model_info.logical`. This prefix exists because units with the same alias can serve on different hosts (e.g., `qwen3.6-coding` on both boltzmann and ampere); without the namespace, the second host's fragment would overwrite the first's.

Example: a fragment from boltzmann carries `model_name: boltzmann-qwen3.6-coding`; one from ampere carries `model_name: ampere-qwen3.6-coding`. Both can coexist in an aggregated config.

## 2. Wiring LiteLLM (hossenfelder :4000)

LiteLLM on hossenfelder loads a static configuration file (never hot-reloaded). To use Roundhouse-discovered models:

1. **Fetch the fragment** (e.g., via systemd timer):
   ```bash
   curl -fsS http://boltzmann.fritz.box:8090/api/routing-config -o /etc/litellm/fragments/boltzmann.yaml
   ```

2. **Merge into the served config:**
   The fragment is a complete `model_list:` document. Merge by concatenating lists:
   ```python
   # pseudo-code
   merged = yaml.load(primary_config)
   fragment = yaml.load(boltzmann_fragment)
   merged['model_list'].extend(fragment['model_list'])
   serve(merged)
   ```

3. **Reload LiteLLM** (signal or API call, depends on LiteLLM setup).

### ⚠️ WARNING: Namespace Your Fragments

**Never point two hosts' fragments at the same bare `model_name` without a namespace prefix.** Example:

```yaml
# ❌ WRONG: boltzmann and ampere both send model_name: qwen3.6-coding
# The second fragment silently overwrites the first
- model_name: qwen3.6-coding
  litellm_params: { model: openai/qwen3.6-coding, api_base: http://boltzmann.fritz.box:8085/v1 }
- model_name: qwen3.6-coding
  litellm_params: { model: openai/qwen3.6-coding, api_base: http://ampere.fritz.box:8085/v1 }

# ✓ CORRECT: Roundhouse includes the host prefix automatically
- model_name: boltzmann-qwen3.6-coding
  model_info: { logical: qwen3.6-coding, host: boltzmann, ... }
- model_name: ampere-qwen3.6-coding
  model_info: { logical: qwen3.6-coding, host: ampere, ... }
```

The prefix is generated automatically; you do not add it. If you create a custom entry for another system, include its hostname.

## 3. Wiring the Bespoke llm-proxy

For the internal Roundhouse proxy (not LiteLLM):

1. **Fetch the JSON twin:**
   ```bash
   curl -fsS http://boltzmann.fritz.box:8090/api/routing-config.json -o /tmp/routes.json
   ```

2. **Map entries to endpoints:**
   ```python
   # For each model in model_list:
   for model in routes['model_list']:
       name = model['model_name']
       api_base = model['litellm_params']['api_base']  # e.g., http://boltzmann.fritz.box:8085/v1
       on_demand = model['model_info'].get('on_demand', False)
       rung = model['model_info'].get('rung')
       
       # Store mapping: name -> api_base
   ```

3. **Handle cold models:**
   If `on_demand == true` and `rung == 'OFF'`, the model is waiting for warm-up. Before forwarding a request:
   - POST `/api/warm` (see section 4 below) with the model's logical name
   - Wait for the model to reach READY
   - Then forward the request

## 4. The Warm Hook: Cold-path Recipe

The warm hook is documented in the routing-config header comment:

```
# warm-hook: POST http://boltzmann.fritz.box:8090/api/warm
```

### Request Body

```json
{
  "logical": "qwen3.6-coding"
}
```

or

```json
{
  "unit": "qwen3.6-coding.service"
}
```

Either `logical` or `unit`, not both. See section 4.5 of the MVP5 spec for the full error matrix.

### Response Codes

| Code | Status | Meaning |
|------|--------|---------|
| 200 | `already_warm` | Unit is already READY or becoming ready (rung ∈ STARTING/LOADING/READY) |
| 200 | `already_queued` | This unit is already pending warm (idempotent) |
| 202 | `{"rollout_id"}` | Warm-up started; monitor `/api/rollouts/{id}` for progress |
| 202 | `{"queued": true}` | Queued behind another operation; will fire when slot frees (≤ 3 s) |
| 404 | `unknown_unit` / `unknown_alias` | Unit/alias does not exist |
| 422 | `preflight_failed` | Unit exists but is `[RETIRED]` |
| 409 | `warm_queue_full` | Different warm already queued; retry after cancel |
| 422 | `not_on_demand` | Unit is not marked `# roundhouse: on-demand` — consent denied |
| 422 | `consent_unfittable` | Warming would require stopping unmarked (non-consenting) units |
| 422 | `ambiguous_alias` | Logical name matches multiple units (should not happen in practice) |

### Cold-path Algorithm

```python
# Pseudo-code for llm-proxy or external system

def call_model(model_name, request):
    # Check if cold
    model = routing_config[model_name]
    if model['model_info']['rung'] != 'OFF':
        # Already hot or warming; call directly
        return forward_to_api_base(model['litellm_params']['api_base'], request)
    
    if not model['model_info'].get('on_demand'):
        # Not marked; cannot warm
        return error_403('Model not available and not marked for warm-up')
    
    # Try to warm
    warm_resp = POST /api/warm with logical=model_name

    if warm_resp.status == 200:
        pass                        # already_warm — fall through and call it
    elif warm_resp.status == 202 and warm_resp.get('rollout_id'):
        # Warm switch started: poll the OPERATION, not GET /api/warm
        # (pending is null once a warm has STARTED; records carry phase, not rung)
        while True:
            op = GET /api/rollouts/<warm_resp['rollout_id']>
            if op['phase'] == 'done': break
            if op['phase'] in ('failed', 'restore_failed'): return error_503(op)
            sleep(1)
    elif warm_resp.status == 202 and warm_resp.get('queued'):
        # Parked (depth-1 queue). Poll GET /api/warm: when 'pending' clears,
        # check 'last' — disposition 'started' means it fired (then poll the
        # model endpoint until it answers); any other disposition (dropped,
        # cancelled, consent_unfittable) means it will NOT run: surface it.
        while True:
            state = GET /api/warm
            if state['pending'] is None:
                if state['last'] and state['last'].get('disposition') not in (None, 'started'):
                    return error_503(state['last'])
                break
            sleep(1)
    elif warm_resp.status in (409, 422):
        # Queue full / not marked / unfittable: DATA — surface it to the caller
        return error_service_unavailable(warm_resp.body)
    else:
        return error_500(warm_resp.body)
    
    # Now forward the original request
    return forward_to_api_base(model['litellm_params']['api_base'], request)
```

Do not spin on 409 `warm_queue_full`; surface it to the caller. Do not retry 422 errors; they are policy violations, not transient failures.

## 5. Token Provisioning

The warm hook requires bearer-token authentication. Roundhouse writes a single token file on the host:

- **Location:** `~/.config/roundhouse/token` (mode 0600; owned by the roundhouse user)
- **Content:** a single line, the bearer token
- **Usage:** include in the `Authorization: Bearer <token>` header

To provision the proxy:

1. Copy the token from the Roundhouse host to the proxy host's secret store:
   ```bash
   # On boltzmann (Roundhouse host)
   cat ~/.config/roundhouse/token  # e.g., "abc123xyz"
   
   # On hossenfelder (proxy host) — store in your system's secret mechanism
   # Examples:
   echo "abc123xyz" > /opt/llm-proxy/.token  # if llm-proxy reads from a file
   # Or populate environment var / k8s secret / vault / etc.
   ```

2. Include it in warm requests:
   ```bash
   curl -X POST \
     -H "Authorization: Bearer abc123xyz" \
     -H "Content-Type: application/json" \
     -d '{"logical": "qwen3.6-coding"}' \
     http://boltzmann.fritz.box:8090/api/warm
   ```

3. Optional: include `X-Roundhouse-Requester` header to label the warm source:
   ```bash
   -H "X-Roundhouse-Requester: llm-proxy"
   ```
   (Value is sanitized and logged; useful for debugging and audit trails.)

### Token Rotation

1. **Generate a new token** on the Roundhouse host.
2. **Replace** `~/.config/roundhouse/token` with the new value.
3. **Restart Roundhouse** (systemctl restart):
   ```bash
   systemctl --user restart roundhouse.service
   ```
4. **Update the proxy's secret** with the new token.
5. **Restart the proxy** or reload its config.

Old tokens become invalid after the restart; no gradual transition window.

## 6. Marking a Unit On-Demand

To make a unit eligible for warm-up, add a marker comment anywhere in its systemd unit file:

```ini
[Unit]
Description=Qwen 3.6 Coding (27B MoE)

[Service]
# roundhouse: on-demand
Type=simple
ExecStart=/usr/local/bin/llama-server --model /var/lib/models/qwen36-27b-a3b-coder.gguf --port 8085
```

**Important:** Roundhouse **parses units once at startup**. Adding or removing the marker requires:

```bash
systemctl --user restart roundhouse.service
```

The marker means both:
- **May be started autonomously** by warm requests (no human intervention needed)
- **May be stopped autonomously** to free memory for other models (symmetric consent)

The always-on trio (`llama-embed`, `qwen3.6-coding`, `llama-task`) should **remain unmarked** — they are never stopped, and warm requests are forbidden for them.

### Alternative Syntaxes

Both of these are recognized (same substring mechanism as roundhouse's manage/ignore directives):

```
# roundhouse: on-demand
; roundhouse: on-demand
```

The marker can appear on any line, any position (though convention places it near the [Service] section for readability).

## 7. Operational Notes

### Queue Semantics

The warm queue has **depth 1**:
- While a warm is pending, a duplicate request for the SAME unit answers `200 already_queued`; a request for a DIFFERENT unit answers `409 warm_queue_full` — depth is exactly 1, nothing else queues
- The parked warm fires within ≤ 3 seconds after the slot becomes free
- A restart clears the queue (in-memory only; no persistence)

### Slot Conflicts

If a human initiates a switch while a warm is queued:
- The human switch claims the slot immediately
- The queued warm remains parked and fires after the human switch completes

### Failed Warms — two different situations

- **Fire-time drop** (unfittable / cancelled / preflight fail at fire): the queue is
  cleared, NO slot is held, the disposition lands in `GET /api/warm`'s `last`.
- **A warm switch that STARTED and then failed**: this is a normal failed switch —
  it HOLDS the operation slot until a human restores or dismisses it in the UI
  (or via the operations API). `POST /api/warm/cancel` cancels only a PARKED
  warm; it can never free a held slot.
- No cancellation; queues are FIFO (human ops always get priority, but queue survives)

If a warm is queued and a human initiates a *different* warm (different target), use `POST /api/warm/cancel` first:

```bash
curl -X POST \
  -H "Authorization: Bearer $TOKEN" \
  http://boltzmann.fritz.box:8090/api/warm/cancel
```

Response: `200 {"cancelled": true, "unit": "qwen3.6-coding.service"}` or `404 {"error": "no_pending"}`.

### Failed Warm

If a warm-up fails (preflight or consent check after queuing):
- The slot remains held (the queue does not auto-retry)
- A human must `POST /api/warm/cancel` or `POST /api/rollouts/<failed_id>/dismiss` to clear it
- Then the proxy can retry

### Advertised Host

By default, Roundhouse reports its hostname as seen by `os.uname()[1]`. On a container or VM, this may not resolve on the LAN:

```bash
# Inside container (hostname might be random):
$ hostname
e3f9a2c1b5d2

# Not resolvable from hossenfelder; warm-hook won't work.
```

Pass `--advertise-host` to override:

```bash
# In roundhouse.service ExecStart (or container run command):
ExecStart=/usr/bin/python3 roundhouse.py --serve --actuate --port 8090 --advertise-host boltzmann.fritz.box
```

This affects the `warm-hook` header and `litellm_params.api_base` URLs only; `model_info.host` is the machine's own hostname regardless.

### Monitoring

- **Check routing-config:** `curl http://boltzmann.fritz.box:8090/api/routing-config`
- **Check pending warm:** `curl http://boltzmann.fritz.box:8090/api/warm`
- **Monitor warm execution:** Watch the UI stepper (`/api/events` SSE) or poll the rollout (`GET /api/rollouts/{id}`)

## 8. The Fleet Fragment: `/api/routing-config/fleet`

Two more unauthenticated, live-read endpoints exist alongside the ones above,
merging in every fleet peer's own routing fragment:

- **`/api/routing-config/fleet`** (text/yaml) — the local fragment plus each
  `up`-and-`fresh` fleet peer's fragment, concatenated.
- **`/api/routing-config/fleet.json`** (application/json) — the JSON twin.

The consumer-side pattern is the one Section 2 already describes: pull, merge
into your served config, reload. The only change is that a single URL now
covers the whole declared fleet instead of one host:

```bash
curl -fsS http://boltzmann.fritz.box:8090/api/routing-config/fleet.json \
  -o /etc/litellm/fragments/fleet.json
```

Pulling this one document from boltzmann is equivalent to pulling
`/api/routing-config` from boltzmann *and* from every fleet peer boltzmann has
declared and merging them yourself — Roundhouse has already done the merge.
`/api/routing-config` and `/api/routing-config.json` (Section 1) are
unaffected by any of this: they stay local-only, exactly as before.

### Header comments (YAML)

`/api/routing-config/fleet`'s header carries more than the local route's three
lines, in a frozen order:

```yaml
# generated-by: roundhouse@boltzmann (fleet)
# generated-at: 2026-08-14T12:00:00Z
# contributor: boltzmann (local)
# contributor: ampere (fetched 2026-08-14T11:59:30Z)
# excluded: cern (down, stale: connect)
# conflict: model_name 'boltzmann-qwen3.6-coding' also advertised by ampere — ampere's entry dropped, boltzmann's kept
model_list:
  - model_name: ...
```

One `# contributor:` line per host that actually contributed (local first,
then fleet peers by name, each carrying the timestamp it was fetched at); one
`# excluded:` line per declared fleet peer that did **not** contribute, in the
form `(<reachability>, <fed state>: <reason class>)`; one `# conflict:` line
per dropped `model_name` collision. There is no `# warm-hook:` line — see
below.

The `.json` twin carries the same information as fields instead of comments:
`contributors`, `excluded`, `conflicts`, and `model_list` — and, like the YAML
document, **no `warm_hook` key**. Both documents say what they are in the same
words: `generated_by` / `# generated-by:` reads `roundhouse@<host> (fleet)`,
where the local route reads `roundhouse@<host>`. A fragment saved to disk is
therefore self-describing — a consumer can tell a fleet document from a local
one without remembering which URL it came from.

### Merge semantics

Entries are concatenated **verbatim**: no re-namespacing, no `api_base`
rewriting, no recomputed fields. Each host authors its own entries — the exact
dicts Section 1's namespacing rule already describes — and the aggregator only
concatenates them: local entries first, then fleet peers sorted by name, each
peer's own entries sorted by `model_name`. Roundhouse does not touch another
host's `litellm_params` or `model_info`, whether that host is the one serving
the fragment or a peer folded into it.

A `model_name` collision is **first-wins**: the first occurrence in merge
order (local, then peers by declaration name) is kept, and every later
occurrence of the same `model_name` is dropped — never a silent overwrite.
The drop is surfaced in both documents: a `# conflict:` comment in the YAML,
and an entry in the JSON twin's `conflicts` list (`{"model_name": ...,
"kept_source": ..., "dropped_source": ...}`).

Because `model_name` is `{host}-{logical_alias}` (Section 1), two hosts
serving the same logical alias normally produce two distinct entries —
`boltzmann-qwen3.6-coding` and `ampere-qwen3.6-coding` do not collide. The
conflict path exists for when the effective `model_name` matches anyway —
most commonly two hosts (or two test instances) that report the same
underlying hostname, or a misconfigured `--advertise-host`. That is exactly
why entries carry the host prefix in the first place: the namespace is what
keeps a shared alias from colliding, and the conflict path is the safety net
for the case where it doesn't.

### An absent host's entries vanish — by design

A fleet peer contributes to `/api/routing-config/fleet` only while its
reachability is `up` **and** its federated state is `fresh`. The moment either
condition fails — the peer goes down, or a fetch starts failing — its entries
leave the fragment on the very next read. There is no grace period and no
cached fallback here: a router must never be handed a backend on a host that
is not there.

This is a deliberate contrast with the roster at `GET /api/fleet`, which does
the opposite on purpose: it keeps a stale peer's last-known units, marked
`stale: true`, so an operator can see what was last running. The roster is
for looking at; the routing fragment is for handing to something that will
try to connect to what it lists. See `docs/PEERS.md` for the fed-state
machine (`never`/`fresh`/`stale`) behind this distinction.

### Per-host warm hooks

The fleet fragment carries no `# warm-hook:` line and no `warm_hook` JSON
key — a single hook URL would be wrong for every entry except the ones the
local host authored. A consumer that needs to warm a peer's model gets that
peer's warm hook the same way it gets boltzmann's: by pulling that peer's own
`/api/routing-config` (Section 4) directly. The fleet fragment tells you what
exists and where; it does not collapse every host's warm-up contract into
one.

---

**Last Updated:** 2026-08-13 (MVP5 Specification)
