app-console-slm — inference infrastructure monitoring console

@@ -7,31 +7,25 @@ type: app
content_type: topic
quality: complete
index_group: input-and-developer-surfaces
short_description: "Terminal console cartridge showing live AI inference infrastructure state — model health, GPU nodes, queue depth, and daily spend — with per-tier kill switches."
short_description: "Terminal console cartridge showing live AI inference infrastructure state — model health, the burst-GPU fleet, queue depth, and daily spend — read-only, with no controls of its own."
status: active
bcsc_class: public-disclosure-safe
last_edited: 2026-06-09
last_edited: 2026-08-22
editor: pointsav-engineering
cites: []
references: []
paired_with: app-console-slm.es.md
---

# app-console-slm — inference infrastructure monitoring console

**Correction (2026-08-02):** the console does not provide keyboard controls for the kill switch or routing policy — those claims below (the K-key dialog, the P-key policy control, per-node toggle) don't match the real cartridge. `app-console-slm/src/cartridge.rs`'s actual `handle_event` only wires `r`/`R` (refresh) and `?` (help) — its own help text confirms this is the complete keybinding set. The kill-switch *concept* is real, but it lives server-side as an operator billing/cost-cap gate (`service-slm/crates/slm-doorman/src/flow_gate.rs`, `express_lane.rs`, `cost_ledger.rs` — "nothing bypasses the kill switch") with no TUI control surface exposing it; the four-policy routing scheme (balanced/drain-batch/drain-express/local-only) is likewise real and changeable via `POST /v1/flow/policy`, but not from this console. The panel names, poll interval, and read-only status display are accurate — only the write/control-surface claims (K/P/G keys) are fabricated. **Flagged, not resolved.**

app-console-slm is a terminal user interface (TUI) cartridge for the operator console
that displays the live state of the AI inference infrastructure. It shows the health
of the local inference model, the status of [[yoyo-compute-substrate|remote GPU nodes]], the depth of the
priority queue, the organizational graph entity count, and the current day's spending.
It provides keyboard controls for adjusting the routing policy and toggling per-tier
[[spot-vm-lifecycle-kill-switch|kill switches]].
that displays the live state of the AI inference infrastructure: local model health,
the burst-GPU fleet, the entity graph, queue depth, and the current day's spending.
It is a read-only dashboard. Every write operation it displays — routing policy,
[[spot-vm-lifecycle-kill-switch|the kill switch]] — happens through a separate API surface, not through this console.

The console runs in a terminal window on the same node as the [[service-slm|inference gateway]]. It
requires no browser, no network connection to an external service, and no
authentication beyond local shell access. It is the operator's primary dashboard for
understanding and controlling the inference layer.
authentication beyond local shell access.

## Display panels

@@ -40,88 +34,65 @@ ten seconds. The operator can trigger an immediate refresh at any time with the 

### Gateway panel

The gateway panel shows the current state of the inference router: whether it is
running, the active routing policy (balanced, drain-batch, drain-express, or
local-only), and the availability of each tier. A green indicator marks a tier as
available. A yellow indicator marks a tier as degraded — available but with recent
failures. A grey indicator marks a tier as offline.

The gateway panel also shows the active routing policy and, if a tier's kill switch
is closed, an explicit "kill: CLOSED" label.

### GPU node fleet panel
Shows whether the router is running, the local model's reachability and
tokens-per-second throughput, the burst tier's circuit-breaker state, and the node
class currently serving requests.

The fleet panel shows each configured remote GPU node with its current state. States
are: stopped (VM is off, no billing), starting (VM is booting, billing has begun),
available (VM is ready and healthy), failed (VM failed to start or become healthy),
and zombie (VM is running but unresponsive). For available nodes, the panel shows the
most recent probe latency in milliseconds.
### YoYo Fleet panel

Each node has an independent kill switch. The K key opens a dialog to toggle the
kill switch for any node, or to close all switches globally.
Lists each configured burst-compute node by name alongside its lifecycle state: one
of unknown, stopped, staging, running, available, failed to start, or zombie
(running but no longer answering health probes). The panel highlights only
available nodes as healthy; stopped, failed, and zombie nodes share a muted style.

### Organizational graph panel
### DataGraph panel

The graph panel shows the total entity count in the organizational knowledge graph,
the number of distinct edge types present, and the timestamp of the most recent
successful extraction. It shows the circuit breaker state for the graph service:
if the inference router's graph query path has experienced repeated failures and
opened its circuit, the panel displays the time elapsed since the circuit opened.
Shows the total entity count in the knowledge graph and the burst tier's
circuit-breaker state (the same circuit shown in the Gateway panel, since both
reflect the same tier).

### Queue panel

The queue panel shows the current depth of each priority queue level. P0 holds
background classification tasks. P1 holds extraction tasks awaiting a GPU node.
P2 holds training corpus generation and apprenticeship work. The panel also shows
the total completed and the current poison count — tasks that have failed the
maximum number of retry attempts and require operator review.
Shows extraction-queue depth: pending, in flight, paused, and completed counts,
plus how many jobs sit in quarantine. A poison count — jobs that failed enough
retry attempts to need operator review — is highlighted whenever it is non-zero.

### Cost panel
### Cost Today panel

The cost panel shows the current day's spending across all tiers in the deployment's
configured currency. The panel breaks down spending by node label: the batch node,
the express node, and the external API (if configured). This gives the operator
immediate visibility into whether a scheduled nightly drain has concluded and at
what cost.
Shows the current day's total spend, broken into the burst-compute portion and the
VM-hours portion, alongside the day's request count.

## Keyboard controls

| Key | Action |
|---|---|
| R | Immediate refresh — re-queries all status endpoints |
| K | Kill switch dialog — toggle per-tier or global kill switch |
| P | Policy dialog — select routing policy (balanced / drain-batch / drain-express / local-only) |
| G | Graph detail — show entity type breakdown and recent extraction activity |
| ? | Help overlay — show all keybindings |
| Esc | Close the help overlay |
| Q | Quit |

Routing-policy changes and the kill switch are real, operator-controlled mechanisms.
The kill switch is a hard stop no request can bypass; the routing policy (balanced,
drain-batch, drain-express, or local-only) is switchable at runtime. Both live behind
the gateway's own API, not this console — it only displays their effects.

## Technical characteristics

The console is a library crate that implements the Cartridge trait for the operator
console chassis. It loads at [[use-f-key-model|slot F9]]. Communication with the inference gateway uses
standard HTTP against the gateway's monitoring endpoints; no special protocol is
required. The console performs only read operations by default; write operations
(kill switch toggles, policy changes) require explicit keyboard confirmation.
standard HTTP against the gateway's monitoring endpoints.

The console uses a background polling task that fetches status data every ten seconds
and sends it to the rendering task via a channel. The rendering task does not block
on network requests; it displays whatever data arrived most recently. This design
ensures the console remains responsive even when the gateway is slow to respond.

The display degrades gracefully when individual status endpoints are unavailable.
Missing panels show a "— unavailable —" indicator rather than preventing the console
from rendering.
on network requests; it displays whatever data arrived most recently, so the console
stays responsive even when the gateway is slow to respond.

Plain-text mode is available via the `--plain` flag for terminal environments without
unicode support. Unicode status symbols are replaced with ASCII equivalents.

## Relationship to the inference gateway

The console is a read-mostly observer of the inference gateway. It does not participate
in routing decisions. Kill switch and policy commands sent through the console take
effect immediately in the gateway, but the console does not verify the effect beyond
showing the updated state on the next refresh cycle.

The console is deployed alongside the inference gateway on the same node. It does not
require network connectivity to external services to function. If the inference gateway
is unreachable, the console continues running and shows all panels as unavailable.
The console is a pure observer: it makes no write calls to the inference gateway at
all. It is deployed alongside the gateway on the same node and requires no network
connectivity to external services. If the gateway is unreachable, the console keeps
running and shows each panel as unavailable rather than crashing.