SLM and Yo-Yo operational state
service-slm is the platform's Ring 3 component — the optional
intelligence layer. It is a tiered inference router that clusters and contributors use to
delegate routine work: editorial polish, mechanical schema-conforming edits, bilingual
translation drafts, and structured-output generation. Work is handled locally or on a
dedicated GPU burst VM, without routing to a third-party API by default. Rings 1 and 2
(boundary ingest and knowledge processing) function fully without it — Ring 3 is structurally
optional.
Yo-Yo is the platform's on-demand GPU burst instance — a GCE VM that runs a larger model than the local tier can, starts on demand, and shuts down after a period of inactivity. A third tier (external API) exists in the routing configuration but is unused in normal operation, reserved for cases genuinely requiring it.
The Doorman boundary
Every inference request crosses the Doorman before reaching a model
tier. The Doorman does not run as its own standalone process — it is bundled together with
service-content inside a single service (local-totebox.service). Its responsibilities cover
the full request lifecycle: holding every API key so no key is dispersed across call sites,
routing requests to the correct tier by complexity, appending every transit to a per-tenant
audit ledger, and draining the apprenticeship brief queue described below.
Local tier — always available
The local tier runs llama-server (the C++ HTTP server from llama.cpp) on the workspace VM's
own CPU. The model loaded is a quantized OLMo 3 7B Instruct build — the instruction-tuned
variant, not the "Think" reasoning variant. Throughput on a CPU-only workspace VM is on the
order of a few tokens per second — sufficient for short briefs and trivial completions, not for
routine editorial work at scale. This latency ceiling is what motivated the burst tier below.
Yo-Yo tier — burst GPU
The Yo-Yo instance runs llama-server with GPU support on a separate GCE instance in
us-central1-a, on hardware with one NVIDIA L4 GPU (24 GB VRAM), provisioned on-demand
rather than as a spot instance — spot capacity for this GPU class proved unreliable across
multiple US zones during initial bootstrapping. The model is a larger OLMo 3 model tuned for
deeper reasoning. Network access to the instance's inference port is restricted by firewall
rule to the workspace VM's internal address only, and every request is authenticated with a
bearer token the Doorman holds.
Currently down. As of this writing, the live Doorman health check reports the Yo-Yo tier's circuit open on all three of its configured labels, due to sustained health-probe failures — this tier has not been serving requests for an extended period. Requests that would route here currently fall back or queue rather than complete on this tier; this is a live operational fact, not a design description.
Provisioning
A fresh Yo-Yo instance is built from a startup script covering package installation, a CUDA
toolkit and llama-server build from source, model download, bearer-token generation, and
systemd unit configuration — a documented, multi-step process whose iteration history
(driver/kernel version mismatches, compilation memory limits, download reliability) is
preserved in the script's own inline comments and the workspace changelog.
The apprenticeship brief queue
Every commit triggers a capture hook that writes an engineering corpus tuple and a shadow brief to a durable queue. The Doorman's drain worker polls this queue, dispatches each brief to the local tier by default (or the burst tier above a size threshold), and on completion writes a corpus tuple at the review stage. This mechanism is durable across Yo-Yo idle-shutdown windows, Doorman restarts, and apprentice timeouts — the queue accumulates while the burst tier is stopped, and the backlog drains without loss once it restarts. As of this writing, the live queue reports several thousand pending entries and a large poisoned (failed-and-quarantined) count relative to completions — worth a dedicated look by whoever owns this pipeline, not something this article resolves.
Cost ceiling — the idle-shutdown monitor
An idle-shutdown monitor polls the Yo-Yo VM for active inference activity on a regular schedule and stops the instance after a sustained period with none, keeping always-on GPU cost from applying to idle time. The monitor runs from the workspace VM rather than the Yo-Yo VM itself, since the workspace VM's service account holds the cloud permissions needed to stop an instance and the Yo-Yo VM's does not.
What runs on the burst tier
The platform's engineering workflow routes routine work here: mechanical documentation
updates, schema-conforming edits, pattern-based refactors, bilingual translation drafts,
routine status reports, and boilerplate code. Architectural decisions, novel design, and
cross-layer coordination route to a frontier-model tier instead. service-slm is the
multiplier for routine work; the frontier model is reserved for judgment calls.
See also
- Compounding substrate — the architectural pattern this implements
- AI inference service — service-slm's tier-routing overview
- Apprenticeship substrate — how training signal accumulates from operational corpus tuples
- Brief queue substrate — the durable queue connecting the brief queue to tier processing
- WORM ledger substrate: four-layer architecture and two boot envelopes — the audit ledger that records every external call
References
- Optional Intelligence Layer — Ring 3 is structurally optional; Rings 1 and 2 function without it.
- AllenAI OLMo 3 model family. Apache 2.0 (model weights); Open Data Commons (training data). [olmo3-allenai] https://huggingface.co/allenai
- NI 51-102 Continuous Disclosure Obligations. British Columbia Securities Commission. [ni-51-102] https://www.bcsc.bc.ca/securities-law/law-and-policy/instruments-and-policies/5-ongoing-requirements-for-issuers-insiders/current/51-102
- OSC Staff Notice 51-721 Forward-Looking Information Disclosure. Ontario Securities Commission. [osc-sn-51-721] https://www.osc.ca/en/securities-law/instruments-rules-policies/5/51-721/osc-staff-notice-51-721-forward-looking-information-disclosure