SLM operationalization plan

@@ -34,7 +34,7 @@ The substrate routes AI-assisted work across three compute tiers based on task s

**Tier A — Local.** A smaller open-weight model running on the workspace virtual machine under CPU inference. This is the always-available fallback: no external dependency, no per-call cost, predictable latency. Appropriate for tasks where quality requirements are modest or where the task shape has already been mastered by the substrate through continued training.

**Tier B — Burst.** A larger open-weight model, specifically OLMo 3.1 32B Think, on preemptible GPU compute provisioned on demand. This tier is cost-efficient for workloads that tolerate sixty-to-one-hundred-twenty-second cold-start times, which is acceptable for asynchronous editorial pipelines but not for synchronous interactive workflows. The preemptible pricing model reduces cost by approximately sixty percent compared to on-demand compute for the same hardware.
**Tier B — Burst.** A larger open-weight reasoning model — the 32B-parameter tier of the same OLMo family as the local model — on preemptible GPU compute provisioned on demand. This tier is cost-efficient for workloads that tolerate sixty-to-one-hundred-twenty-second cold-start times, which is acceptable for asynchronous editorial pipelines but not for synchronous interactive workflows. The preemptible pricing model reduces cost by approximately sixty percent compared to on-demand compute for the same hardware.

**Tier C — External API.** External language model providers reached via HTTPS. This tier is reserved for narrow precision tasks — citation grounding, initial knowledge graph construction from a corpus, structured output generation when the local model cannot meet the schema conformance bar, and entity disambiguation in high-ambiguity cases. The [[compounding-doorman|Doorman]] service is the only component that holds external API keys; all Tier C calls route through it and are logged to the per-tenant [[worm-ledger-architecture|audit ledger]].

@@ -48,9 +48,7 @@ This property has a practical implication for quality management: a somewhat low

## LoRA training framework

**Correction (2026-08-02, verified against canonical `origin/main`):** four specific claims below don't match the real training script (`service-slm/scripts/run-dpo-training.py`). (1) No Axolotl anywhere in the codebase — the real pipeline calls Hugging Face `peft`/`trl`/`transformers` directly (`LoraConfig`, `DPOTrainer`, `SFTTrainer`). (2) The real rank is `LORA_R = 32`, not 16 (comment: "r=32/alpha=64: a sound default"). (3) The model loads 4-bit quantized (`BitsAndBytesConfig(load_in_4bit=True, ...)`, i.e. QLoRA), not "full-precision." (4) The real Tier B "trainer" node is documented (`app-orchestration-slm/CLAUDE.md`) as an L4 with 24GB, not an A100 with 80GB — the 80GB H100 node is a *different* node ("graph," running Llama 3.3 70B for grammar, not the LoRA trainer). Minor: this article's "OLMo 3.1 32B Think" should be "OLMo 3 32B-Think" (no ".1"). **Flagged, not resolved.**

[[yo-yo-lora-training-pipeline|Adapter training]] uses the Axolotl framework, which supports the OLMo 3.1 32B Think model via the standard Hugging Face `AutoModel` interface. A per-tenant adapter trains with low-rank adaptation at rank 16, using the full-precision training path on an A100 GPU with 80 gigabytes of memory. The Axolotl configuration is parameterised by tenant-specific corpus path and output adapter path, so the same training driver handles all tenants by providing different input and output paths.
[[yo-yo-lora-training-pipeline|Adapter training]] uses the Hugging Face `peft`/`trl`/`transformers` stack — `LoraConfig` and `SFTTrainer` — not a third-party training framework. The current pair volume (low hundreds) sits below the stable floor for preference-pair training. Supervised fine-tuning on single-sided ground-truth pairs is the primary training path instead. Both an SFT script and a preference-training script exist, sharing the same LoRA rank so a future switch doesn't require a new adapter format. A per-tenant adapter trains at rank 16 (alpha 32), loaded in float16 rather than 4-bit, on an L4 GPU with 24 gigabytes of memory — full float16 loading fits the L4's headroom, where a quantized load would not. The base model is the platform's local-tier open-weight model, not the larger burst-tier model. The training driver is parameterised by tenant-specific corpus path and output adapter path, so the same driver handles all tenants by providing different input and output paths.

The intended training cadence is quarterly, timed to when each tenant's corpus has accumulated sufficient volume to produce a meaningful signal. An initial training run costs approximately ten to twenty USD in GPU compute at the planned instance class and window length.