AI inference service

An AI request that leaves the building cannot be audited and cannot be recalled. service-slm is the platform's AI inference gateway — the workspace that houses the [[doorman-protocol| Doorman]] router and its supporting crates — and its central property is that every inference call, whatever tier ultimately serves it, crosses the Doorman's audit boundary first.

The Doorman routes, callers hint

service-slm implements the platform's tier-routing discipline: a caller submits a complexity hint, not a tier choice, and the Doorman picks one of three routes based on that hint plus live budget state.

Route	Where it runs	Model
Local	On the same host as the Doorman	Quantized OLMo 3 7B, served over HTTP
Yoyo	A preemptible multi-cloud GPU burst instance	A larger OLMo 3 model tuned for deeper reasoning
External	A licensed third-party API, allowlist-gated	A frontier model, for narrow precision-critical tasks only

These three internal routing tiers are a distinct system from the customer-facing commercial subscription ladder described in PointSav-LLM — the source code itself names the routing enum deliberately to avoid the two colliding.

The Doorman audit boundary

Every prompt and completion captured by the Doorman is written to an audit path before the response returns to the caller, forming the institutional record of every AI decision. The Doorman exists for three reasons:

1. Regulatory. ISO/IEC 42001, the AI management-system standard ¹, calls for an immutable log of AI-assisted decisions.
2. Operational. A self-healing system needs a corpus of its own past behaviour to improve against; the audit capture provides it.
3. Sovereign. No request reaches a third-party API without first passing through a boundary the operator controls.

Full detail on the Doorman's own routing and audit mechanics: Doorman protocol.

Model selection

The canonical local model is from the OLMo family, which ships with fully open weights and training-data documentation ² — a prerequisite for continued pre-training on an operator's own corpus, the long-term path to a domain-specialised model.

Why a small model, by default

A frontier-scale model imposes costs the Local tier is built to avoid: cloud egress, tens of gigabytes of RAM, and a request that cannot be meaningfully audited. A quantized 7B model is sufficient for most requests and fits inside the cost envelope of a low-cost node running alongside the rest of the platform. Specialisation and tiering, not scale by default, is the design principle — the Yoyo and External tiers exist precisely for the requests where more capability is genuinely needed.

See also

• Doorman protocol — the Doorman's routing and audit mechanics in detail
• Model tier discipline — the tier-routing discipline this service implements
• PointSav-LLM — the distinct, customer-facing commercial tier ladder
• SYS-ADR-07 — structured data never routes through AI; the Doorman implements this boundary
• Run local SLM inference — step-by-step guide: start the SLM service and submit inference requests from the console or API
• Run your first SLM query — step-by-step guide: read the Doorman health dashboard and submit your first prompt

1. ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system. https://www.iso.org/standard/81230.html ↩

2. Groeneveld, D. et al. 'OLMo: Accelerating the Science of Language Models.' arXiv:2402.00838, 2024. https://arxiv.org/abs/2402.00838 ↩