PPN hypervisor resource pool

The PointSav Private Network (PPN) hypervisor layer is designed to manage a per-node pool of CPU and RAM, dynamically allocating those resources across the virtual machines it runs. This is the intended mechanism by which the PPN would give more or less compute capacity to each Totebox Archive VM in response to workload demand. Neither mechanism described below is built yet — os-infrastructure's real boot code is a bare-metal Multiboot2 stub (framebuffer text output and an mDNS peer scan) with no resource-pooling logic, and no virtio_balloon or cgroups v2 cpu.weight code exists anywhere in the crate. This article describes the planned design.

One pool per physical node

Each physical PPN node — a GCP instance, an on-premises server, a leased machine — controls a pool bounded by its own hardware. The pool is not shared across nodes. A node with 31 GB of RAM manages 31 GB; it does not borrow from a neighbouring node.

Cross-node workload placement is a separate concern: the Totebox Orchestration Layer (gateway-orchestration-command-1) decides which physical node a cluster-totebox instance runs on, based on MBA pairing and available capacity signals. Once that decision is made, the receiving node's hypervisor manages the local resource pool for that VM. The PPN pool and the Totebox scheduler are orthogonal.

Memory pool: virtio_balloon (planned)

The intended primary memory reclaim mechanism is the virtio_balloon paravirtual device. Once built, every VM provisioned by os-infrastructure would start with a balloon driver, running as a standard kernel module inside the guest operating system.

How inflation works (reclaiming memory):

The hypervisor (balloon controller) signals the balloon driver to inflate by N pages.
The driver allocates those pages inside the guest, removing them from the guest's usable address space.
The hypervisor recovers those physical pages for the node-level pool.
The pool grows by N pages; the guest's available RAM shrinks by N pages.

How deflation works (giving memory back):

The hypervisor signals the balloon driver to deflate.
The driver releases balloon pages back into the guest's free list.
The guest's available RAM grows; the pool shrinks.

The pool at any instant:

pool_available = physical_ram − Σ(balloon_minimum across all VMs)

Each VM has a minimum balloon reservation below which the controller will not inflate. This prevents a VM from being starved of memory when the node is under pressure.

CPU pool: vCPU scheduling weights (planned)

CPU pool management is designed around the Linux cgroups v2 cpu.weight interface. Each QEMU process (one per VM) would be placed in a cgroup with a weight drawn from the capability ledger. Under CPU contention, the scheduler would distribute vCPU time proportionally to those weights; when the node is not under contention, all VMs would run at full speed regardless of weight.

A cluster-totebox VM running an active inference workload (via service-slm) could be assigned a higher weight than an idle archive VM. The ledger entry would be the authoritative weight, applied by os-infrastructure at VM launch and adjustable live — once this mechanism is built.

Relationship to os-orchestration

os-orchestration is a data-layer aggregator. It is designed to aggregate data access across Totebox Archives using capability-based queries that return only result rows, never raw records. It is stateless and holds no keys to archives.

os-orchestration does not allocate CPU. It does not adjust memory. It does not communicate with the hypervisor balloon controller. The two layers are designed to be blind to each other:

The hypervisor knows a VM is consuming N pages and Y vCPU percent. It does not know whether the VM is running os-totebox, os-orchestration, or anything else.
The Totebox Archive inside the VM knows nothing about balloon inflation, cgroup weights, or which physical node it is on.

This is the isolation invariant: the hypervisor has zero read capability over VM-internal state.

Freely transferable archives

Because the hypervisor is designed to manage only VM lifecycle and resource allocation — not the data inside the VMs — a Totebox Archive can be stopped, the disk image copied to another node, and restarted there without any change to its data or its identity; this property holds regardless of whether the resource-pooling mechanisms above are built. The destination node's hypervisor would allocate resources from its own pool for the relocated VM, once that pool mechanism exists.

This is the freely transferable property of Totebox Archives: the bootable disk image is the archive; the resource pool is the node's infrastructure. Moving the image moves the archive. The new node's pool absorbs the workload.

Implementation status

The virtio_balloon device flag is available in QEMU 7.x and can be added to a VM launch command to install the balloon driver in a guest — as a standalone QEMU capability, independent of whether os-infrastructure uses it. Today, os-infrastructure does not: no code in the crate adds the flag, inflates or deflates a balloon, or applies a cgroups v2 weight. A separate, standalone demo script exercises the mechanism manually via the QEMU monitor, launching a single test VM with the balloon device present from boot and letting an operator drive it by hand:

(qemu) info balloon      # show current guest-visible RAM
(qemu) balloon 128       # request guest to give back memory down to 128 MB
(qemu) info balloon      # confirm reclaim

That script proves the underlying QEMU mechanism works; it is not evidence that os-infrastructure has integrated it. The balloon controller — the component that would decide when to inflate or deflate each VM's balloon in response to demand signals, and the cgroups v2 weight assignment described above — are both planned work, not yet started.

Planned: cross-node resource extension

The per-node pool is the implemented layer. The planned distributed extension is intended to allow VMs to borrow compute from other physical nodes in the mesh when local capacity is under pressure.

Reboot not required. Standard pool operations — balloon inflation, deflation, and cgroups v2 weight changes — are dynamic. The balloon controller signals the in-guest driver; the driver responds; the node pool adjusts. No guest restart or host reboot is needed. This holds for both the current manual-operator flow (QEMU monitor) and the planned automated controller.

virtio-mem (upstream Linux kernel since 5.8; QEMU since 5.1) is the intended mechanism for the cross-node layer. Where virtio_balloon inflates and deflates a single device, virtio-mem supports fine-grained hot-plug and hot-unplug of individual memory blocks. The intended model: a lending node advertises unused blocks to a requesting VM on another node over the WireGuard mesh. The seL4 capability model is intended to ensure the lending node retains no read capability over the blocks it lends — the physical pages are exclusively mapped into the borrowing VM's address space.

Cross-node placement decisions are intended to remain with gateway-orchestration-command-1 (Totebox Orchestration layer). The distributed capability ledger — planned for development in moonshot-protocol and moonshot-database — is intended to carry cryptographically signed lending grants keyed to each node's pairing-ceremony identity. Revocation is intended to propagate as a Merkle DAG gossip across the mesh without relying on a central authority.

The automated balloon controller — the component inside os-infrastructure that would trigger inflation and deflation in response to demand signals — is a planned milestone that precedes the cross-node lending layer.

Navigate

Topics

Guides

Resources

PointSav network