VM Management

VM lifecycle and management

Compute providers manage virtual machines through the vm.Manager interface. The production implementation uses Lima (Linux virtual machines via QEMU) to create isolated Ubuntu 24.04 VMs with SSH access. VMs are provisioned on lease acceptance and torn down on settlement. Boot time is approximately 21 seconds from lease acceptance to a running VM with SSH.

Manager interface

type Manager interface {
    Provision(leaseHash string, res Resources, accessPubKey string) (*Info, error)
    Teardown(leaseHash string) error
    Exec(leaseHash string, command string) (*ExecResult, error)
    DialSSH(leaseHash string) (net.Conn, error)
    Get(leaseHash string) (*Info, error)
    List() []*Info
}

Method

Description

Provision

Create and start a VM with the specified resources. Injects the consumer's SSH public key via cloud-init. Returns VM info including the SSH local port.

Teardown

Stop and delete the VM via limactl delete --force. Called automatically on lease settlement.

Exec

Execute a shell command inside the VM via limactl shell. Returns exit code, stdout, and stderr.

DialSSH

Open a TCP connection to the VM's SSH port on localhost. Used by the tunnel protocol to proxy SSH sessions from remote consumers.

Get

Retrieve the current state of a VM by lease hash from the in-memory map.

List

Return all VMs managed by this provider.

Lima implementation

The LimaManager (vm/lima_manager.go) manages QEMU-based VMs using the limactl CLI tool from the Lima project.

Architecture

xe-node process (provider)
    │
    ├── LimaManager
    │     ├── limactl create (from YAML template)
    │     ├── limactl start (boots QEMU VM)
    │     ├── limactl shell (exec commands)
    │     └── limactl delete --force (teardown)
    │
    └── QEMU processes (one per active lease)
          ├── xe-<leaseHash[:12]> → Ubuntu 24.04 VM
          └── xe-<leaseHash[:12]> → Ubuntu 24.04 VM

Initialisation

When a provider node starts with --provide, the LimaManager is created:

mgr, err := vm.NewLimaManager(cfg.DataDir, cfg.LimactlPath)
  • LIMA_HOME is set to {dataDir}/lima — all VM state, disk images, and sockets live here
  • Template directory is {dataDir}/lima-templates — YAML templates are written here before limactl create
  • Images directory is {dataDir}/images — the Ubuntu cloud image is cached here
  • On startup, limactl --version is executed to verify the tool is available
  • If the Ubuntu cloud image is not present locally, it is downloaded automatically from the upstream Ubuntu cloud images server (~600 MB)
  • The manager logs the Lima version and advertised resource capacity

VM naming convention

VMs are named xe-{leaseHash[:12]} — the first 12 hex characters of the lease block hash. This keeps names short while avoiding collisions. For example, lease hash 6226fb52501a9052b533... creates a VM named xe-6226fb52501a.

Provision flow

When Provision(leaseHash, resources, accessPubKey) is called:

  1. Convert access key — the accessPubKey (ed25519 public key as 64-char hex string) is converted to SSH authorized_keys format. This is the consumer's key for SSH access to the VM.

  2. Render template — a Lima YAML template is generated referencing the local Ubuntu 24.04 cloud image with the requested resources and SSH key:

    vmType: "qemu"
images:
  - location: "file:///var/lib/xe-node/images/ubuntu-24.04-x86_64.img"
    arch: "x86_64"
cpus: 1
memory: "512MiB"
disk: "5GiB"
mounts: []
containerd:
  system: false
  user: false
provision:
  - mode: system
    script: |
      #!/bin/bash
      mkdir -p /root/.ssh && chmod 700 /root/.ssh
      echo '<ssh-ed25519 key>' >> /root/.ssh/authorized_keys
      chmod 600 /root/.ssh/authorized_keys
      sed -i 's/#PermitRootLogin.*/PermitRootLogin prohibit-password/' /etc/ssh/sshd_config
      sed -i 's/#PubkeyAuthentication.*/PubkeyAuthentication yes/' /etc/ssh/sshd_config
      sed -i 's/#PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config
      systemctl restart sshd

    The image is referenced via file:// from the local cache at {dataDir}/images/. SSH is pre-installed in the Ubuntu cloud image — the provision script only injects the consumer's key and restarts sshd.

  3. Write template — saved to {dataDir}/lima-templates/{vmName}.yaml

  4. Create VM — limactl create --name={vmName} {templatePath} uses the cached Ubuntu cloud image as a copy-on-write backing file (instant, no download)

  5. Start VM — limactl start {vmName} boots QEMU with KVM acceleration. Ubuntu boots in ~15 seconds, cloud-init injects the SSH key, and Lima confirms SSH readiness. Total: ~21 seconds.

  6. Wait for SSH — polls limactl list --json every 2 seconds for up to 2 minutes until the VM reports an SSH port. This port is on 127.0.0.1 and is the tunnel target.

  7. Store state — the VM info (lease hash, SSH port, status, resources) is stored in the in-memory vms map

Resource constraints

Resource

Minimum

Default

Description

vCPUs

1

From lease

QEMU -smp parameter

Memory

512 MiB

From lease

QEMU -m parameter

Disk

5 GiB

From lease

QCOW2 disk image size

If the lease requests less than the minimum, the minimum is used.

Teardown

When Teardown(leaseHash) is called (triggered by lease settlement):

  1. Execute limactl delete --force {vmName} — this stops the QEMU process and removes all VM artifacts (disk, sockets, logs)
  2. Delete the template file from {dataDir}/lima-templates/
  3. Remove the entry from the in-memory vms map

Command execution

Exec(leaseHash, command) runs commands inside the VM via Lima's shell mechanism:

limactl shell {vmName} -- sh -c "{command}"

Returns an ExecResult with: - ExitCode — the command's exit code - Stdout — captured standard output - Stderr — captured standard error

SSH connection

DialSSH(leaseHash) opens a raw TCP connection to the VM's SSH port:

net.Dial("tcp", fmt.Sprintf("127.0.0.1:%d", vm.sshPort))

This is used by the tunnel protocol to proxy SSH sessions from remote consumers through libp2p streams to the VM's local SSH server.

Data types

Resources

type Resources struct {
    VCPUs    uint64 `json:"vcpus"`
    MemoryMB uint64 `json:"memory_mb"`
    DiskGB   uint64 `json:"disk_gb"`
}

Info

type Info struct {
    LeaseHash   string     `json:"lease_hash"`
    Status      string     `json:"status"`
    Resources   *Resources `json:"resources,omitempty"`
    CreatedAt   int64      `json:"created_at"`
    Error       string     `json:"error,omitempty"`
}

VM status values

Status

Description

provisioning

VM is being created (limactl create/start in progress)

running

VM is active, SSH port is available, accepting commands and tunnel connections

stopped

VM has been torn down after lease settlement

error

VM encountered a fatal error during provisioning or execution

ExecResult

type ExecResult struct {
    ExitCode int    `json:"exit_code"`
    Stdout   string `json:"stdout"`
    Stderr   string `json:"stderr"`
}

VM lifecycle

lease_accept confirmed
        │
        ▼
  ┌──────────────┐    vm_credentials     ┌──────────┐
  │ Provision    │ ──────────────────────►│ Consumer │
  │ (limactl     │    (direct msg)       └──────────┘
  │  create+start)│
  └──────────────┘
        │
        ▼
  ┌──────────────┐
  │ Running      │ ◄── SSH tunnel from consumer (via libp2p)
  │ (Ubuntu VM)  │ ◄── Exec commands via API or messaging
  └──────────────┘
        │
        │  lease expires + settle
        ▼
  ┌──────────────┐
  │ Teardown     │
  │ (limactl     │
  │  delete)     │
  └──────────────┘
  1. Provision — triggered by autoAcceptLease() after the lease_accept block is confirmed (~21 seconds). Lima creates an Ubuntu 24.04 VM from the cached cloud image with the consumer's SSH key injected via cloud-init.
  2. Credential delivery — the provider sends a vm_credentials direct message to the consumer with the VM's SSH connection details.
  3. Running — the consumer can access the VM via:
    • SSH tunnel through the SSH gateway (remote access via libp2p)
    • Exec API via POST /vms/{lease}/exec (HTTP API)
    • Direct messaging via the vm_exec protocol (libp2p)
  4. Teardown — triggered by settleLease() after the lease_settle block is confirmed. The VM is deleted and its resources freed.

VM access methods

The SSH gateway provides transparent remote access to VMs. The consumer connects via standard SSH:

ssh -i ~/.ssh/lease_key -p 2222 <leaseHash>@<gateway-host>

The gateway authenticates using the AccessPubKey from the lease block, then tunnels the session through libp2p to the provider node, which proxies to the VM's local SSH server. See SSH Gateway & Tunnel Protocol for full details.

HTTP exec API

For programmatic access, the provider exposes an exec endpoint:

POST /vms/{leaseHash}/exec
Content-Type: application/json

{"command": "uname -a"}

Response:

{
    "exit_code": 0,
    "stdout": "Linux lima-xe-6226fb52501a 6.8.0-106-generic #106-Ubuntu SMP ...\n",
    "stderr": ""
}

Direct messaging (vm_exec)

Consumers can also execute commands via the libp2p messaging protocol:

Consumer                    Provider
   │                           │
   │  vm_exec {lease, cmd}     │
   │──────────────────────────►│
   │                           │  verify consumer identity
   │                           │  limactl shell {vm} -- sh -c {cmd}
   │  ExecResult {code, out}   │
   │◄──────────────────────────│

The provider node verifies the requesting peer owns the consumer account before executing.

Message handlers

The node registers VM-related message handlers:

Message

Direction

Description

identify

Any

Returns the node's account address

vm_credentials

Provider → Consumer

Delivers VM SSH connection info after provisioning

vm_exec

Consumer → Provider

Executes a command in the VM via limactl shell

vm_status

Consumer → Provider

Returns current VM info

Provider node flags

Providers enable compute leasing with the following CLI flags:

Flag

Type

Default

Description

-provide

bool

false

Enable provider mode

-vcpus

int

2

Number of vCPUs available for leasing

-memory

int

2048

Memory in MB available for leasing

-disk

int

20

Disk in GB available for leasing

-ssh-port

int

0

SSH gateway listen port (0 = disabled)

-limactl-path

string

limactl

Path to limactl binary

xe-node -provide -vcpus 4 -memory 8192 -disk 100 -ssh-port 2222

When -provide is set, the node:

  1. Initialises the LimaManager with the configured limactl path.
  2. Sets up the tunnel protocol handler for incoming SSH tunnel requests.
  3. Advertises available resources via marketplace gossip every 5 minutes.
  4. Watches for incoming lease blocks every 5 seconds and automatically accepts them.
  5. Runs the settleLoop() to auto-settle expired leases every 10 seconds.

Marketplace discovery

Providers and consumers find each other through the marketplace gossip topic (xe/marketplace):

  1. Consumer publishes a ResourceRequest with desired vCPUs, memory, disk, and duration.
  2. Provider receives the request, checks available resources, and replies with a ResourceOffer including the computed cost.
  3. Consumer receives the offer and creates the lease block.

The entire negotiation happens over gossip before any on-chain blocks are created.

Infrastructure requirements

Lima uses QEMU as its virtualisation backend. The host must have:

  • QEMU installed (qemu-system-x86_64)

  • Lima installed (limactl v1.0.6+)

  • KVM support — Lima requires /dev/kvm (hardware virtualisation extensions: Intel VT-x or AMD-V). Hosts without KVM (most cloud VPS) cannot run Lima VMs.

  • KVM permissions — the xe user must be able to access /dev/kvm. Since pm2's uid/gid setting drops supplementary groups, add a udev rule:

    # /etc/udev/rules.d/99-kvm.rules
KERNEL=="kvm", MODE="0666"

  • Non-root execution — Lima refuses to run as root. The node process must run as a non-root user (e.g., the xe service user).

  • Disk space — the Ubuntu cloud image is ~600 MB. Each running VM uses additional disk for its QCOW2 overlay (copy-on-write from the shared base image).

[!WARNING] KVM requirement Standard cloud VPS instances typically do not expose hardware virtualisation. Bare-metal servers or VPS with nested virtualisation enabled are required for Lima VM support. Without KVM, the lease lifecycle (creation, acceptance, attestation, settlement) still works — only the actual VM boot fails.

Manager interfaceLima implementationArchitectureInitialisationVM naming conventionProvision flowResource constraintsTeardownCommand executionSSH connectionData typesResourcesInfoVM status valuesExecResultVM lifecycleVM access methodsSSH gateway (recommended)HTTP exec APIDirect messaging (vm_exec)Message handlersProvider node flagsMarketplace discoveryInfrastructure requirements