Incus virtual machines

incus Runs each app in its own Incus virtual machine, so no two apps share a kernel. Incus decides which cluster member a VM runs on. Pando reaches each VM through a forwarding agent it runs there, and a network ACL admits nothing else. Built images reach the VMs through the install's image registry. The edge keeps running on the Docker runtime.

Setup

  1. Where is Incus?

    Settings address

  2. Connect over HTTPS

    Skip this step for a unix:// address.

    1. On the Incus server, run incus config set core.https_address=:8443.
    2. Run openssl req -x509 -newkey rsa:3072 -nodes -days 3650 -subj /CN=pando -keyout pando.key -out pando.crt.
    3. Run incus config trust add-certificate pando.crt --restricted --projects default.
    4. Paste /var/lib/incus/server.crt below, or /var/lib/incus/cluster.crt on a cluster.
    5. Paste pando.crt and pando.key together below.

    Settings server_certificate, tls_client

  3. Choose a network

    1. Run incus network list to see the networks.

    Settings network, proxy_sources

  4. Set up the forwarding agents

    1. Run pando host-agent new-authority and paste its output below.

    Settings agent_authority, agent_image

Settings (22)
addressIncus address

unix:///var/lib/incus/unix.socket when Pando runs on the Incus server, or https://host:8443 with a client certificate.

string · For example unix:///var/lib/incus/unix.socket

server_certificateServer certificate

For an https:// address: /var/lib/incus/server.crt on the Incus server, or cluster.crt on a cluster.

string

tls_clientTLS client certificateCredential

For an https:// address: a client certificate and its key, as PEM, added with incus config trust add-certificate.

string

networkNetworkRequired

The Incus network each app's VM is attached to. Pando must be able to reach addresses on it. On a cluster, an OVN network, or a bridge with a target member.

string · For example incusbr0

proxy_sourcesPando's addressesRequired

The addresses Pando connects to app VMs from, as seen on that network, as addresses or ranges.

string

agent_authorityAgent certificate authorityRequiredCredential

The output of pando host-agent new-authority. Pando issues its own certificate and each VM's agent's from it.

string

agent_imagePando imageRequired

The image of this version of Pando, which each VM's forwarding agent and each restricted app's egress gateway run. Every VM must be able to pull it.

string · For example trypando/pando:0.4.0

projectProjectAdvanced

The Incus project VMs, volumes and the guest image are made in. A restricted client certificate must be allowed it and the network's project.

string · Default default

storage_poolStorage poolAdvanced

The pool for each VM's disk and the apps' volumes.

string · Default The pool of the project's default profile's root disk

targetCluster targetAdvanced

A cluster member, or @ and a cluster group, that VMs run on.

string · Default Any member, as Incus places them

profilesProfilesAdvanced

Incus profiles applied to each VM. Pando sets the VM's disk and NIC itself; a profile that adds another NIC gives the VM a way in that skips Pando.

string · Default None

guest_imageGuest imageAdvanced

An Incus image alias or fingerprint to start VMs from, with Podman serving /run/podman/podman.sock and socat installed.

string · Default Debian 13 with Podman, prepared by Pando on first use

agent_portAgent portAdvanced

The port each VM's agent listens on, and the only one the network ACL opens, to Pando's addresses.

int · Default 7443

root_size_bytesVM disk sizeAdvanced

Each VM's disk, in bytes, for images and what workloads write outside their volumes.

int · Default 21474836480 (20 GiB)

volume_size_bytesVolume sizeAdvanced

The size of each app volume, in bytes, unless the app asks for more.

int · Default 10737418240 (10 GiB)

guest_overhead_bytesVM memory overheadAdvanced

Memory each VM has beyond its app's limits, in bytes, for the guest system and Podman.

int · Default 536870912 (512 MiB)

default_cpusDefault CPUsAdvanced

CPUs for the VM of an app that sets no limits.

int · Default 1

default_memory_bytesDefault memoryAdvanced

Memory for the VM of an app that sets no limits, in bytes.

int · Default 1073741824 (1 GiB)

network_poolApp network rangeAdvanced

The IPv4 range app networks inside each VM take their addresses from. Must not overlap the Incus network.

string · Default 10.213.0.0/16

network_block_bitsApp network sizeAdvanced

The size of each app's private network inside its VM, as a prefix length from 24 to 29.

int · Default 28

total_cpu_millisCPU for PandoAdvanced

Thousandths of a core Pando may use across the members, when it shares them with other work.

int · Default What the members have

total_memory_bytesMemory for PandoAdvanced

Bytes of memory Pando may use across the members, when it shares them with other work.

int · Default What the members have