Skip to content

Build and destroy a ceremony box

The estate's root credentials are readable by one age identity that nobody holds and no host keeps. Its private key is split among K of N holders and reconstructed in memory, for the minutes a rotation takes, on a machine made for that rotation and destroyed after it. This page is that machine: how it is built, how you know it is right, and how it is taken away. The rotation itself — the shares, the listener, what is rotated — is Rotate a root credential.

What a ceremony box is

A virtual machine on the lab host, carrying CIS Level 2 Server whole and two roles: hardening and ceremony-box. It exists between one create and one destroy, and it is not in inventory/hosts.yaml — a machine made for one rotation is not part of the estate's standing declaration.

Three properties of the machine matter more than any control, and hardening/profiles/ceremony-box.yaml names them as its requires:

Property What it means How it is read
no_swap nothing the ceremony holds can be paged to a disk /proc/swaps carries no row
no_core_dumps a crash writes no memory image kernel.core_pattern is \|/bin/false and fs.suid_dumpable is 0
no_other_role nothing else on the machine could read the process no other role's paths or programs are there

They are read before anything is applied, and a plan carrying a refusal applies nothing at all, so a box that is wrong in any of the three is never converged into looking right.

A virtual machine and not a container. A machine container shares the lab's kernel: /proc/swaps is the lab's and kernel.core_pattern is not the box's to set, so the two properties that matter most are exactly the ones a container cannot answer for.

The facts hash

The three properties, as name=value lines sorted by name, hashed with SHA-256. create prints it; hardening/profiles/ceremony-box.yaml records it as expected_facts_hash; nerthus vault ceremony start refuses a box whose reading differs, before it opens a listener.

A hash that differs is not a mystery — create prints each fact beside it, so the line that moved is the answer. It moves for exactly three reasons: the box is wrong, the profile gained or lost a required property, or a property's expected value changed. All three are things somebody did, and all three are a merge request against Nerthus.Infra.

Before you build one

  • A holder's public key. The box accepts that one key and no other; create writes authorized_keys whole rather than appending to it.
  • A Nerthus.Infra checkout, for the profile, the pins and the box's door.
  • The lab reachable over SSH. create refuses any host that does not carry the lab role, and refuses a lab that serves a site.

The procedure

Dry run first. It reads the declaration, prints what it would launch, and reaches no machine:

cd ~/gitlab/margonem/nerthus/it/Nerthus.Config
nerthus-config ceremony-box create --infra ../Nerthus.Infra --key ~/.ssh/id_ed25519.pub

Then the build itself:

nerthus-config ceremony-box create --infra ../Nerthus.Infra --key ~/.ssh/id_ed25519.pub --no-dry-run

It launches the machine, converges hardening and then ceremony-box, plans both again, reads the three properties and prints their hash and the box's grade. It exits non-zero when the second plan is not clean, when the hash differs from the profile's, or when any control fails without an exemption.

The door is separate. inventory/cloudflare.yaml declares nerthus-admin-ceremony-box, whose host names the role rather than a machine — one tunnel serves whichever box is current, because a tunnel created and destroyed at every ceremony would be a Cloudflare resource nobody could keep track of. The connector is converged only when you hand create the token:

nerthus-config ceremony-box create  --tunnel-token ~/ceremony-token --no-dry-run

Without it the box has no door, which is the right answer for a box reached from the lab with incus exec.

What is not on the box

Platform itself. The ceremony runs nerthus vault ceremony, and that command and its runtime arrive with the task that writes it. Installing it here would mean either resolving dependencies no pins/<family>.yaml names, or a container runtime whose files another role already owns — so the box is the machine and the posture, and the runtime is the ceremony's.

Destroying it

nerthus-config ceremony-box destroy --infra ../Nerthus.Infra --no-dry-run

It refuses while a ceremony process is running on the box: a box deleted mid-rotation loses whatever the ceremony had not yet committed, which is a rotation half done at the issuers and recorded nowhere. End the ceremony first.

Afterwards it reads the storage pool back rather than trusting the delete, and reports any volume still named after the box. It is gone is the one claim this command makes, so it is the one thing it verifies.

When something refuses

What you see What it means
does not carry the lab role the host you named is not the lab; the harness runs nowhere else
carries no site the lab has a site declared on it, which ruling 56 says it may not
the box does not hold no_swap the machine has a swap device — a container, or an image that makes a swapfile
declares no nerthus-admin-ceremony-box tunnel you passed --tunnel-token and the declaration has no door to converge
is running a ceremony destroy found a ceremony process; end it rather than killing the machine