Skip to content

Commission a rented host

Nerthus.Core (until cutover). This page describes the frozen system that runs today and is deleted at cutover. Replaced by: Adopt a host.

A rented server arrives with a root password in cleartext, password authentication on, no firewall and a public IPv4. This page is the hour between that mail and Bootstrap a fleet host, and its order is the whole of it: verify, key, rotate, vault, shred, harden. eder, the fleet's first rented host, was commissioned this way on 2026-08-21.

How you know you are here

You bought a machine and a supplier mailed you its address, its host-key fingerprints and a provisioning password. Nothing else on this site applies yet — bootstrap.sh refuses a box with no docker, and running it on a box with a mailed password would put the fleet's device keys beside a credential with copies nobody can count.

A contributor device skips this page entirely. It has no mailed password and no public address.

Why this order

Each step is only safe because the one before it happened.

Step What it would cost out of order
Verify the fingerprints a password typed into an unverified host is a password given away, and the supplier's mail is the only out-of-band channel this box will ever have
Install a key, prove it rotating a password over a session that is itself password-authenticated, unproven, is how a box locks you out
Rotate the password the mailed value is an exposure with a clock on it, not a secret to be filed
File the replacement the value exists in /etc/shadow as a hash and nowhere else; lose it and the provider console is closed too
Shred the plaintext a laptop copy is the copy an audit cannot enumerate
Harden until sshd and the firewall are closed, every step above protects one of two doors

1. Verify the host keys before the first password

Read the three fingerprints from the machine and compare them, byte for byte, against the supplier's mail. Do this before sending anything.

ssh-keyscan <address> 2>/dev/null | ssh-keygen -lf -

All three must match. Pin them in ~/.ssh/known_hosts on the workstation that will administer the box. A rebuild changes them, and that is the point: the next mismatch is a question, not a prompt to accept.

2. Install a key, and prove it before anything depends on it

Put the operator's public key in /root/.ssh/authorized_keys, mode 0600, directory 0700. Then prove key-only authentication works while the password still does:

ssh -o BatchMode=yes -o PasswordAuthentication=no root@<address> id -un

root on stdout is the proof. Anything else means stop — the password is still the only way in and it is about to be destroyed.

3. Rotate the mailed password

Generate the replacement on the workstation, 32 random alphanumerics, and set it over stdin so the value never appears in an argument vector on either machine:

printf 'root:%s' "$new" | ssh root@<address> chpasswd

4. Prove both directions, and mind the negative control

The new password must be accepted and the mailed one refused. The second half is the one that is usually skipped, and the obvious script gets it wrong.

A wrong password hangs; it does not fail

sshd re-prompts on a wrong password, and an expect script that has sent its one value simply waits for its whole timeout. That timeout looks exactly like the refusal you wanted, and it would arrive whether the password worked or not. Set NumberOfPasswordPrompts=1 and branch explicitly on Permission denied. Without both, the negative control is a timeout wearing a result's clothes.

Record two lines and keep them: the new password ACCEPTED, the old one REJECTED.

5. File the replacement in the vault

A host credential that is not a tunnel token still lives at sejf/hosts/<town>.yaml in Nerthus.Core.Infra.CommonThe secret tree owns the layout. Encrypt to both declared recipients, add the inventory.yaml row, and verify the round trip with the operator identity and the break-glass identity before deleting anything.

Then run the selftest, which is what notices a wrong recipient:

pwsh -c 'Invoke-Build sejf:selftest'

6. Shred every plaintext copy

The mailed value and the replacement, on the workstation and in any scratch directory. Then sweep the box: no /root/.bash_history, and authorized_keys holding exactly the keys you put there.

The credential now exists as a hash in /etc/shadow and as ciphertext in the vault. That is the intended state, and it is why step 5 comes before this one.

7. Close the two doors

Keep the root password. Disabling password authentication on sshd does not make it idle — it promotes it to the recovery credential, because the provider's out-of-band console is a password prompt with no key path. It is the door left when sshd is misconfigured or the box boots into an emergency shell.

  • An admin account that is not root. One shell account in sudo, the operator key in its ~/.ssh/authorized_keys. Set the shadow password field to *, never !: passwd -l writes !, and OpenSSH refuses a locked account before it reads any key, with User <name> not allowed because account is locked. passwd -S prints L for both, so the two are indistinguishable in its output — read /etc/shadow to tell them apart. Then decide NOPASSWD deliberately and say which posture you chose in sudoers.d/, because sudo that costs a password needs an account that has one.
  • sshd, as a file in /etc/ssh/sshd_config.d/ rather than edits to sshd_config, which a package upgrade overwrites:

    PermitRootLogin              prohibit-password
    PasswordAuthentication       no
    KbdInteractiveAuthentication no
    

    prohibit-password rather than no: the converge agent runs as root, and a locked-out root on a rented box is a support ticket. Run sshd -t, reload, then open a second session from a new shell before closing the first. Confirm with sshd -T, which reports the effective config rather than the file's intent.

  • A firewall, and there is almost nothing to allow. The host opens no inbound port for the fleet: compose/docker-compose.yml declares no ports: key and cloudflared runs network_mode: "service:nerthusd", so reads arrive through the tunnel and reach the daemon on loopback. Allow established and related, loopback, ICMP and ICMPv6 — dropping ICMPv6 breaks IPv6 neighbour discovery, not just ping — and inbound tcp/22. Drop the rest inbound, leave outbound open. Use nftables; persist it and reboot to check it survived, which is the only test that distinguishes a firewall from a ruleset that happens to be loaded.

    A published container port is reachable through a closed firewall

    Docker inserts its own DOCKER chain ahead of the INPUT policy. The rule above holds only while the fleet's compose file publishes nothing. A later change that adds a ports: line takes the daemon out from behind this firewall, silently.

  • Unattended security upgrades, with automatic reboots off. A reboot the fleet did not schedule takes the daemon and the tunnel with it, and the health sweep reports a host that is fine as a host that is gone.

  • The hostname set to the town slug, so the machine answers with the name the roster, the DNS record and the tunnel already use.

How you know you are out of it

  • the three fingerprints match the mail and are pinned
  • the mailed password is REJECTED and its replacement ACCEPTED
  • the replacement decrypts from the vault with both identities and exists in no plaintext anywhere
  • sshd -T reports passwordauthentication no and permitrootlogin prohibit-password
  • a non-root account opens a session with the key and its sudo posture is written down
  • nft list ruleset is non-empty after a reboot

Then go to Bootstrap a fleet host at step 1.

See also

  • Bootstrap a fleet host — what this page hands off to, and the rented-server shape
  • The secret tree — where sejf/hosts/<town>.yaml sits and what an inventory row owes
  • Rotate secrets — re-minting a tunnel token or a PAT, which is a different act on a running host
  • Offboard a host — undoing all of this, including the grants provision never made