Rotate a root credential¶
A root credential is one that mints others and that no rotation of theirs replaces: an account
login, its two-factor recovery codes, a provider login, the recovery age identity. They stay out
of the vault entirely (ruling 39) — out of sops, off every host, out of CI — and in a person's keychain
or on offline media. Every one of them has a tier: root row in
Nerthus.Infra/vault/inventory.yaml naming its holder, its ceiling and the section of this page that
rotates it.
nerthus secrets verify refuses a root credential whose value is in a vault. The absence of one is
the correct state and is reported as unprobeable with the reason, not as a missing value — the two
look alike in a report and mean opposite things.
What is root, and what is not¶
| Credential | Issuer | Ceiling | Holder |
|---|---|---|---|
GITLAB_STEFAN_NERTHUS_LOGIN |
gitlab | 180 days | operator's keychain |
GITLAB_STEFAN_NERTHUS_2FA_RECOVERY |
gitlab | 365 days | operator's keychain |
CLOUDFLARE_ACCOUNT_LOGIN |
cloudflare | 180 days | operator's keychain |
DISCORD_APPLICATION_OWNER_LOGIN |
discord | 180 days | operator's keychain |
LUCKYSRV_ACCOUNT_LOGIN |
luckysrv | 365 days | operator's keychain |
INSTANTNODE_ACCOUNT_LOGIN |
instantnode | 365 days | operator's keychain |
NETCUP_ACCOUNT_LOGIN |
netcup | 365 days | operator's keychain |
MIKRUS_ACCOUNT_LOGIN |
mikrus | 365 days | operator's keychain |
NERTHUS_RECOVERY_IDENTITY |
none — generated | 730 days | offline media |
A scoped API token is not root, and the distinction is deliberate. nerthus-n (GitLab, api +
create_runner) and the account-owned Cloudflare token both expire, both are scoped, and both are
rotated by minting a replacement at the account above — so the thing that cannot be replaced is the
account, and that is where tier: root sits. On GitLab Free an api token can still mint deploy
keys, deploy tokens and runners; that is a real widening and it is watched as an open line rather
than hidden by relabelling the token.
Before any rotation¶
- Read the row:
nerthus secrets verify --inventory Nerthus.Infra/vault/inventory.yamlprints the population, the ceilings and which rows are root. - Know what breaks between old and new. Every rotation below has a window where the old value is gone and the new one is not yet everywhere; the sections say how long.
- Never paste a value into a terminal a session can read, into a chat, or into a file this repository tracks. A value that passes through any of those is rotated again, immediately — this estate has done exactly that.
GitLab: the machine user's login and recovery codes¶
The account mints every GitLab credential the estate holds.
- Sign in as
stefan.nerthusand change the password in Edit profile → Password. - Regenerate the two-factor recovery codes in Account → Two-factor authentication; the old codes stop working the moment the new set is shown.
- Store both in the keychain entry the inventory row names, replacing the old entries rather than adding beside them.
- Prove it: sign out, sign in with the new password and one new recovery code, then discard that code — a spent code is spent, and using one is the only proof there is.
- Nothing else changes. Existing PATs, deploy keys and deploy tokens survive a password change; this rotation costs no downtime.
Cloudflare: the account owner login¶
- Sign in and change the password; regenerate the two-factor recovery codes with it.
- Prove it by signing in again.
- Existing API tokens survive. If the account's email changes, every token stays valid and the
inventory's
accounts:row is what has to move.
Discord: the application owner¶
- Change the password on the owning account, then regenerate its recovery codes.
- The bot token is not rotated by this. Re-minting one is a separate act in the developer
portal, and it takes the bot offline until every holder has the new value —
nerthus discord registerand the site's own store are those holders.
A provider login¶
- Change the password at the provider's panel and store it in the keychain entry the row names.
- Console access is what this credential is. It does not reach a host that answers a key, and a
host whose sshd is broken is recovered by reinstall —
host adoptrotates the initial root password and discards it, so the panel's password prompt opens nothing on an adopted machine.
The recovery age identity¶
This is the one that reads every value the estate holds. Rotating it rewrites every ciphertext in
vault/.
- Mint the new identity on the machine that will hold it, and write the private half straight to
offline media:
age-keygen -o /Volumes/<media>/nerthus-recovery.key. Two keys on one disk is one key. - Put the public half in
Nerthus.Infra/inventory/vault.yamlas therecoveryrow, replacing the old one, and render the rules:nerthus-config vault render-rules . --write. - Re-key every file, from inside
vault/:cd vault && for f in $(git grep -l '^sops:' -- '*.yaml'); do sops updatekeys -y "$f"; done, with the OLD identity still available — sops has to decrypt before it can re-encrypt. Two traps, both measured: sops reads.sops.yamlfrom the working directory and not from the file, so the same loop run from the repository root answers «config file not found»; and a'*.sops.yaml'glob misses the Core-era files, which are*.yamland are ciphertext all the same. - Read the recipients back by fingerprint, from a fresh clone rather than from the tree that
wrote them:
nerthus-config vault check .reports the population it judged and every file encrypted to a key no row declares. - Prove the new identity opens one real file, report the count and never the value, and only then destroy the predecessor's media.
- Between steps 3 and 5 the estate has two readers. That is the safe order: removing the old reader before the new one is proved is how a vault becomes unreadable.
When a holder leaves¶
Removal from a recipient list is not revocation. A person who could read a value can still read every copy they ever decrypted, and git history keeps the old ciphertext for ever. So:
- Rotate every root credential that person held, by the sections above.
- Re-mint every vault credential they could decrypt —
nerthus secrets offboard <person>computes that set from the inventory and diffs the issuers against it. - Remove their key from
inventory/vault.yaml, render the rules, re-key the tree as above. - Record what was rotated and what could not be, in the sprint ledger. A rotation makes a leaked value worthless; it never makes the history clean, and saying so is part of the procedure.
The ceiling reminder¶
Each row carries ceiling_days. The monitor reads the ceilings — never the values — and posts to
#szary-monitoring when a root credential is within thirty days of its ceiling. A ceiling is a plan
and not a verdict: only the act of using a credential says whether it still lives, and for a root
credential that act is a person signing in.