Reach a host when the mesh is down¶
The hosts of a site talk to each other over WireGuard, and the site answers the world through one tunnel. When either stops, the ordinary ways in stop with it. This page is the way in that depends on neither.
The door¶
Every host with any role carries the admin role beside it: one named Cloudflare tunnel per host,
<host>-admin.nerthus.pl, behind Cloudflare Access, carrying SSH and nothing else. The connector
dials out, so a machine with no public address, behind somebody else's NAT, with every inbound
port closed, is still reachable — and it is a service rather than a container, because this has to
work when Docker does not.
Opening it¶
That reads the host's admin tunnel row from the declaration, so the hostname is never typed from
memory, and hands the terminal to ssh through cloudflared access ssh. Nothing listens on your
machine: the tunnel is spoken on stdio, so two sessions never collide over a port.
--print shows the command and runs nothing, which is what to paste into an incident note.
You need cloudflared locally — the same binary the cookbook places on a host. Without it the
command refuses and says so rather than failing inside ssh with something obscure.
The first time, and the trap¶
Opening the door starts a Cloudflare Access login in a browser. Two login methods are enabled on
the nerthus-zt team: a Cloudflare account, and a one-time PIN sent by mail.
A blocked user sees exactly what a working one sees
The PIN reaches only an address the Access policy already allows. Somebody who is not on the policy's subject list is shown the identical "A code has been emailed to you" and simply never receives one.
So when the admin door is first reported broken, check the policy's subject list before you check mail delivery. Nothing in the browser distinguishes the two, and the time goes into spam folders and mail logs while the answer is a missing row.
The subject list is the H-P10.3 roster. It is identity data, so it lives in sops rather than in a declaration, and it is edited where the policy is defined rather than by this tool — a configuration tool that could rewrite who may open a door would be a second place access is granted from.
The third door¶
If Access itself is what has failed, the provider's console remains: every host has a console
credential row in the vault (NERTHUS_HOST_<n>_CONSOLE_PASSWORD, and FROG01_SUDO_PASSWORD for the
Mikrus box). That path needs nothing of ours at all, which is the point of keeping it.
What this is not¶
It is not a way in for the estate's own programs. Nothing automated uses this door: the agent pulls over HTTPS, the site is reached through its own tunnel, and jobs run where they are queued. It exists for a person, at the moment the ordinary paths are gone.