Check the quota¶

Nerthus.Core (until cutover). This page describes the frozen system that runs today and is deleted at cutover. Replaced by: not yet written.

Read the fleet's free-tier usage once a month and act if it is trending toward a cap. Everything fits inside free tiers because the continuous work was kept out of CI; Fleet owns that budget doctrine.

The allowances¶

Resource	Free allowance	What the fleet uses
GitLab CI minutes	400 / month	~3 min per fleet merge, plus the lore repo's own pipelines
Worker requests	100,000 / day	96 sweeps/day, plus a handful of manual reads, plus one `nerthus-cors` invocation per API request
Worker cron triggers	5	1 (`/15 * * *` health sweep)
KV reads	100,000 / day	0 in standby (monitor healthy); ~192/day with the sweep active, plus one per `nerthus-cors` request
KV writes	1,000 / day	0 in standby; ~96/day with the sweep active (the heartbeat floor), plus provision runs
Cloudflare tunnels	1,000 / account	one per host
DNS records	1,000 / zone	one per host, plus the apex and services

Every number in this table names the cadence it assumes, and the cadence is */15 * * * *. That sentence is the correction: this page's arithmetic used to assume a fifteen-minute sweep while the trigger said * * * * *, the two documents never met, and the gap went unnoticed for a month. A budget that does not name its cadence is a budget nobody can check. ci/Get-NerthusKvUsage.ps1 reads the cron out of worker/wrangler.toml and prints it beside the measurement, so the assumption and the fact land on adjacent lines.

Ninety-six sweeps a day, and the write count is the same number. Each sweep reads hosts.json and fleet-state.json, so 192 KV reads. The state write is conditional — it fires on a state change and, unchanged, once per STATE_HEARTBEAT_MS (15 minutes) — and at this cadence one cron tick is one heartbeat, so writes and sweeps coincide at ~96. That coincidence is deliberate: it means the write budget can be checked by reading the cron rather than by trusting a conditional to hold.

Measured 2026-08-20: the cap had been exceeded every day for at least a week

The cron was * * * * * — 1,440 sweeps a day — and the conditional write was supposed to hold writes at ~96 regardless. It did not, because hosts[].syncAgeMin is a clock and it was left in the change comparison, so every sweep counted as a change and the conditional wrote unconditionally.

day (UTC)	KV reads	KV writes	against the 1,000/day cap
2026-08-14	2 760	1 370	over
2026-08-15	2 850	1 600	over
2026-08-16	3 040	1 310	over
2026-08-17	2 790	1 360	over
2026-08-18	2 870	1 320	over
2026-08-19	3 230	1 420	over
2026-08-20	2 430	1 120	over, by 21:00 UTC

The visible symptom was not an alert. It was fleet:provision failing on an unrelated push with Cloudflare error 10048 — «your account has reached the free usage limit for this operation for today».

Both halves are fixed: the clock is out of the comparison (fleet-health.js d9c07b6) and the cron is */15 (deployed 2026-08-20). The cost of the second half is a real number and not a cost saving: worst-case detection latency for a host going dark is now fifteen minutes rather than one. What it buys back is three constants in fleet-health.js that were already denominated in fifteen minutes and read wrong at one — the heartbeat floor, downStreak's conversion into lastSeenUpUtc, and the monitor freshness bound documented as "two missed 15-minute sweeps".

2026-08-21 is the first full day at */15, and its numbers are the ones that verify the fix

The cron changed at 20:46 UTC on the 20^th, so that day's total is a mixture of both cadences and proves nothing either way. Cloudflare's analytics also lag — measured, a Worker that had swept twice in thirty-one minutes showed no movement in the read counter — so a short window after a change is not a measurement. Take a full UTC day.

Expect ~192 reads and ~96 writes from the sweep, plus nerthus-cors reads tracking API traffic. A write column materially above ~96 over a full day means the conditional is not holding, and the first thing to suspect is another clock in the change comparison.

Reads are the honest instrument for "did the sweep run". Every sweep reads two keys before deciding whether to write, so reads track sweeps while writes track changes. That is precisely why a clock in the change comparison was invisible in the write column's intent and glaring in its measurement — and why reads, not writes, are what the shadow-week comparison uses to assert the Worker executed.

nerthus-cors is the second KV consumer. It reads read-fanout.json on every non-preflight request, and two more keys when that value exists, so the read column tracks API traffic and not just the sweep. Only the sweep writes.

Which KV column applies. With the self-hosted monitor healthy, the health worker stands down before touching KV, so its steady state is zero KV operations — the quota question disappears structurally, not arithmetically, for the sweep's half of it. The ~192-reads + ~96-writes/day arithmetic is the fallback-active budget: it applies while the monitor is dark, and it is the column that applies today. The nerthus-cors reads are unaffected by the gate and continue in either mode. The two watchers is the model; Fleet owns the gate's mechanics.

The monitor exists now, and the standby column is still not the one to read

The monitor has been sweeping on frog01.mikr.us since 2026-08-20 and publishing on dev-restoration.nerthus.pl. The gate stays unarmed until a seven-day shadow week shows the two watchers agree on the same fleet, because the Worker is the fallback and a fallback that disagrees with the primary is worse than no fallback. Until then both sweep, and the active column applies.

The check (first working day of the month)¶

CI minutes. Group Settings > Usage Quotas. Read the previous month's total. Expect well under 100 minutes across all projects; investigate anything above 250.
Where they went. If the number is high, sort the project list. The usual suspects are a pipeline that lost its rules: and now runs on every branch, and a retry loop on a failing job.
Worker and KV — measure it, do not compute it. From a fleet-contract checkout:
```
pwsh -File ci/Get-NerthusKvUsage.ps1 -Days 30
```
It reads Cloudflare's own kvOperationsAdaptiveGroups — reads and writes per day per namespace — prints the committed cron beside them, and exits 2 naming any day over the write cap and 3 when the meter could not be read at all. 3 is not zero usage, and the two are kept apart deliberately: a job that reports no drift over an unread estate is the failure this whole check exists to catch.

Expect invocations flat at ~96/day. KV usage above the applicable column without a matching sweep count is the stale-deployment signature — run wrangler deployments list and decommission any fleet-health* deployment that is not the current one (each stale duplicate silently burns a full sweep budget per day, and is the most plausible reading of any account-level 50 % alert). 4. The heartbeat. Confirm the daily Raport dobowy summary arrived in Discord every day last month. A missing summary means the sweep itself was down; it is the only alert that fires unconditionally. 5. Log into the monitor box, as a person. The box is deleted after three months without a login and the fleet's eyes are on it:
```
pwsh -File ./Confirm-NerthusRestorationLogin.ps1     # in Nerthus.Infra.Restoration
```
A machine also tries, monthly. Nobody can prove the machine's attempt counted, which is why this step is on your list and not only in a pipeline — Keep the monitor box alive has the whole argument. 6. Check when the automated one last succeeded.
```
pwsh -File ./Invoke-NerthusKeepaliveWatch.ps1
```
stale or blind means the monthly job has stopped running — nothing failed, nothing happened, and that is the failure mode with no red of its own.

When CI minutes run short¶

In order of preference:

Find the regression. Nearly every overrun is a job that stopped being branch-scoped. Compare against the templates: fleet:provision on the default branch and pushes only, package on main and multi-host-infra only.
Wait for the reset. Quotas reset on the 1^st. If the fleet is stable, a few days without provisioning changes nothing; the hosts keep converging, because converging is not a CI activity.
Escape hatch: a self-hosted runner. Stand up a Docker-executor runner on a machine you own; its minutes do not count against the quota. Registered against the it group and left untagged, it serves every project with no .gitlab-ci.yml change — Set up a self-hosted runner is the one-script procedure. The cost is a machine that must stay up and patched, so take it only if the overrun is structural.

Do NOT buy compute minutes as a first response. On this project an overrun is a symptom of a misconfigured pipeline almost every time.

When the KV write cap is hit¶

The sweep writes ~96 state documents a day against a cap of 1,000, so it cannot reach the cap on its own while the comparison is right. Check three things, in this order:

A clock in the change comparison. This is what happened on 2026-08-20 and it is the failure to suspect first, because it is invisible in the design and obvious in the meter: any per-sweep-growing field left in comparableStateDoc makes every sweep "changed" and the conditional write unconditional. Four such fields are dropped by name — updatedUtc, takeover.lastSeenUpUtc, hosts[].avail, hosts[].syncAgeMin — and a fifth added later would do it again.
Extra writers. Stale fleet-health deployments each writing on their own cron.
A cadence or heartbeat change. A cron faster than STATE_HEARTBEAT_MS, or that constant lowered.

While the cap is exceeded (until 00:00 UTC) the damage reaches further than the sweep: every KV write on the account fails, including fleet:provision's, so the roster the Worker reads silently stops tracking fleet/hosts.yaml. Alerting degrades too — the sweep still compares in memory and still posts Discord transitions, but last-known state stops persisting, so some transitions are re-reported. Step 3 above finds all of it.

The monitor is immune to this by construction

Its state lives in files on a box with no write cap. The same defect there would have cost nothing, which is the structural argument for the migration rather than the arithmetic one.

Ceilings worth knowing early¶

A Worker may make 50 subrequests per invocation, so the sweep covers about 45 hosts. Beyond that it must be sharded across cron minutes.
Accounts created after 2026-01-27 are capped at 3 top-level groups. Never plan a new top-level namespace as an escape hatch; everything nests under margonem.
Free namespaces cap membership at 5 users for private top-level groups. The machine user takes a seat, so keep the group public or keep membership at five.