Grade a host against CIS¶

Every host in this estate owes the CIS Level 1 Server benchmark of its own distribution, and the hosts that carry a site or a runner owe a Level 2 subset on top of it. Two separate instruments measure whether a host actually holds a control, and nerthus-config hardening report prints what each of them said, side by side, control by control.

nerthus-config hardening status nerthus-host-2          # every step's verdict, nothing applied
nerthus-config hardening report nerthus-host-2          # the grade, both instruments, per control
nerthus-config --json hardening report nerthus-host-2   # the same, as data
nerthus-config hardening report --world lab.yaml --family debian --role hardening --arm container

Why two instruments¶

The first instrument is the estate's own: the hardening cookbook's steps. A step that sets net.ipv4.conf.all.rp_filter also declares that it is the control sysctl_net_ipv4_conf_all_rp_filter, so planning the cookbook against a host already says whether that control holds. This is fast, it runs on every converge, and it has one weakness that no amount of care removes: it is a guard inside the thing it guards. The people who wrote the check also wrote the change it checks, from the same reading of the benchmark. If that reading was wrong, both halves are wrong together and the report is confidently green.

The second instrument reads the host from outside that opinion. On Debian, Ubuntu and AlmaLinux it is OpenSCAP evaluating the CIS profile of the SCAP Security Guide; on Alpine, which has no CIS benchmark at all, it is Lynis. Neither knows what our cookbook intended. It runs nightly, from a cron drop-in the cookbook writes, and leaves its answer in /var/lib/nerthus/scan/.

Two instruments that always agreed would be one instrument. The value is in the rows where they do not.

The six grades¶

not-provable-here has two causes and they are the same statement. A machine container shares its host's kernel, so a container reporting ok for a sysctl is reporting the lab's kernel and not its own — the catalogue's provable_in says which controls that covers. And a scanner check that reported notapplicable, notchecked, notselected or informational did not run, so it has said nothing about the host. Neither is ever promoted to a pass. A check that did not happen is not evidence that a host is correct, and a report that blurred the two would be worse than no report.

Reading a disagreement¶

A disagree row is not a failing control and not a passing one. It is the report saying that two checks of one subject cannot both be right, and that a person has to look. The row quotes both readings — the step's verdict with its own reason, and the scanner's rule id with its result — so looking usually takes a minute.

There are exactly three honest ways to close one:

1. Fix the host. The scanner is right and the step was measuring the wrong thing, or measuring the right thing too loosely. Correct the step in the hardening cookbook. This is the common case and it is why the second instrument earns its keep.
2. Correct the catalogue. The two are measuring genuinely different subjects and the correspondence between them was wrong. The control's measured_by row is regenerated from the content by tools/cis-catalogue.py; on Alpine the row may instead be wrong in hardening/scanner/lynis-map.yaml, which is hand-written.
3. Exempt the control. The host cannot hold it without losing the function it exists for. Write the row in hardening/exemptions.yaml with the control, the scope, the reason, a ruling and an expires date at most ninety days out. An exemption with no date, or a date further out than that, is refused by validate; an exemption that has run out is a finding at the next report.

What is not on that list is deciding which instrument you prefer. If a disagreement is closed by believing one side without establishing why the other was wrong, the next disagreement on the same subject will be closed the same way and the second instrument has been switched off by habit.

What the scanner reads¶

The SCAP content is the ComplianceAsCode release the catalogue itself was cut from, fetched once per host from a URL pinned by digest in pins/<family>.yaml. It is deliberately not the distribution's own scap-security-guide package: those packages lag the release they ship on, so far that two of this estate's three benchmarked families package no content for their own version at all, and a scanner reading them would grade each host against the benchmark of the release before its own. One release across the estate also means the scanner's rule ids and the catalogue's control ids are the same strings, so nothing has to be translated between them.

Alpine is the exception twice over. It has no CIS benchmark, so its catalogue is derived — the intersection of the other three, less the subjects Alpine has not — and there is no datastream for OpenSCAP to read. Its second instrument is Lynis, whose test ids are its own namespace, so hardening/scanner/lynis-map.yaml says which control each test answers for. That map is short on purpose: Lynis answers with one compound verdict where CIS asks per control — a single result for every sshd option at once, another for every umask file at once — and hanging twenty controls on one such verdict would turn one suggestion into twenty disagreements, none of which names what is actually wrong. A control whose only Lynis counterpart is a compound verdict is left to the steps alone.

The scan on the host¶

The cookbook installs the scanner, writes /usr/local/lib/nerthus-hardening/scan.sh and schedules it nightly — a /etc/cron.d drop-in on the three families that have one, /etc/periodic/daily on Alpine. The script reduces the tool's output to {control: result} and writes /var/lib/nerthus/scan/<date>.json, keeping a fortnight of them, with latest.json beside them.

hardening report does not open a second connection to fetch it: the cookbook reports the latest scan as the hardening_scan fact, so the reading arrives on the plan the report was already running. A host converged a minute ago has not reached its first nightly run, and the report says so plainly rather than treating an absent scan as an agreeing one — the instrument counts it prints are what each instrument actually read, never what the catalogue says it would read.

To grade against a scan you hold on disk — a lab run, or a file copied off a host — pass it with --scan.

Exit codes¶

report exits 1 on a failing control, on a disagreement, or on an exemption past its date, and 0 otherwise. status exits 0 when every step holds, 2 when a step would change the host, and 1 on a problem or a probe that could not be read. Both change nothing, ever.