What the language layer is for¶

Nerthus.Core (until cutover). This page describes the frozen system that runs today and is deleted at cutover. Replaced by: not yet written.

The corpus is Polish, it is written by hand, and half its proper nouns are invented. Everything else in Nerthus reads it as text: a name is a string, a search is a substring, a speaker is whatever stood before the colon. The language layer is the one component that reads it as language — that Solmyrze and Solmyra are one place, that powiedział is a verb and therefore not somebody's name, that the four words between two commas are a person being spoken to.

It is Nerthus.Lang, the ecosystem's one Python project, and it has two halves that are easy to confuse because they ship in one repository:

Half	What it produces	Who consumes it	State
The annotation pipeline	annotation sidecars under `nerthus.lang/`, plus four corpus files rebuilt from them	seven readers in Nerthus.Core, none of which fails without it	on, as a scheduled CI job on `repozytorium-lingwistyczne`, which is where the tree lives
The model layer (`frames/`)	a frame grammar over governance prose, a `spancat` trained on confirmed samples, and a yardstick that scores one against the other	`GET /model/*` and the Model screen	off, deliberately, and the gate that would open it is a number in a file

frames/ is 18 of the module's 26 Python modules at Nerthus.Lang f7f767d. A reader who takes "the annotation pipeline" as the whole project has the smaller half.

What it can answer¶

That two spellings are one word. lemmas.tsv maps a folded surface to the lemma the corpus most often attested for it — 106 742 rows over 2 207 sources, measured 2026-08-12 at repozytorium-fabularne-dev 49fc214. Search builds its index through that map, so a query for mieście finds miasto.

That a name written in an oblique case is still that name. names.tsv does the same for whole names rather than tokens, because a span carries Fortu Eder where a token carries only Fortu. Name resolution reads it as one stage among several.

That an indexed name is the thing the index says it is. The entity store is read at annotation time and its headings, its @alias rows, the inline [surface](/Postaci/X.md) links and the Lokalizacje: session lines become matcher patterns that run before the statistical component. Measured A/B over 60 documents: a verbatim occurrence of an indexed name is found and typed correctly 59.6 % of the time without it and 92.6 % with it, and type contradictions fall from 1 504 to 6. Matching is case-sensitive, which is the whole collision guard — Karczma is an indexed place and karczma is an inn, and only one of them is claimed.

Where a name sits inside a line. spans records character offsets, which is what lets GET /entities/{name}/mentions highlight the mention rather than merely report the line.

What grammatical case a form stands in. From format 6 the sidecar carries morph[] and the corpus carries morph.tsv: part of speech, case, an attestation count and SGJP's own verdict on the reading. It is not every reading — only the ones with a reader — and Annotation sidecars states the rule and its cost.

Which strings in a speaker column are not speakers. The corpus's own VERB readings, with SGJP's verdict beside each, are enough to tell Zgrzyt powiedział, raczej do siebie from a nick — which is why the hygiene worklist is not full of sentences. Where no morphology has been published the test falls back on the shape of a Polish infinitive, which ends in -ć.

What a governance post asserts. The frame grammar reads "Shalley została nowym Namiestnikiem" as an appoint on the namiestnik ref, and Governance proposals is its output contract.

What it deliberately cannot¶

This is the half a newcomer currently has to reconstruct out of a format reference and a CI page, and every refusal below is a measurement rather than an omission.

Morfeusz is never asked about a proper noun¶

CONTENT_POS includes PROPN and the dictionary is kept away from it on purpose. A dictionary of Polish has no entry for an invented name, and its opinion is usually about a different word: on a sample of the entity index, Morfeusz turned Martel into Martela, Drzewiec into Drzewce and Karka into karko — a surname inflected wrongly, a village renamed, and a nonsense common noun. What stands instead is spaCy's lookup lemma, which is the surface itself for an undeclined name and a guess for a declined one.

So the sidecar's lemma on a persName is a guess, and the page that publishes it says so. Declining invented names is name resolution's job, and it is built for exactly the words a dictionary lacks.

One custom dictionary was refused, and only one¶

Project 33 measured a custom Morfeusz lexicon queried ahead of SGJP and returned NO-GO on two numbers: 0-of-42 coverage of the forms it was built for, and 71:1 shadowing — for every case it answered, it displaced seventy-one SGJP answers that were already right. (Project 33's figures, quoted here and not re-derived by this page; they were taken over the dev clone before this project's window.)

Read the scope precisely, because the refusal is easy to over-read. It rejected one lexicon in one position. It said nothing about reading the morphology SGJP already provides — a reader of SGJP's own output cannot shadow anything, since shadowing is a property of answering first — and nothing about spaCy's morphologizer, which is a different component, is contextual, and is the only channel that assigns case to the invented names the dictionary cannot see at all. 33 also recorded its own ending rules as sound, at 0 cross-entity collisions in 1 889 forms — again 33's figure, not this page's.

The rule that came out of it is the programme's, not just the layer's: classify from corpus evidence, not from a hand-written table put in front of the evidence. The distinction is now load-bearing rather than academic: the sidecar's morph[] array publishes the morphologizer's reading and SGJP's verdict on it as separate fields, so the dictionary can veto a reading the tagger invented without ever answering ahead of it.

The dependency parser is switched off, and the cost of turning it on is now measured¶

pipeline.py loads spaCy with exclude=["parser"]. Lemmas, tags and entity offsets are wanted; parse trees are not.

For most of this layer's life the stated reason was that the parser is "roughly half the runtime", a figure that entered at the initial import with no benchmark anywhere in the repository. There is one now — python -m nerthus_lang benchmark — and the claim did not survive it.

The share is platform-dependent, so there is no single number and this page refuses to pick one. Its share of accounted time is 11.00 % on arm64 Darwin and 15.81 % on the aarch64 runner where the annotation actually runs; on the other of the report's two instruments — re-running the pipeline without it — excluding it saves 11.29 % and 17.95 % respectively. Both instruments are far from half on both platforms, and quoting any one of the four alone is the mistake this split exists to prevent. Measured over a seeded 60-document slice of repozytorium-fabularne-dev 49fc214, three repeats, the runner reading by GitLab job 15863499040. Work on Nerthus.Lang carries both tables and the command that regenerates them.

And runtime turned out not to be the reason. The exclusion now rests on the parser not being output-neutral: on that slice it moves 61 name-bearing spans, and adjudicated against the lore's own entity index it removes 13 spans the index already holds and adds none.

What the exclusion costs was always the better-established half. On 30 transcripts and 2 483 name mentions, 45.8 % of name mentions sit at a subject arc — who is the subject of this sentence, answered directly — and 1.2 % at a vocative arc, a channel for address that needs no model. 35.3 % of predicated sentences carry no subject at all, which is Polish pro-drop and the honest ceiling on any single-sentence rule rather than a parse failure. (These four figures are project 53's survey, taken 2026-08-12 over the dev clone at 49fc214; they are quoted here and were not re-derived by this page. A second measurement of the same three quantities exists and does not agree: pipeline.py at Nerthus.Lang f7f767d gives 44.8 % at a subject arc, 1.90 % at a vocative arc and 46.9 % with no subject, over 1 688 mentions in 30 transcripts rather than 2 483. Neither has been shown to be a correction of the other — the populations differ — and the disagreement is published rather than resolved by picking one.)

The exclusion still stands, and now stands on a measurement rather than on a sentence.

The model layer ships off, and the gate is a subtraction¶

Nothing trains automatically, nothing promotes automatically, and no consumer sees a model proposal until a number says it is worth it. The number is in frames-config.json under engage_gate.surfaces.<surface>, and it is two recalls and their difference:

frames_only_recall — what the rule-based grammar reaches on that surface's eval set.
model_recall — what the same grammar reaches with the model's spans in front of it.
min_recall_gain — what the difference must clear before enabled may be true.

On governance-transitions the committed row is 0.2342 against 0.2658, a gain of 0.0316, against a bar of 0.05 — measured 2026-08-12, event-level, 4-fold by document over the 222 confirmed rows that carry a locator.document, at store commit 057fd64. The model is better and not by enough, and the gate stays shut for a reason somebody can argue with.

That block is the shape every number in this ecosystem should have, and it is worth opening once: it carries its date, its unit, the method naming the store commit and the fold design, the population it was taken over (222 events, 79 documents, 80 posts, one named corpus), the gain, the parts, both controls, and a why. The parts are the part that matters — the aggregate hides that 19 of the 222 rows carry a ref no gazetteer emits and are unreachable by either configuration, and 26 carry an operation no frame emits, where the rules score zero by construction.

Precision falls where recall rises — 0.6047 to 0.3986 on the same run — and the gate does not read precision. It is published beside the gain rather than folded into it.

Every addressee label this corpus can yield is derived, never read¶

The archive format that carries an explicit addressee is RichJson, and this corpus holds zero RichJson transcripts. Receiver, Author and Command are absent from all 201 250 served messages — measured over GET /logs/archive/{log} across all 1 152 archives on 2026-08-12, against a read-only daemon at Nerthus.Core 3979112 reading the lore dev clone tracked at 49fc214. That was project 56's boot and not this page's, which is why the commit differs from the one the reader-table above names; the same absence is stated independently by the daemon's own source comment beside the line serializer, so two instruments agree.

So who was spoken to is always inferred here: from a two-speaker transcript's cast, from turn adjacency, from a vocative case. Anything built on address must publish the inference it used, because there is nothing to check it against.

And two limits on the reading itself¶

The unit is the message the daemon serves, not a line in a file. On this corpus they happen to be the same object — the fragment merger has never merged anything, because it is reachable only from the RichJson path — but that is a measurement and not a licence, and it stops being true the day a RichJson transcript lands.

A transcript read serves local speech only. GET /logs/archive/{log} passes every line through the speech gate before answering: 201 250 messages served out of 209 749 records in the sidecars, so 8 499 line records across 242 of the 1 152 archives are returned by no transcript read. A count taken over the route and a count taken over the files are both right and are not the same number.

Why the annotations are committed rather than computed¶

A host reads nerthus.lang/ off a checkout of the annotation repository, which its sync tick keeps current. It never runs spaCy, never installs Morfeusz, and never pays a model load. The pipeline runs once in CI on a runner sized for it, opens a merge request, and every host gets the result by pulling. A host that has not been given that corpus reads the tree from its lore checkout, and one whose lore checkout has none serves no lemma layer — which every reader degrades to.

This is why degradation, not failure, is the contract: a host whose annotation tree is absent or stale still serves every route, more coarsely. Annotation sidecars lists the seven readers and what each one loses — six of them lose something, and one of the seven has no production caller at all.

Where the labels come from¶

Nothing in this layer learns from anything a person did not rule on. The labelled-sample store grades every row measured, proposed or confirmed, and only confirmed may be trained on — a rule the training job enforces by stamping the artifact with the grade it saw, so an artifact trained on anything else is refused by the engage gate whatever the configuration says.

Two rulings of 2026-09-03 (123-ner-plan/HUMAN.md) narrow that sentence for span layers without lifting it. H123.1: a span layer — the entity NER — may train on measured or silver material (the gazetteer projection, speaker prefixes, links) if and only if its evaluation set is confirmed and cut by document; the gate checks the grade of the eval, and a model whose eval is anything else stays refused. H123.2: a span a narrator wrote — an inline [surface](/Postaci/X.md) link, a Lokalizacje: name, a transcript speaker prefix — is a confirmed row of the entity-span class, accepted_by the commit author with the note authored, not reviewed, on the argument that a person asserting a name in the lore is a stronger act than a person clicking a row. Those rows are the only span-level confirmed material the store holds, and every one says which of the three sources it came from so a consumer can hold one out.

measured is the largest grade by far and it is a log, not supervision: a rule computed it, so a model fitted to it would learn the rule. The store at repozytorium-adnotacji-dev fc46ed1 holds 10 819 rows across 22 classes — 8 691 measured, 1 816 proposed, 312 confirmed (measured 2026-08-12). The confirmed count is the one that bounds everything the model layer can do, and it moves only when a person rules on a row through the Rozstrzyganie screen.