Deposit finetuning samples¶

Nerthus.Core (until cutover). This page describes the frozen system that runs today and is deleted at cutover. Replaced by: none, retired with Core.

You are here because you have data that could train a model over this corpus — a set of exchanges, a batch a model proposed, a harvest out of the lore — and the programme's standing rule is deposit it, do not discard it.

This page is the whole path. The API reference is for somebody reading the store; this is for somebody adding to it.

The one-minute version¶

export NERTHUS_FINETUNE_ROOT=/path/to/repozytorium-probek-dev
cd /path/to/Nerthus.Lang.Annotations

python3 tools/validate_finetune.py            # the gate; exit 0 means it landed
python3 tools/validate_finetune.py --census   # per set, per grade, per origin

Deposit is a git commit through a validator, never an API write. There is deliberately no deposit route: a bad row that reaches the store is worse than a deposit that failed, and the gate is what makes the first impossible. The daemon writes to this store in exactly one place — a human ruling through POST /training/finetune/samples/confirm.

Where things live, and why they are apart¶


the rows	`repozytorium-probek`, a data repository. No `.py` in its history, ever.
the contract	`finetune-schema.json` in Nerthus.Lang.Annotations
the gate	`tools/validate_finetune.py`, beside the contract
the published contract	`schema.json` in the data repository, written by `--publish-schema`

Schema/validator skew is unguarded; schema/data skew is exactly what the validator catches. So the contract travels with the code that enforces it, and the data repository records the schema_version it was written under.

NERTHUS_FINETUNE_ROOT names the checkout and unset is a refusal, never a sibling guess — a run with no data would validate an empty store and print a pass. It is deliberately not NERTHUS_ANNOTATIONS_ROOT: one variable meaning two stores fails silently, and pointing it at the annotation store is refused by name because that store also has a samples/ directory.

Writing rows¶

Use the builders. A hand-built literal can satisfy a schema its own builders would not produce, and then your rows are testing the fixture rather than the contract.

import sys; sys.path.insert(0, "Nerthus.Lang.Annotations/tools")
import finetune_store as Store

Rows = [Store.row(
    Id="55/lore-qa/0001", Set="lore-qa", Grade="proposed",
    Messages=Store.messages(Store.SCHEMA["sets"]["lore-qa"]["system"],
                            "Gdzie leży Kamienica Broka?", "W Nithal."),
    Origin=Store.model("claude-opus-5", "sha256:…", "2026-08-31"),
    Locator=Store.corpus_span("repozytorium-fabularne-prodkopia",
                              "Świat gry/Nithal/Miasto.md",
                              "Świat gry/Nithal/Miasto.md#Kamienica Broka", 145008, 145173),
    Project="55-my-project", Artifact="55-my-project/gen/batch-01.py",
    Deposited="2026-08-31",
    Fields={"answerable": True})]

Store.deposit("lore-qa", Rows)   # validates every row; writes nothing if any fails

deposit() is all-or-nothing: half a project's output is worse than none, because a partial deposit leaves a count nobody can reconcile against the run that produced it.

The four things the gate will refuse you for¶

The grades never blur. confirmed needs a named human acceptance, measured needs a stated threshold, proposed needs the absence of both. A hand-curated set wearing a measured label is the thing this coupling exists to stop.

A model-written row must name its model and its prompt hash. Both, or the row does not land. Without them a later training run cannot be told apart from one that trained on its own output, and the census would publish a proportion nobody could check. This is not a formality — it is the only mechanism.

A pair carries both locators. For a document-pair the assistant turn came from one corpus document and the user turn from another, and each half needs its own corpus, source and content hash. A row that cannot say what it was cut from cannot be checked after the corpus moves.

dedupe is recomputed, never trusted. It is a sha256 over the whitespace-normalised turns; a supplied value that does not match its own messages is a row that would evade the duplicate gate by construction, and two rows carrying one exchange put every count over the set one too high.

Registering a new set¶

A set is registered in finetune-schema.json before a row of it can land, with:

its system prompt — one string for the whole set, so it is not repeated per row and cannot drift between rows;
its document_identity and document_pattern — what a document is for this set. One corpus holds several document families, so the pattern belongs to the set: a session summary's document is a block of Lokacje/*/Sesje.md keyed by its sha256, a lore-qa row's is a section of a world-lore file keyed by its heading. Declaring the pattern on the corpus made every lore-qa row unlandable, which is how this was found;
its known_bias — see below.

State the set's bias in the contract, not in a report¶

Every set carries known_bias, and it is a required part of registering one rather than a courtesy.

A limitation recorded only where its author would look is a limitation nobody downstream ever meets. The session-summary set's bias is that a pair exists only where the transcript was archived — so a fetcher selected the population, and the pairs run about two thirds of the corpus's median length. Anybody computing a recall figure on that set needs that sentence, and putting it in the contract is what makes it travel: into the census, into the export's stamp, and onto the page the consumer reads.

If you do not yet know your set's bias, say that, and say what would have to be measured to know it.

Include the negatives, whatever your set is¶

A set of only positive examples cannot teach a model to decline.

a QA set needs unanswerable questions — otherwise nothing measures confabulation, which is the failure that matters most for a lore assistant;
an agent set needs requests no route serves — otherwise it trains a model to invent endpoints;
a preference set needs denials, which is why a deny ruling keeps the assistant turn: the turn is what was denied, and the row says what it is negative for.

Publish the ratio. A set that is 3 % negatives and does not say so reads as balanced.

State your sampling rule before you write the questions¶

If a model is choosing what to write about, fix the rule first and record it in the row's fields.

A model asked to write questions writes the questions it can answer. The set then comes out easy, the recall figure comes out good, and neither means anything. Choose the spans by a rule that knows nothing about their content — a stride over a sorted list, a filter on length — and write against whatever the rule selected.

Do not cut from a file the daemon generates¶

The contract declares each corpus's generated files and the validator refuses a row whose locator names one. nerthus.entities.md, nerthus.maps.md, nerthus.pu-sessions.md and nerthus.contributors.md are machine-written: a map pass rewrites the entity index, and measured on 2026-08-31 it changed hash twice inside twenty seconds while another host's daemon was running.

A locator into such a file can never keep a true content_hash. Fourteen rows learned this the expensive way — re-cut once, red again before the commit finished, and removed in the end.

They are also the worst source you could pick. An index is metadata, so the questions you get out of it are field lookups. Cut from the hand-written descriptive files: they change when a person changes them, which is a rate you can keep up with.

When the corpus moves under a row anyway¶

It will. tools/recut_locators.py re-locates a row by its document anchor — a heading survives the file being rewritten around it, while offsets do not — and records what the row used to carry in fields.recut: the old hash, the old offsets, the date and the reason.

That is not the same as re-pinning. Overwriting the hash and leaving no trace turns a row cut from one state of the corpus into a row claiming to have been cut from another, and nothing afterwards can tell. A row whose anchor is gone is reported and left drifted, because there is nothing to correct it against and a guess is worse than a red gate.

An ingest should converge, not refuse¶

If rows arrive from somewhere you do not control — an authoring pass, another agent, a colleague's export — make the ingest skip what the store already holds rather than refuse over it.

deposit() is all-or-nothing and refuses duplicates, which is right for one batch and wrong for a re-run: the second attempt fails over the rows that already landed, and whatever is new in the file never lands at all. An ingest that skips-and-reports converges instead. Run it again whenever the source grows.

This is not hypothetical. One ingest here read a file an agent was still writing, took 904 of its eventual 1 030 pairs, and reported the result as final — two seconds before the author finished.

And do not compute a ratio over a population somebody is still producing. Those 126 late pairs were all unanswerable, because that author wrote the answerable ones first and appended the refusals at the end. The census taken mid-write said the set had fallen short of its 30 % refusal target at 27 %. It had not: it was at 30 %, and the number was measuring how far the author had got.

Check content, not only shape¶

The gate checks that a row is shaped like a row. It cannot check that the text is right.

Measured on this store's first harvest: 1 304 session-summary rows passed every schema check, and 100 % of their assistant turns contained the URL of the transcript they were cut from — the gold answer teaching a model to cite its own source. A URL inside a summary is well-formed. Nothing in a schema could have objected.

So when a field is a training target, write a content test beside the gate and populate it from measurement: census what fraction of your corpus carries each kind of furniture before you decide what to strip. tools/test_harvest_cleaner.py is that test for the summary harvest, and every case in it is a defect that actually shipped.

Before you commit¶

python3 tools/validate_finetune.py                 # locators resolved against the corpora
python3 tools/validate_finetune.py --census        # and read what it prints
python3 tools/validate_finetune.py --publish-schema  # only if you changed the contract

Read the census rather than glancing at the exit code. It is where you find out that your set is 90 % one origin, that a document bucket does not reconcile, or that your deposit landed 40 rows short because dedupe refused the duplicates — all of which are true, none of which is an error, and each of which changes what your set is good for.