Skip to content

The speech layer

Nerthus.Core (until cutover). This page describes the frozen system that runs today and is deleted at cutover. Replaced by: not yet written.

The log pipeline archives a transcript and parses it into lines, each carrying the entity its speaker nick resolves to. That resolution sat inside the sidecar, reachable only through a session that references the transcript, one whole transcript at a time. The speech layer indexes it the other way round, so the corpus answers questions about a person: what they said, where, beside whom, and which nicks in the archive reach nobody at all.

This page owns the model. The routes are specified in Logs & audit.

One channel

The layer reads the Lokalny channel. A private message, a group or clan line, a whisper, and a global shout never enter the index, never reach an aggregate, and never reach a caller. There is no parameter that widens it.

That is a scope decision, not a filter placed late: the drop happens as each parsed line is read, before anything is counted or persisted, so no derived artifact on disk holds a private line either.

A line with no channel marker counts as local. Two properties of the parser make that safe rather than a guess:

  • a line carrying a receiver is stamped Prywatny, on every format the parser reads, so an addressed line can never arrive unlabelled;
  • the game marks every non-local channel explicitly, and a rich record whose command carries no channel prefix is a say in the room.

What is left unlabelled is local speech that a copy-paste dropped the tag from — most of a hand-pasted ChatLog, and every line of a Prose summary. Excluding it would leave the whole prose corpus unreadable to this layer while protecting nothing.

One predicate decides every case above, and every reader of the archive passes its lines through it: the routes below, and the speech tier of the search index.

The consequence worth stating plainly: an addressee is not knowable here. A line's explicit recipient lives on the private channel, so the relation this layer draws is co-presence — two entities speaking in the same location segment of the same transcript — and nothing about who addressed whom.

The index

One pass over nerthus.logs/*.json produces three things:

Part Holds
Lines one pointer per local, speaker-bearing line: transcript, line index, time, nick, resolved entity, segment and its resolved location, plus the sessions and date the transcript inherits
Logs one header per archived transcript: format, template, total and local line counts, sessions, fetch time, content hash
Nicks the speaker roster: each distinct nick with its counts, its resolved entity, and whether the archived verdict still agrees with the lore

Records are ordered by date, then transcript, then line, so every listing is stable without re-sorting.

Resolution runs fresh. The nick is resolved against the live name index at build time rather than read back from the sidecar, which is what lets a newly authored @alias reach transcripts fetched years ago without refetching a single one. The sidecar's own verdict is kept beside the fresh one, and a nick where the two disagree is reported as Drifted in the roster.

Line text stays out. A pointer set is small enough to persist and reload; the corpus text would be the archive stored twice. A query hydrates text by reading only the sidecars its matches live in, which is why filtering on text costs more than filtering on a speaker, a session, or a date.

The index is derived. It lives in the gitignored .nerthus/cache/, is rebuilt whenever the archive or the lore model moves, and is never committed. Losing the file costs one rebuild. The freshness check is a stat pass over the archive plus the model fingerprint, the same cheap check the self-heal makes before each request.

Freshness does not watch the daemon's own code, so a resolution change needs a version bump. The check reads three things — the format version, the model fingerprint and the per-sidecar stamps — and none of them moves when the daemon changes how it resolves a nick. A release that resolves differently and leaves the version alone is served the previous answers on every host that already has an index, under a fingerprint the daemon considers current; worse, Drifted reads 0, because the stored records and the roster were written by one generation and agree with each other. Whoever changes what a record means bumps $script:NerthusSpeechIndexVersion, which discards every older cache file and costs one rebuild per host. POST /logs/index/rebuild forces the same thing on a running daemon.

What it answers

Question Read
what did this entity say GET /entities/{name}/lines
how much, under which nicks, where, when GET /entities/{name}/speech
who talks about them GET /entities/{name}/mentions
where do they speak GET /entities/{name}/locations
who is in the room GET /entities/{name}/relations
everything about them, in order GET /entities/{name}/timeline
every local line in the corpus GET /logs/lines
which nicks reach nobody GET /logs/speakers
one transcript, by archive name GET /logs/archive/{log}

Two of these earn their place beyond convenience.

Mentions are the body-mention evidence the participation graph declares out of scope. The graph draws its edges from file placement and structured tags; a person named in the prose of a session they never appear in a tag of is invisible to it. The mention pass stems each word of a line the way the resolver stems a query, so a Polish declined form (Erasterowi) lands on the heading it names. Mentions are answered from persisted postings — word and stem → line ordinals — cached at .nerthus/cache/mention-index.json and stamped with the archive's identity. Only the process that built the speech index holds them in memory; an adopting process re-reads the file when the stamp matches its archive, since an ordinal points into one concatenation of records and means nothing against another. A host whose postings are stale rebuilds them in the background every server.mention_index_interval_min minutes (default 15), and falls back to a transcript scan only until then.

The speaker roster is a hygiene worklist. An unresolved nick means the corpus speaks a name the lore does not know — a character with no block, or a block missing the @alias that would reach it. It plays the part /entities/integrity plays for the index, with the transcript archive as its source instead. Narration that the parse folds out of the speaker column carries no nick into the index at all, so it reaches neither the roster nor any aggregate over it — the log pipeline owns that fold.

What it does not do

  • It draws no conclusion about who addressed whom, for the reason above.
  • It does not read a transcript that was never fetched. The corpus is the committed archive, so a session whose @Logi has never been pulled contributes nothing, and /logs/status is where that gap is visible.
  • It does not resolve a nick the name index cannot. An unknown nick keeps its raw token, stays queryable under it, and is reported unresolved rather than guessed at — the same fuzz-free discipline the rest of name resolution applies to transcript tokens.

See also