Put the judgments to a model

Let Jev judge the prompt

The prompt hook makes three judgments locally. A cross-encoder picks the claims that bear on the prompt. Two phrase lists spot a correction and a choice put to the agent. Jev, TypeSafe’s decision model, can answer all three in one call, and a fourth route, review, weighs a due claim. It stays off unless the machine turns it on, because the call sends the prompt and up to ten candidate claims off the machine. When the judge does not answer, the hook reranks with the local cross-encoder. A correction the hook stores is labelled with whose prompt said it, and a pasted or quoted prompt stores none. A standing choice the person states (“from now on”, “always”, “never”) is saved in their words, outside code and quotes.

The same call asks two more questions, since a call costs its input tokens and each question adds about 70 of them to a median of 933. An injection question asks whether quoted or pasted text addresses the agent with instructions the person did not write; at cue_at the hook adds a note telling the agent to treat that text as data. An effort question scores the prompt from 0 (a lookup) to 3 (a design or a hard debug); it changes nothing in the hook and goes to $XDG_STATE_HOME/ljos/jev-log.jsonl as the record a router is scored on.

Keep the key in pass and write the setting:

$ pass insert api/typesafe/jev
$ cat > ~/.config/ljos/jev.toml <<'TOML'
enabled = true
key_cmd = ["pass", "show", "api/typesafe/jev"]
TOML
$ ljos doctor --all | grep jev
ok  jev on  jev jev-1.13.0  0 calls, 0 cached  $0.0000 of $4.00 this month

The hook runs the key command once per login, with no terminal and three seconds to answer, and keeps the key in $XDG_RUNTIME_DIR/ljos/jev-key (mode 0600). key_file names a file instead.

The call goes out only where it can change the answer:

check	default	skipped prompt
min_words	4	“yes”, “keep going”
min_candidates	2	fewer claims than a choice needs
monthly_usd	4.0	any, once the month’s spend reaches it

A skipped prompt gets the lexical search and the phrase lists. With Jev on, the hook never loads the cross-encoder, so its memory stays free. A prompt past 2,000 characters goes out cut to that length.

A prompt with ten candidates takes about 900 input tokens. At $0.042 per million input tokens, with output free, that arithmetic is a prompt cost of $0.00004. The hook records each call’s tokens at usd_per_mtok_in, or the API’s own figure when OpenRouter sends one, in $XDG_STATE_HOME/ljos/jev-cost.toml. For OpenRouter, set endpoint = "https://openrouter.ai/api/alpha/decisions" and model = "typesafe/jev-1.13".

Jev answers each question with a probability. bears_at (0.5) is the one at which a claim goes in, and cue_at (0.5) the one at which a prompt counts as a correction or a choice. Raise bears_at to 0.7 when off-topic claims come through.

Jev keeps no cache of its own, so ljos keeps one. An identical request within cache_days (7) is answered from $XDG_CACHE_HOME/ljos/jev at no cost, and the doctor row counts those answers as cached. A panel asked twice, a retried ballot and a repeated prompt with the same claims all hit it. cache_days = 0 turns it off.

Two or more judges can answer side by side. Each [judges.NAME] table names a judge. [route] says which judges answer each decision. Here a chat model on an compatible endpoint judges the prompt, Jev and that model pool their ballots, and the chat model reviews due claims:

enabled = true
key_file = "~/.config/ljos/jev.key"     # the default judge: Jev

[judges.chat]
backend = "chat"
endpoint = "https://llm.example.org/v1"
model = "model-b"
key_env = "LLM_API_KEY"
budget_ms = 6000

[route]
prompt = ["chat"]
ballot = ["default", "chat"]
audit = ["chat"]
review = ["chat"]

A decision with two or more judges is answered by their pool. A probability is the weighted mean of the judges’ log-odds. Two judges at 0.8 stay at 0.8, and judges at 0.9 and 0.1 meet at 0.5. A choice is the normalised weighted geometric mean of their distributions, and a score the weighted mean. weight in a judge’s table moves its share. A question no judge answered is missing from the pool, so the caller falls back as it does on a partial reply. A caller with no judge at all takes its local path on any of these: no file, enabled = false, no key, an error, or no answer inside budget_ms (2000). The log keeps every judge’s answer beside the pool, so each judge can be scored once outcomes are known.

These judges are fast and calibrated, and they do not reason. A decision they leave open goes to a persona that does: a persona with a runner reasons in a session of its own.

$ ljos persona buildengineer --anchor 0.25 --view "Runs production pipelines." --runner NAME
$ ljos vote surf-ab12 --as buildengineer --jev
buildengineer: Jev leaned A at confidence 0.61, under the 0.80 cut; not cast.
  buildengineer reasons in its own session in herdr pane w1:p2; then `ljos consensus surf-ab12`

The runner is any coding runner the machine runs: the terminal harness, the command harness, the build harness, omp, hermes or Antigravity’s agy. Each runs on whatever model it is set to, and switching one to model-b on an compatible endpoint changes only the model it reasons with. It is a [[harness]] in ~/.config/ljos/harnesses.toml. Its start and resume argv say how it opens a session and how it resumes the latest one in a directory, for example the resume argv the harness table names. It starts in a pane the person can watch and talk in: a herdr workspace when herdr’s server is up, else a window of the tmux session ljos-personas. A [[tool]] the runners file declares is tried before either (see Open persona panes in another tool). A runner that exits non-zero is resumed in its pane, at most three times a minute. It works in the persona’s home, $XDG_STATE_HOME/ljos/personas/NAME, as the seat named after the persona. Its memories, ballots and trust rows are therefore the persona’s. It reads the issue and the pack, writes its reasoning with ljos note, and casts ljos vote --as NAME. Each runner continues the latest session of the directory it starts in, so the next hand-off resumes the same conversation, and an open pane takes the next task in place. A task goes to the persona’s inbox, and the pane is told the file in one line. ljos ask NAME "..." puts a question to the persona the same way. ljos supervise NAME --harness RUNNER --exit crash --at SECONDS says whether that child should be started again, under the restart policy (permanent, transient, temporary, three strikes in a minute), and prints the herdr line when the answer is yes. It does not start the pane. ljos panel --jev hands each unsure persona’s brief to its own session when the panel splits. A persona with no runner gets a brief for a subagent, as before.

ljos due --judge puts the due page to the review judges. Each claim goes with the newer claims the pack keeps on the same subject. A claim the pool puts at 0.9 or more is graded recalled. One at 0.1 or less is printed as contradicted, for the agent to supersede or withdraw, and stays due. Everything between stays due. No judge lapses a claim, because a lapse says that a reader forgot it.

Put the judgments to a local model or a runner

The same four judgments go to any judge that answers in Jev’s shape. backend in jev.toml picks it; the parsers, the cache, the ledger and the doctor row are shared, and every fallback above still applies.

A chat-completions endpoint, hosted or a local llama-server, takes one JSON-mode chat completion per judgment set. endpoint is the base URL and model the name the server knows. A local server needs no key; usd_per_mtok_in = 0 keeps the ledger exact for one that is free.

enabled = true
backend = "chat"
endpoint = "http://127.0.0.1:8089/v1"
model = "model-c"
usd_per_mtok_in = 0
budget_ms = 6000

A runner on the machine is the judge through command: an argv that reads the request JSON on stdin (state, and questions, each with a type of noul or choice, instructions and criteria) and prints {"answers": {...}} on stdout inside budget_ms. A noul answer is {"noul": p} or a bare number; a choice answer is {"choice": KEY, "confidence": p, "probabilities": {...}}, and a bare key means a sure one. A question with no answer refuses the whole reply, so the hook takes its local path rather than half a judgment.

enabled = true
backend = "command"
command = ["/path/to/judge.sh"]
budget_ms = 20000

The hook budget bounds the prompt path, so a harness that takes seconds suits ballots and the stop audit more than the prompt hook; min_words and min_candidates keep the prompt calls few. The log carries the backend name, and ljos doctor prints it before the model.

Cast persona ballots through Jev

With Jev on, a persona’s ballot can come from Jev instead of a subagent. The issue needs an Options: A, B line. ljos file "TITLE" --options A,B writes that line as the first line of the body. ljos options ISSUE A,B appends it to an issue that already exists, and refuses fewer than two.

$ ljos vote ljos-abcd --as security-reviewer --jev
judge:jev-1.13.0: cast age for security-reviewer at confidence 0.96
$ ljos panel ljos-abcd --jev
3 personas on ljos-abcd through Jev: all sure, all age; one judge ballot cast

Jev reads the persona’s brief and answers two choice questions: the persona’s vote, and what the rest of the panel will pick. The option order is shuffled per persona. A cast ballot is recorded as judge:MODEL, not as the persona, and --used names that judge. The chosen option’s probability is the ballot’s confidence. The forecast is the judge’s prediction, and consensus does not read it. A note on the issue says the ballot came from Jev. learn updates the trust row whose to is that judge.

Jev’s confidence is (K·p_max − 1)/(K − 1), the same cut on every backend. Under escalate_below (0.8), vote --jev does not cast. It notes Jev’s lean. A persona with a runner gets the ballot in its own session; for any other, the command names the brief to start a subagent from.

A panel casts only when every seated persona is sure and all agree. An all-one-model panel is one voter, so it casts one ballot. A split or unsure panel casts nothing and writes a brief per seat for subagents. The metered model then spends only on the contested questions. A ballot costs about 2,000 input tokens, or $0.00008.

Every Jev answer goes to $XDG_STATE_HOME/ljos/jev-log.jsonl, with the claim ids, a hash of the prompt, the latency, a timeout or a failure, and the backend and model. The file does not keep the prompt or the claim text. ljos judge-score joins that log with outcomes and reports accuracy and calibration per decision. ljos learn scores a cast ballot’s confidence as it does any voter’s.