.. title:: Reference :description: The command line, the HTTP API, the atom, the review clock, and the environment variables. Command line ============ ================================================================= ========================================================================================================================================================================================================================================================================================= Verb Does ================================================================= ========================================================================================================================================================================================================================================================================================= ``packset ensure`` start the writer if it is down, print ``PACKSET_URL`` ``packset start`` / ``stop`` the writer alone ``packset status [--all | WS]`` up or down; counts by kind, the pin, the index and dense state; ``--all`` over every workspace ``packset port`` / ``url`` / ``which`` the port, the URL, the daemon binary ``packset remember [--workspace WS] TEXT`` one lesson ``packset prefer [--workspace WS] TEXT`` one standing preference ``packset search [--workspace WS] [--as-of TS] [--rerank] QUERY`` ranked claims live now, or those live at ``TS``: score, kind, id, text; ``--rerank`` runs the measured cross-encoder over the top 20; a lone hit reports ``settled`` (no model call: one candidate cannot be reordered) ``packset due [WS]`` claims whose review clock has run out, soonest first ``packset islands [WS]`` the link graph's clusters, largest first: size and first claim ``packset hubs [WS]`` the claims the link graph turns on, highest first: a weighted PageRank over the links ``packset island [--workspace WS] [--fire] CUE`` the memories a cue activates: activation, seed mark, id, text; ``--fire`` wires the top eight together, held within an hour of the same cue's last fire. The verb takes no persona: a persona's weights come through ``as`` on ``GET /v1/activate``, which ``ljos island --as NAME`` sets ``packset sweep [WS]`` the neglect sweep now: reviews left due past twice their interval lapse; a never-recalled forgettable claim missed three times is tombstoned ``forgotten: neglect`` ``packset forget WS`` drop a scratch workspace's atoms whole, no tombstones and no deed; per-run scratch workspaces call this on the way out so the long-lived writer does not keep every run's atoms ``packset fire [--workspace WS] ID ID...`` these claims fired together; their links gain weight, their other links lose a little ``packset grade ID [--lapsed] [WS]`` mark a review recalled, or lapsed ``packset pin [NAME]`` read or set the pinned set; an empty name clears it ``packset accessions [WS]`` deed accessions the live claims cite ``packset atoms [--as-of TS] [WS]`` live-now claims, or those live at ``TS`` ``packset citers ACCESSION [WS]`` the live claims citing one accession ``packset export --into DIR [WS]`` claims to ``DIR/.jsonl``; cited accessions to stdout ================================================================= ========================================================================================================================================================================================================================================================================================= A workspace argument defaults to ``PACKSET_WORKSPACE``, else the git remote of the working directory, else ``default``. Every verb takes ``-h`` or ``--help``: it prints that verb's usage and runs nothing, wherever it appears before ``--``. A flag the verb does not take is refused rather than read as a workspace, an id or a word of the text. ``--`` ends the flags, so ``packset forget -- -odd`` names a workspace that starts with a dash. HTTP ==== The writer listens on ``127.0.0.1`` only. Bodies are JSON. +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Request | Does | +=================================================================================+===========================================================================================================================================================================================================================================================================================================================+ | ``GET /health`` | ``packsetd`` | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET /v1/status?workspace=`` | counts, pin, index and dense state; ``live_cap``, and ``forgotten_by_reason`` (``neglect``, ``the live cap``, or the reason a forget gave) | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET /v1/workspaces`` | names with live counts | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``POST /v1/atoms`` | store one claim; returns the stored record, or the live duplicate | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET /v1/atoms?workspace=&as_of=`` | live claims, or those live at ``as_of`` | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET /v1/atoms/?workspace=`` | one record, whatever its state | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``POST /v1/atoms/update`` | merge fields into one live claim | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``POST /v1/atoms/delete`` | tombstone; ``why`` names the deed that withdrew it | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``POST /v1/grade`` | ``recalled`` true or false; moves ``due_at`` | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET /v1/search?workspace=&q=&limit=&as_of=&rerank=`` | ranked hits, the engine, the stage; a hit the encoder named carries ``dense_rank`` and ``cosine`` | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET /v1/islands?workspace=`` | the link graph's communities, largest first | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET /v1/hubs?workspace=&limit=`` | the claims the link graph turns on, highest first, with their scores | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET /v1/activate?workspace=&q=&limit=&fire=&as=`` | the cluster a cue activates: top hits as seeds, two hops along the links, as ``island``; ``seeds``, ``agreed_seeds`` (two scorers agreed), ``weak`` (no seed both scorers agreed on), ``fired`` and ``held`` (the same cue fired within the hour, so nothing moved), ``as``; ``fire=1`` wires the top eight; ``as=NAME`` | | | reads and writes that persona's weights (``link_weights_by``), the shared ones untouched | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``POST /v1/sweep`` | ``workspace``; the neglect sweep now: ``lapsed``, ``forgotten``, ``forgotten_ids``. The writer runs it on a workspace's first write of a day | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``POST /v1/forget`` | ``workspace``; drop a scratch workspace's atoms whole, no tombstones and no deed: ``workspace``, ``forgotten``. Retraction stays ``POST /v1/atoms/delete``, which names the deed | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``POST /v1/fire`` | ``ids`` that fired together: weights up, missing links made, other links decayed; the same set within an hour of its last fire is held (``held: true``), so two or more seats closing one issue tighten it once | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``POST /v1/consolidate`` | the replacement rule over the live set in written order; ``apply`` false reports the pairs, true closes the earlier of each | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET /v1/accessions?workspace=`` | accessions the live claims cite | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET /v1/citers?workspace=&accession=`` | claims citing one accession | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET/POST /v1/pin`` | the pinned set | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET/POST /v1/set`` | a set's three cards | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET /v1/pack?workspace=`` | the cards and the atoms as one document | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET/POST /v1/user``, ``/v1/memory`` | the seat card, the workspace card | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``POST /v1/attach`` | hold one body for the next turn | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``GET/POST /v1/proposals``, ``POST /v1/proposals/accept``, ``POST /v1/compact`` | proposals a person accepts into claims | +---------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ A refusal is a 4xx with ``error`` as one line: the reason a claim was not stored, quoted back. The atom ======== Schema ``inside.atom/v1``. Fields the writer reads: +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Meaning | +======================================+======================================================================================================================================================================================================================================================================================================================================================================+ | ``kind`` | one of ``voice``, ``habit``, ``cache-pointer``, ``preference``, ``lesson``, ``goal``, ``conclusion``, ``card_line``, ``summary``, ``note``, ``correction``, ``belief``, ``trust``, ``persona``, ``prediction``, ``rule``, ``outcome``, ``message``, ``receipt``, ``group`` | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``level`` | ``explicit`` for a claim a person or seat wrote | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``origin`` | ``user-declared``, ``agent-derived``, ``tool-echo``, ``external`` or ``peer``; a record with none reads as ``user-declared`` and stays unwritten. An ``agent-derived`` write and a hook write are proposals until an accept. Accept of a held write stores that atom and raises its origin to ``user-declared``. The same text held again is the open proposal | | | already on file. Content typed ``authoritative`` on a tool echo or an external atom ranks and promotes at that origin | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``text`` | the claim: at most two sentences of at most twenty-five words for an agent's proposal or a hook's write; a person's write past that is stored, with the reason in ``prose.warning``; mail may be longer, up to 500 characters | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``workspace`` | required | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``set`` | optional; a checked name | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``entities`` | names the claim is about; an entity shaped like an accession must be one; ``seat:`` names the seat that wrote the claim and is not a topic, so it links and matches nothing | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``ts``, ``valid_from``, ``valid_to`` | first written, and the window it was true; ``valid_to`` closes on supersession; no rewrite moves ``ts`` | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``updated`` | the last write: a link rewrite, a review, a closed window, an update or a tombstone; a pack written before 0.15.3 gets it at open, and ``ts`` goes back to ``valid_from`` or ``source.at`` where a rewrite had moved it | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``supersedes`` | the ids a contrary claim closed: an explicit id, a correction sharing an entity, a rewrite (token Jaccard 0.6), or the same opening words with a new object; a rewrite never crosses a different ``from``, ``to``, ``about``, ``agent``, ``issue`` or ``name``, so two voters' rows or two agents' forecasts stay two records | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``due_at``, ``review`` | the review clock: ``stability``, ``difficulty``, ``reps``, ``last``, ``interval_s``, ``neglected`` (sweeps that found it overdue); ``add`` starts none for a trust row, a persona, an outcome or mail, and the sweep skips them even when a grade has started one | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``forgotten`` | on a tombstone the writer made: ``neglect`` (a forgettable claim never recalled and missed three sweeps) or ``the live cap`` (the least retrievable claims past the workspace's cap); a forget by hand carries its own reason. Preferences, rules, readings, goals, trust rows, personas, predictions and outcomes are never forgotten by the writer | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``links`` | peers by entity overlap, at most eight, chosen by relative-neighbourhood pruning; ``fire`` adds peers up to eight | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``link_weights`` | weight per link in (0, 1], absent meaning 0.5; moved by ``fire`` | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``link_weights_by`` | a persona's own weights by name, written by a fire ``as`` that persona and read only through its lens; the nodes and links stay shared | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``from``, ``to``, ``weight`` | a ``trust`` row: one edge of an influence graph, weight in (0, 1]; ``entities`` scope it to domains | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``name``, ``anchor`` | a ``persona``: a voter and how far it moves off its own ballot, in [0, 1]; the text is its view, ``entities`` its domains | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``issue``, ``agent``, ``expect`` | a ``prediction``: one voter's forecast on an issue, an option or an object of option to share, read by Prelec's rule, which names the answer more popular than forecast | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``pattern``, ``verdict`` | a ``rule``: a glob over a command line and ``deny`` or ``ask``; the text is the reason | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ``issue``, ``choice`` | an ``outcome``: the option an issue closed on; read beside the issue's ballots, it says which voters were right, so a settle can discount voters who err together | +--------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ A ``message`` is one letter, a ``receipt`` is one read, and a ``group`` atom is one membership change. A later mail atom does not close an earlier one, and a repeat is kept. Search and recall leave mail out unless ``kind`` names it. ``GET /v1/atoms`` still lists it. Fields the writer does not model round-trip untouched. The review clock ================ A new claim is due after one day. ``grade`` with ``recalled`` grows stability by how overdue the claim was; ``lapsed`` halves it and raises difficulty. The retrievability at ``t`` days since the last review with stability ``S`` is ``R = (1 + 19/81 * t/S)^(-1/2)``, so ``R(S) = 0.9``. ``PACKSET_DECAY=fsrs`` scales a search score by ``R``, floored at 0.25; cards are never scaled. Environment =========== ========================================================== ========================= ================================================================================================================================================================================================================================ Variable Read by Meaning ========================================================== ========================= ================================================================================================================================================================================================================================ ``PACKSET_URL`` clients the writer; ``INSIDE_MEMORY_URL`` is an alias ``PACKSET_PORT`` ``packset``, ``packsetd`` the port; default 8761 ``PACKSET_HOME`` ``packsetd`` the pack home; default ``$XDG_DATA_HOME/packset`` (``~/.local/share/packset``), or the previous default while only that holds the pack (``packset migrate-home`` moves it); two writers on one host want two homes and two ports ``PACKSET_WORKSPACE`` clients the workspace when none is given ``PACKSET_TIMEOUT_MS`` clients how long one request may take, default thirty seconds; a busy writer is asked again after 50, 150 and 450 ms inside it ``PACKSET_WORKERS`` ``packsetd`` requests answered at once, default 4, at most 8; with two or more, searches and writes hold all but one ``PACKSET_QUEUE`` ``packsetd`` connections that may wait to be read, default 128, at most 4096; searches and writes that a worker has read but cannot yet run are bounded the same way; past either a request is answered 503; ``/health`` still answers ``PACKSET_FUSE``, ``PACKSET_DIVERSIFY``, ``PACKSET_DECAY`` writer the panel ``PACKSET_LIVE_CAP`` writer live claims a workspace keeps before the write that crosses it forgets the least retrievable forgettable ones; default twenty thousand, ``off`` for none; a write that forgot says ``forgot: N`` ``PACKSET_TRACE_WRITES`` writer a file that receives one line per write with the time each stage took ``PACKSET_EMBED_QUERY_WORKERS`` writer query encoders kept side by side, default 1; each loads one model into memory; a waiter takes whichever slot frees first; a batch spreads over the free slots; writes yield a slot to a waiting query ``PACKSET_STEM`` writer ``off`` turns lexical stemming off ``PACKSET_EMBED`` writer the encoder binary; else beside the writer, else on ``PATH`` ``PACKSET_EMBED_CACHE`` encoder where models live; default ``$XDG_CACHE_HOME/packset/embed`` ``PACKSET_EMBED_MODEL_PATH`` encoder hub layout of the named model (``onnx/model.onnx`` and the tokenizer files); a present directory is used as-is ``ORT_LIB_PATH``, ``ORT_LIB_LOCATION`` build directory of a source or system build of the Open Neural Network Exchange (ONNX) Runtime; ``ORT_PREFER_DYNAMIC_LINK=1`` links the shared library; ``ORT_DYLIB_PATH`` is that library for ``--features load-dynamic`` ``HF_HUB_OFFLINE`` encoder ``1`` refuses the model fetch; a seeded directory still loads ``LJOS_OFFLINE`` encoder ``1`` refuses every model fetch, rerank and sparse included; a cached or seeded model still loads ``PACKSET_RERANK``, ``PACKSET_RERANK_MODEL`` writer, encoder the second stage on every search, and its model ========================================================== ========================= ================================================================================================================================================================================================================================ Crates ====== ================== ================================================================================================================================================================================================================================= Crate Carries ================== ================================================================================================================================================================================================================================= ``packset-core`` the atom, the prose limit, BM25+, the scan, the panel, decay, review clock ``packset-daemon`` ``packsetd``: the Lightning Memory-Mapped Database (LMDB) store, HTTP, projections, the encoder child; a library in the workspace whose binary the ``packset`` crate builds, unpublished since 0.10.0 (crates.io stops at 0.9.21) ``packset-client`` the HTTP client every tool uses ``packset-cli`` the ``packset`` crate on crates.io: ``packset``, ``packsetd``, ``packset-mcp`` ``packset-mcp`` the read-only Model Context Protocol (MCP) surface, built by ``packset`` from its source tree ``packset-embed`` dense, late, sparse and cross-encoder models as one child process ================== =================================================================================================================================================================================================================================