docsv0.5.1

urna retrieve

Query a .urna file with text and get JSON or JSONL of cited spans, each with its exact cosine score, stored text, citation and file hashes.

urna retrieve runs the same offline embedding, model gate and routing as urna ask, and prints the hits as JSON for a program or an agent: one object per hit with the exact cosine score, the stored canonical text, the citation_id and the hashes that identify the file.

Usage

urna retrieve [OPTIONS] <FILE> <QUERY>

Arguments

ArgumentDescription
<FILE>Path to the .urna file.
<QUERY>The query text, as one argument. Quote it in the shell.

Options

OptionDefaultDescription
-k, --k <K>10Number of hits to return.
--format <FORMAT>jsonljsonl: one compact JSON object per line. json: one pretty-printed array.
--embedder <EMBEDDER>routed by the manifest modelPath to a query-embedder script to run instead of the routed one. It must speak the query embedder protocol.
--candidates <CANDIDATES>4*k, at least 64The HNSW beam on an hnsw file, the candidates per path on a hybrid file. Ignored on an exact file.
--model-path <MODEL_PATH>noneLocal model directory, passed to the embedder as --model-path: the potion table directory for a potion corpus, the model snapshot directory for a registry model.
-h, --helpPrint help.

Behavior

retrieve and ask share one code path up to the output:

  1. Open and fully validate the file.
  2. Route the query embedder by the manifest embedding_model: names starting with minishlab/potion run embed_query_potion.py, every other name runs embed_query_model.py. --embedder replaces the choice. --mrl-dim is passed when the manifest records full_dim.
  3. Apply the model gate: model name, dim and vector length, the placeholder model_hash, then model_hash equality. retrieve has no flag to skip it.
  4. Route by the manifest index_type (hnsw, hybrid, else exact) and rerank every candidate with exact cosine.
  5. Attach the stored canonical text of each hit and print.

The interpreter and script lookups are in Query embedder protocol.

The installed binary answers potion corpora only

The installed embedder payload has no embed_query_model.py. On a corpus built with a registry model (wemm, jina, clip, siglip2), an installed binary exits 1 with embedder script not found: python/forge/embed_query_model.py (override with --embedder). Run retrieve from a checkout with the model's dependencies installed. See Open a corpus you downloaded and Known limits.

Preset-built files take the HNSW route

Files built with the hybrid preset declare index_type = "hnsw", so retrieve never runs their BM25 section, and a --candidates value below the file's ef_construction (400 by default) does not change the beam. See Known limits.

Output

stdout carries only JSON, so it pipes cleanly into jq or another program. stderr carries [urna] embedder interpreter: <path> and, on failure, Error: <message>.

With --format jsonl each hit is one line, highest score first. With no hits nothing is printed. With --format json the hits are one pretty-printed array, [] when empty.

Hit fields

Every hit carries these 11 fields, in this order:

FieldTypeMeaning
chunk_idstringsha256: plus 64 hex digits. The content-derived identity of the chunk.
scorenumberThe exact cosine similarity between the query and the chunk, recomputed by the rerank. Not a candidate-generator proxy and not a probability.
score_typestringAlways "cosine".
source_uristringThe source the chunk came from, as the builder recorded it.
offset_startintegerStart of the stored span. Its unit depends on the builder: a UTF-8 byte offset for builder.chunk_text, the row ordinal for urna build corpora (the span is ordinal to ordinal + 1), a blob-relative byte range when the file has a blob span overlay.
offset_endintegerEnd of the stored span, in the same unit.
citation_idstringurna://<content_hash>/<chunk_id>. Resolves with urna cite.
textstringThe stored canonical text of the chunk: the same bytes urna cite prints. Never a reopen of the original source.
file_hashstringsha256: over the whole file: the same digest sha256sum computes.
content_hashstringsha256: over the decoded canonical sections. Stable across text encodings; every citation carries it.
rerank_sourcestring"full_precision" when the rerank read float32 vectors, "stored_precision" when it read float16, int8 or int4. The same value on every hit of one call.

The Python UrnaFile.retrieve returns the same 11 fields as RetrieveHit.

Real output on the quickstart corpus (-k 2 --format jsonl, stdout only):

{"chunk_id":"sha256:b5dfeb09a643f6f0afde3f361316dde3ea503ff86c525d297e6ba79b5781a4be","score":0.5004974007606506,"score_type":"cosine","source_uri":"demo/03-citations.md","offset_start":7,"offset_end":8,"citation_id":"urna://sha256:1147b2560863331b21bd9d60fe6bdd99507dc34e17108444dc38194f8e6f09df/sha256:b5dfeb09a643f6f0afde3f361316dde3ea503ff86c525d297e6ba79b5781a4be","text":"because the citation points at content, two people who build the same logical corpus on two machines get the same citation, and a stored corpus and a compressed one cite identically. resolving a citation returns the exact canonical text and the original byte span it came from, which is what lets an agent quote a source it can prove.","file_hash":"sha256:e4d5f8907faad38c192dc6929e36dbf16db558410b2dae5abb4d65f10f508832","content_hash":"sha256:1147b2560863331b21bd9d60fe6bdd99507dc34e17108444dc38194f8e6f09df","rerank_source":"full_precision"}
{"chunk_id":"sha256:2be5a0f62d1bb7a69556b1a59d15e3a7d7cf9991baa579e83c1ae67cdd657748","score":0.27243560552597046,"score_type":"cosine","source_uri":"demo/03-citations.md","offset_start":8,"offset_end":9,"citation_id":"urna://sha256:1147b2560863331b21bd9d60fe6bdd99507dc34e17108444dc38194f8e6f09df/sha256:2be5a0f62d1bb7a69556b1a59d15e3a7d7cf9991baa579e83c1ae67cdd657748","text":"the returned similarity score is a real cosine value, recomputed by an exact rerank, never an approximate proxy. a result you can cite is a result you can trust.","file_hash":"sha256:e4d5f8907faad38c192dc6929e36dbf16db558410b2dae5abb4d65f10f508832","content_hash":"sha256:1147b2560863331b21bd9d60fe6bdd99507dc34e17108444dc38194f8e6f09df","rerank_source":"full_precision"}

The quickstart corpus was built by urna build, so offset_start and offset_end are row ordinals (7 to 8), not byte offsets.

Exit codes

CodeMeaning
0The search ran, including a run with no hits.
1Any error: the file failed to open or validate, the embedder was not found or failed, the model gate refused, or the search rejected the query. The message is on stderr.
2Usage error: a missing argument, an unknown flag, or a --format value other than jsonl or json.

Examples

Print the score and citation of each hit:

urna retrieve corpus.urna "how do citations work" -k 3 2>/dev/null \
  | jq -r '[.score, .citation_id] | @tsv'
0.5004974007606506	urna://sha256:1147b2560863331b21bd9d60fe6bdd99507dc34e17108444dc38194f8e6f09df/sha256:b5dfeb09a643f6f0afde3f361316dde3ea503ff86c525d297e6ba79b5781a4be
0.27243560552597046	urna://sha256:1147b2560863331b21bd9d60fe6bdd99507dc34e17108444dc38194f8e6f09df/sha256:2be5a0f62d1bb7a69556b1a59d15e3a7d7cf9991baa579e83c1ae67cdd657748
0.25516077876091003	urna://sha256:1147b2560863331b21bd9d60fe6bdd99507dc34e17108444dc38194f8e6f09df/sha256:19f36b3e072d553eb83626bf30db5e8f3b1f729a495f7826999ffeae53848e6e

Take the top hit from the array form:

urna retrieve corpus.urna "can I use this offline" -k 1 --format json 2>/dev/null | jq '.[0].text'

Resolve the first citation back to its stored text:

cid=$(urna retrieve corpus.urna "how do citations work" -k 1 2>/dev/null | jq -r .citation_id)
urna cite corpus.urna "$cid"

For agent integration (a tool definition that calls retrieve, and cite as the verification step), see Give a corpus to an agent.

urna ask

Ask a .urna file a question in plain text and get the stored text of each hit with its urna:// citation, embedded offline and model-gated.

urna build

Reference for urna build, the launcher of the declarative corpus build: flags, how it finds the forge and Python, stages, output and exit codes.

On this page

UsageArgumentsOptionsBehaviorOutputHit fieldsExit codesExamples