docsv0.5.1

UrnaFile

Reference for urna.UrnaFile, the read-only handle on a .urna file, with its 13 properties and 11 methods for search, retrieve, blobs and validation.

urna.UrnaFile is a read-only, memory-mapped handle on one .urna file. It has one constructor (open), 13 read-only properties and 11 methods: five search paths, retrieve, and six accessors for ids, blobs, metadata and validation.

Open a file

import urna

db = urna.open("corpus.urna")          # or urna.UrnaFile.open("corpus.urna")
UrnaFile.open(path: str) -> UrnaFile   # staticmethod

open maps the file read-only and validates all of it before it returns: magic (URNA, or the legacy NEST of 0.4.0), version, header checksum, file size, section bounds and alignment, the encoding and checksum of every section, the manifest and its contract, the footer hash, header and manifest agreement, and a walk of the embeddings for NaN and Inf. It then decodes chunk ids, spans, and any HNSW, BM25, graph, blob and space sections. Any failure raises ValueError, so an object you hold has already passed validation.

RuleDetail
Path typestr only. pathlib.Path raises TypeError; pass str(path)
Constructorurna.UrnaFile(path) raises TypeError: cannot create 'builtins.UrnaFile' instances
Closingthere is no close() and no context manager. The mapping is released when the object is garbage collected
Picklingnot picklable (TypeError: cannot pickle 'builtins.UrnaFile' object). Open the file again in each process
Type nameprints as builtins.UrnaFile

Because there is no close(), drop every reference to the object before you replace or delete the file. On Windows a mapped file cannot be deleted.

Properties

All properties are read-only.

PropertyTypeValue
embedding_dimintstored dimension. For a file built with mrl_dim, the truncated dimension
n_embeddingsintnumber of vectors, equal to the number of chunks
dtypestr"float32", "float16", "int8" or "int4"
simd_backendstr"scalar", "avx2" or "neon". URNA_FORCE_SCALAR=1 forces "scalar"
has_annboolthe file has an HNSW section
has_bm25boolthe file has a BM25 section
has_graphboolthe file has a graph_adjacency (0x0C) section
has_blobsboolthe file has a blob_refs (0x14) section
has_spacesboolthe file has a space_table (0x15) section
space_nameslist[str]names of the named spaces, [] when there are none
model_hashstrthe manifest model_hash
file_hashstrsha256: over the whole file
content_hashstrsha256: over the decoded canonical sections, the first half of every citation

What each hash covers is on Citations and hashes.

Query vectors and k

Every search method takes a query vector and k. The same rules apply to all of them.

InputResult
a list or tuple of float or int, or a 1-d numpy array (float32 or float64)accepted
a 2-d numpy array, a str, a dictValueError: invalid query vector: TypeError: ...
an empty listValueError: empty query
NaN or Inf insideValueError: NaN or Inf in query
all zerosValueError: zero-norm query
wrong lengthValueError: dimension mismatch: expected D, got N
a vector that is not unit lengthaccepted. The runtime L2-normalizes the query, so scores do not change
k of 0 or negativeValueError: invalid k: 0
k larger than the number of chunksreturns every chunk
k as float or strTypeError: 'float' object cannot be interpreted as an integer
k of 2**31 or moreOverflowError
k as a numpy integeraccepted

The runtime cannot tell which model embedded a bare vector. Only retrieve and search_space can check the model, and only when you pass expected_model_hash.

Search methods

Every search method returns a list of SearchHit, best score first. score is always the exact cosine between the query and the stored vector, whichever path found the candidate. How the paths differ is on Search paths and the exact rerank.

UrnaFile.search(query, k) -> list[SearchHit]

Exact cosine over every row. Recall is 1.0. Hits report index_type="exact" and reranked=False.

search_ann

UrnaFile.search_ann(query, k, ef) -> list[SearchHit]
ParameterTypeDescription
queryvectorthe query
kinthits to return
efintHNSW beam width. Required, there is no default

HNSW collects candidates, then every candidate is rescored with exact cosine. Hits report index_type="hnsw" and reranked=True. Without an HNSW section it runs search() and the hits report index_type="exact".

A small ef does not narrow the beam

The beam is the largest of ef, k and the ef_construction stored in the file. urna.build uses hnsw_ef_construction=400 by default, so any ef below 400 runs at 400 on those files. See Known limits.

search_hybrid

UrnaFile.search_hybrid(query, query_text, k, candidates) -> list[SearchHit]
ParameterTypeDescription
queryvectorthe query vector
query_textstrthe text for the BM25 leg
kinthits to return
candidatesintcandidates per leg. Required

The vector leg takes HNSW with a beam of at least candidates and k, or the exact top candidates when the file has no HNSW. The BM25 leg takes its top candidates for query_text. The two lists are united with reciprocal-rank fusion, and every member of the union is rescored with exact cosine. Hits report index_type="hybrid" and reranked=True.

There is no fallback to search(). Without a BM25 section the lexical leg is empty and the hits still say "hybrid". Without HNSW and with candidates smaller than k, you get fewer than k hits, and candidates=0 returns [].

Hybrid ranks by cosine only

The final order is the exact cosine of each candidate. The fusion only decides which chunks enter the union: a BM25 match can add a candidate, but it never ranks above a chunk with a higher cosine. See Known limits.

search_graph

UrnaFile.search_graph(query, k, hops=1, ef=100) -> list[SearchHit]
ParameterTypeDefaultDescription
queryvectorthe query
kinthits to return
hopsint1breadth-first hops over the chunk graph
efint100seeds: the exact top max(ef, k)

Scores every row to pick the seeds, expands hops over the chunk graph (the frontier is capped at 8 times the seeds), then reranks the union with exact cosine. Hits report index_type="graph" and reranked=True. Without a graph section it runs search().

search_space

UrnaFile.search_space(name, query, k, expected_model_hash=None) -> list[SearchHit]
ParameterTypeDefaultDescription
namestrthe space name, one of space_names. Note that it comes first
queryvectora query embedded with the space's model, with the space's dimension
kinthits to return
expected_model_hashstr or NoneNonewhen set, must equal the space's own model_hash

Exact cosine over one named space. There is no fallback to the text vectors. An unknown name raises ValueError: embedding space not found: <name>, checked before the hash. A hash that differs raises ValueError: model_hash mismatch in space <name>: the query was embedded with ..., but the space vectors were embedded with .... The query length must equal the space dimension, not embedding_dim.

Hits report index_type="space" and reranked=False. Their embedding_model field still names the text model of the corpus, not the space's model. Spaces are explained on Media and named spaces.

retrieve

UrnaFile.retrieve(query, k, candidates=None, hops=1, ef=100, expected_model_hash=None) -> list[RetrieveHit]
ParameterTypeDefaultDescription
queryvectorthe query vector, already embedded
kinthits to return
candidatesint or NoneNonecandidate budget. None means max(4 * k, 64)
hopsint1used only on the graph route
efint100beam on the HNSW and graph routes
expected_model_hashstr or NoneNonethe model_hash of the embedder that produced query. When set, it must equal the corpus model_hash

Returns a list of RetrieveHit: the fields of a search hit plus the stored text of the chunk and the precision the rerank read.

retrieve runs three steps.

  1. The model gate, only when expected_model_hash is given. A mismatch raises before the query is even parsed.
  2. A search, routed by the manifest index_type.
  3. The stored canonical text is attached to each hit by chunk_id. This is the same text urna cite returns, never a reread of the original source.
Manifest index_typeUrnaFile.retrieve runsurna ask and urna retrieve run
"hnsw"search_ann(q, k, max(ef, cand))search_ann(q, k, cand)
"hybrid"search_hybrid(q, "", k, cand)search_hybrid(q, query_text, k, cand)
anything elsesearch(q, k)search(q, k)

cand is candidates, or max(4 * k, 64) when it is None. The code also has a graph route, but the manifest admits only exact, hnsw and hybrid, so no file reaches it and hops has no effect today. Python has no query_text parameter, so on a "hybrid" file the BM25 leg of retrieve gets an empty string and adds nothing. To use BM25 from Python, call search_hybrid yourself.

The model gate is opt-in in Python

The CLI always checks the query model against the corpus. Python checks only when you pass expected_model_hash. Without it, a query from another model returns hits whose cosine is valid and whose meaning is wrong. search, search_ann, search_hybrid and search_graph take no hash at all. Pass the hash on every retrieve call:

hits = db.retrieve(qvec, 5, expected_model_hash=emb.model_hash())

The mismatch error reads: model_hash mismatch: the query was embedded with X, but the corpus was built with Y. Results would be cosine-valid but semantically wrong. Pass expected_model_hash=None to bypass this check. The check is a plain string comparison. See The model gate.

The hybrid preset never routes to hybrid

A file built with preset="hybrid" by urna.build or by urna build declares index_type = "hnsw", not "hybrid". retrieve, urna ask, urna retrieve and urna search-text take the HNSW route on it and never read its BM25 section. See Build presets and Known limits.

Other methods

chunk_ids

UrnaFile.chunk_ids() -> list[str]

Every chunk_id in file order, which is the order the chunks were given to the builder. Each call returns a new list. The position of a hit in this list is its ordinal: db.chunk_ids().index(hit.chunk_id).

blob_refs

UrnaFile.blob_refs() -> list[dict]

The blob_refs (0x14) table, one dict per media blob, in table order. Returns [] when the file has no blobs.

KeyTypeValue
content_hashstrsha256:<64 hex> of the blob bytes
original_uristrwhere the blob came from, or the sidecar that holds it
byte_lenintsize of the blob
inlinedboolthe bytes are inside this file (section 0x17)

blob_bytes

UrnaFile.blob_bytes(index: int) -> bytes

The bytes of one inlined blob, by its position in blob_refs(). When the blob is not inlined, when the index is out of range, or when the file has no blobs, it raises ValueError: blob <i> is not inlined in this file: open the media sidecar named by its blob_refs uri, or rebuild with [output] embed_media. A negative index raises OverflowError.

inspect

UrnaFile.inspect() -> dict

The same document as urna inspect --json, parsed into a dict.

KeyContent
magic, version_major, version_minor, format_version, schema_versionheader and format versions
embedding_dim, n_chunks, n_embeddings, file_sizesizes
manifestthe full manifest: embedding_model, embedding_dim, n_chunks, dtype, metric, score_type, normalize, index_type, rerank_policy, capabilities, chunker_version, model_hash, plus title, version, created, description, authors, license, mrl_dim, full_dim and capabilities_ext when set
sectionsa list of section_id, name, encoding, offset, size, checksum (hex without prefix)
blobsthe same list blob_refs() returns, or None when the file has no blobs
spacesa list of name, space_index, dim, dtype, model_hash, n_vectors, band_bytes, or None
file_hash, content_hash, simd_backendas the properties

The manifest fields are described on Manifest.

validate

UrnaFile.validate() -> bool

Re-reads the mapped file and checks it again: every checksum, the footer hash, the manifest contract, the NaN and Inf walk, and the search contract. Returns True, or raises ValueError.

validate() does not check inlined blobs against their blob_refs SHA-256. The Rust urna validate does. For a file with inlined media, run the binary.

Cost and concurrency

  • retrieve decodes the canonical text of every chunk on each call to attach text to the hits. On a large corpus that decode dominates the call. If you only need ids and scores, call a search method.
  • chunk_ids() copies the full list on each call. Keep the result if you need it more than once.
  • search_graph scores every row to pick its seeds.
  • No method releases the GIL, so threads in one process do not search in parallel. For parallel queries, run several processes, each with its own urna.open.

Hit fields are on SearchHit and RetrieveHit.

On this page