UrnaFile
Reference for urna.UrnaFile, the read-only handle on a .urna file, with its 13 properties and 11 methods for search, retrieve, blobs and validation.
urna.UrnaFile is a read-only, memory-mapped handle on one .urna file. It has one constructor (open), 13 read-only properties and 11 methods: five search paths, retrieve, and six accessors for ids, blobs, metadata and validation.
Open a file
import urna
db = urna.open("corpus.urna") # or urna.UrnaFile.open("corpus.urna")UrnaFile.open(path: str) -> UrnaFile # staticmethodopen maps the file read-only and validates all of it before it returns: magic (URNA, or the legacy NEST of 0.4.0), version, header checksum, file size, section bounds and alignment, the encoding and checksum of every section, the manifest and its contract, the footer hash, header and manifest agreement, and a walk of the embeddings for NaN and Inf. It then decodes chunk ids, spans, and any HNSW, BM25, graph, blob and space sections. Any failure raises ValueError, so an object you hold has already passed validation.
| Rule | Detail |
|---|---|
| Path type | str only. pathlib.Path raises TypeError; pass str(path) |
| Constructor | urna.UrnaFile(path) raises TypeError: cannot create 'builtins.UrnaFile' instances |
| Closing | there is no close() and no context manager. The mapping is released when the object is garbage collected |
| Pickling | not picklable (TypeError: cannot pickle 'builtins.UrnaFile' object). Open the file again in each process |
| Type name | prints as builtins.UrnaFile |
Because there is no close(), drop every reference to the object before you replace or delete the file. On Windows a mapped file cannot be deleted.
Properties
All properties are read-only.
| Property | Type | Value |
|---|---|---|
embedding_dim | int | stored dimension. For a file built with mrl_dim, the truncated dimension |
n_embeddings | int | number of vectors, equal to the number of chunks |
dtype | str | "float32", "float16", "int8" or "int4" |
simd_backend | str | "scalar", "avx2" or "neon". URNA_FORCE_SCALAR=1 forces "scalar" |
has_ann | bool | the file has an HNSW section |
has_bm25 | bool | the file has a BM25 section |
has_graph | bool | the file has a graph_adjacency (0x0C) section |
has_blobs | bool | the file has a blob_refs (0x14) section |
has_spaces | bool | the file has a space_table (0x15) section |
space_names | list[str] | names of the named spaces, [] when there are none |
model_hash | str | the manifest model_hash |
file_hash | str | sha256: over the whole file |
content_hash | str | sha256: over the decoded canonical sections, the first half of every citation |
What each hash covers is on Citations and hashes.
Query vectors and k
Every search method takes a query vector and k. The same rules apply to all of them.
| Input | Result |
|---|---|
a list or tuple of float or int, or a 1-d numpy array (float32 or float64) | accepted |
a 2-d numpy array, a str, a dict | ValueError: invalid query vector: TypeError: ... |
| an empty list | ValueError: empty query |
| NaN or Inf inside | ValueError: NaN or Inf in query |
| all zeros | ValueError: zero-norm query |
| wrong length | ValueError: dimension mismatch: expected D, got N |
| a vector that is not unit length | accepted. The runtime L2-normalizes the query, so scores do not change |
k of 0 or negative | ValueError: invalid k: 0 |
k larger than the number of chunks | returns every chunk |
k as float or str | TypeError: 'float' object cannot be interpreted as an integer |
k of 2**31 or more | OverflowError |
k as a numpy integer | accepted |
The runtime cannot tell which model embedded a bare vector. Only retrieve and search_space can check the model, and only when you pass expected_model_hash.
Search methods
Every search method returns a list of SearchHit, best score first. score is always the exact cosine between the query and the stored vector, whichever path found the candidate. How the paths differ is on Search paths and the exact rerank.
search
UrnaFile.search(query, k) -> list[SearchHit]Exact cosine over every row. Recall is 1.0. Hits report index_type="exact" and reranked=False.
search_ann
UrnaFile.search_ann(query, k, ef) -> list[SearchHit]| Parameter | Type | Description |
|---|---|---|
query | vector | the query |
k | int | hits to return |
ef | int | HNSW beam width. Required, there is no default |
HNSW collects candidates, then every candidate is rescored with exact cosine. Hits report index_type="hnsw" and reranked=True. Without an HNSW section it runs search() and the hits report index_type="exact".
A small ef does not narrow the beam
The beam is the largest of ef, k and the ef_construction stored in the file. urna.build uses hnsw_ef_construction=400 by default, so any ef below 400 runs at 400 on those files. See Known limits.
search_hybrid
UrnaFile.search_hybrid(query, query_text, k, candidates) -> list[SearchHit]| Parameter | Type | Description |
|---|---|---|
query | vector | the query vector |
query_text | str | the text for the BM25 leg |
k | int | hits to return |
candidates | int | candidates per leg. Required |
The vector leg takes HNSW with a beam of at least candidates and k, or the exact top candidates when the file has no HNSW. The BM25 leg takes its top candidates for query_text. The two lists are united with reciprocal-rank fusion, and every member of the union is rescored with exact cosine. Hits report index_type="hybrid" and reranked=True.
There is no fallback to search(). Without a BM25 section the lexical leg is empty and the hits still say "hybrid". Without HNSW and with candidates smaller than k, you get fewer than k hits, and candidates=0 returns [].
Hybrid ranks by cosine only
The final order is the exact cosine of each candidate. The fusion only decides which chunks enter the union: a BM25 match can add a candidate, but it never ranks above a chunk with a higher cosine. See Known limits.
search_graph
UrnaFile.search_graph(query, k, hops=1, ef=100) -> list[SearchHit]| Parameter | Type | Default | Description |
|---|---|---|---|
query | vector | the query | |
k | int | hits to return | |
hops | int | 1 | breadth-first hops over the chunk graph |
ef | int | 100 | seeds: the exact top max(ef, k) |
Scores every row to pick the seeds, expands hops over the chunk graph (the frontier is capped at 8 times the seeds), then reranks the union with exact cosine. Hits report index_type="graph" and reranked=True. Without a graph section it runs search().
search_space
UrnaFile.search_space(name, query, k, expected_model_hash=None) -> list[SearchHit]| Parameter | Type | Default | Description |
|---|---|---|---|
name | str | the space name, one of space_names. Note that it comes first | |
query | vector | a query embedded with the space's model, with the space's dimension | |
k | int | hits to return | |
expected_model_hash | str or None | None | when set, must equal the space's own model_hash |
Exact cosine over one named space. There is no fallback to the text vectors. An unknown name raises ValueError: embedding space not found: <name>, checked before the hash. A hash that differs raises ValueError: model_hash mismatch in space <name>: the query was embedded with ..., but the space vectors were embedded with .... The query length must equal the space dimension, not embedding_dim.
Hits report index_type="space" and reranked=False. Their embedding_model field still names the text model of the corpus, not the space's model. Spaces are explained on Media and named spaces.
retrieve
UrnaFile.retrieve(query, k, candidates=None, hops=1, ef=100, expected_model_hash=None) -> list[RetrieveHit]| Parameter | Type | Default | Description |
|---|---|---|---|
query | vector | the query vector, already embedded | |
k | int | hits to return | |
candidates | int or None | None | candidate budget. None means max(4 * k, 64) |
hops | int | 1 | used only on the graph route |
ef | int | 100 | beam on the HNSW and graph routes |
expected_model_hash | str or None | None | the model_hash of the embedder that produced query. When set, it must equal the corpus model_hash |
Returns a list of RetrieveHit: the fields of a search hit plus the stored text of the chunk and the precision the rerank read.
retrieve runs three steps.
- The model gate, only when
expected_model_hashis given. A mismatch raises before the query is even parsed. - A search, routed by the manifest
index_type. - The stored canonical text is attached to each hit by
chunk_id. This is the same texturna citereturns, never a reread of the original source.
Manifest index_type | UrnaFile.retrieve runs | urna ask and urna retrieve run |
|---|---|---|
"hnsw" | search_ann(q, k, max(ef, cand)) | search_ann(q, k, cand) |
"hybrid" | search_hybrid(q, "", k, cand) | search_hybrid(q, query_text, k, cand) |
| anything else | search(q, k) | search(q, k) |
cand is candidates, or max(4 * k, 64) when it is None. The code also has a graph route, but the manifest admits only exact, hnsw and hybrid, so no file reaches it and hops has no effect today. Python has no query_text parameter, so on a "hybrid" file the BM25 leg of retrieve gets an empty string and adds nothing. To use BM25 from Python, call search_hybrid yourself.
The model gate is opt-in in Python
The CLI always checks the query model against the corpus. Python checks only when you pass expected_model_hash. Without it, a query from another model returns hits whose cosine is valid and whose meaning is wrong. search, search_ann, search_hybrid and search_graph take no hash at all. Pass the hash on every retrieve call:
hits = db.retrieve(qvec, 5, expected_model_hash=emb.model_hash())The mismatch error reads: model_hash mismatch: the query was embedded with X, but the corpus was built with Y. Results would be cosine-valid but semantically wrong. Pass expected_model_hash=None to bypass this check. The check is a plain string comparison. See The model gate.
The hybrid preset never routes to hybrid
A file built with preset="hybrid" by urna.build or by urna build declares index_type = "hnsw", not "hybrid". retrieve, urna ask, urna retrieve and urna search-text take the HNSW route on it and never read its BM25 section. See Build presets and Known limits.
Other methods
chunk_ids
UrnaFile.chunk_ids() -> list[str]Every chunk_id in file order, which is the order the chunks were given to the builder. Each call returns a new list. The position of a hit in this list is its ordinal: db.chunk_ids().index(hit.chunk_id).
blob_refs
UrnaFile.blob_refs() -> list[dict]The blob_refs (0x14) table, one dict per media blob, in table order. Returns [] when the file has no blobs.
| Key | Type | Value |
|---|---|---|
content_hash | str | sha256:<64 hex> of the blob bytes |
original_uri | str | where the blob came from, or the sidecar that holds it |
byte_len | int | size of the blob |
inlined | bool | the bytes are inside this file (section 0x17) |
blob_bytes
UrnaFile.blob_bytes(index: int) -> bytesThe bytes of one inlined blob, by its position in blob_refs(). When the blob is not inlined, when the index is out of range, or when the file has no blobs, it raises ValueError: blob <i> is not inlined in this file: open the media sidecar named by its blob_refs uri, or rebuild with [output] embed_media. A negative index raises OverflowError.
inspect
UrnaFile.inspect() -> dictThe same document as urna inspect --json, parsed into a dict.
| Key | Content |
|---|---|
magic, version_major, version_minor, format_version, schema_version | header and format versions |
embedding_dim, n_chunks, n_embeddings, file_size | sizes |
manifest | the full manifest: embedding_model, embedding_dim, n_chunks, dtype, metric, score_type, normalize, index_type, rerank_policy, capabilities, chunker_version, model_hash, plus title, version, created, description, authors, license, mrl_dim, full_dim and capabilities_ext when set |
sections | a list of section_id, name, encoding, offset, size, checksum (hex without prefix) |
blobs | the same list blob_refs() returns, or None when the file has no blobs |
spaces | a list of name, space_index, dim, dtype, model_hash, n_vectors, band_bytes, or None |
file_hash, content_hash, simd_backend | as the properties |
The manifest fields are described on Manifest.
validate
UrnaFile.validate() -> boolRe-reads the mapped file and checks it again: every checksum, the footer hash, the manifest contract, the NaN and Inf walk, and the search contract. Returns True, or raises ValueError.
validate() does not check inlined blobs against their blob_refs SHA-256. The Rust urna validate does. For a file with inlined media, run the binary.
Cost and concurrency
retrievedecodes the canonical text of every chunk on each call to attachtextto the hits. On a large corpus that decode dominates the call. If you only need ids and scores, call a search method.chunk_ids()copies the full list on each call. Keep the result if you need it more than once.search_graphscores every row to pick its seeds.- No method releases the GIL, so threads in one process do not search in parallel. For parallel queries, run several processes, each with its own
urna.open.
Hit fields are on SearchHit and RetrieveHit.
Module urna
The urna Python package in 0.5.1, its seven public names, what the pip wheel ships, what needs a repo checkout, and how to install it.
SearchHit and RetrieveHit
Fields of urna.SearchHit and urna.RetrieveHit, the hit objects urna returns from Python, what each field means, and how to turn a hit into a dict.