urna.build

urna.build writes one .urna file from chunks you have already embedded. It does not embed anything: you pass the text, the spans and the vectors, plus the name and model_hash of the model that produced them. It is in the wheel, so it needs no repo checkout.

urna.build(
    output_path, embedding_model, embedding_dim, chunker_version, model_hash, chunks, *,
    title=None, version=None, created=None, description=None, authors=None, license=None,
    provenance=None, reproducible=False, preset="exact", text_encoding=None, dtype=None,
    mrl_dim=None, with_hnsw=None, with_bm25=None, with_graph=False, graph_top_m=8,
    blob_refs=None, blob_data_paths=None, chunk_blob_spans=None, spaces=None,
    hnsw_m=16, hnsw_ef_construction=400, hnsw_seed=42,
) -> str

29 parameters: the first six are required and may be passed by position, the other 23 are keyword-only. A seventh positional argument raises TypeError: build() takes 6 positional arguments but 7 were given. The return value is output_path.

Required parametersCopy Anchor Link

Keyword parametersCopy Anchor Link

PresetsCopy Anchor Link

A preset sets four defaults. An explicit text_encoding, dtype, with_hnsw or with_bm25 overrides the preset's value. Names are case-sensitive, and an unknown name raises ValueError.

micro is a recipe, not a preset: preset="micro" raises ValueError. Build it with explicit knobs:

urna.build(..., text_encoding="zstd", dtype="int8", mrl_dim=256, with_hnsw=True)

Sizes, recall and when to pick each one are on Build presets and Presets and stored precision.

Chunk dictsCopy Anchor Link

Each item of chunks is a dict with five keys. Extra keys are ignored.

A missing key raises ValueError: chunks[i] missing <key>, and an item that is not a dict raises ValueError: chunks[i] is not a dict.

The file keeps the chunks in the order you pass them. That order is the order of UrnaFile.chunk_ids() and of the chunk graph.

byte_start and byte_end are stored as given, returned as offset_start and offset_end, and hashed into the chunk_id. If you want them to point into a source document, compute them as UTF-8 byte offsets into that document. See Offsets.

Model identityCopy Anchor Link

embedding_model and model_hash tell every reader which model produced the vectors. The CLI uses them to pick a query embedder and to refuse a mismatch. For a file the installed urna ask can answer, embed with the bundled potion table and pass its identity exactly:

from urna.embed_potion import potion_embedder

emb = potion_embedder()
urna.build(path, emb.embedding_model, emb.embedding_dim, "my-chunker/1", emb.model_hash(), chunks)

The CLI picks the potion embedder when embedding_model starts with minishlab/potion, then checks the name, the dimension and the hash. Other models are covered on Embedders.

model_hash accepts uppercase hex, but embedders report lowercase, so an uppercase hash never matches at query time.

Media and named spacesCopy Anchor Link

These parameters attach media blobs and extra embedding spaces. They are excluded from content_hash, so adding them does not change any citation. The concepts are on Media and named spaces.

blob_refs items, all four keys required:

blob_data_paths items: a str path or None, one per blob_refs record, in the same order. Rust reads the files. An unreadable path raises ValueError: blob_data_paths[i] (<path>): <os error>. Peak memory is about twice the size of the media.

chunk_blob_spans items, all three keys required, one per chunk in chunk order:

When the overlay is present, hits and urna cite report these spans instead of the chunk spans.

spaces items:

Spaces are numbered from 1 in list order. More than 15 raises ValueError: at most 15 non-text spaces, got N.

What build checksCopy Anchor Link

urna.build raises before writing when any of these fail:

• preset, text_encoding and dtype are known names.
• mrl_dim is between 1 and embedding_dim, and an int4 build has an effective dimension divisible by 64.
• chunks is a non-empty list of dicts with the five keys.
• Every embedding has length embedding_dim and no NaN or Inf.
• byte_end is not smaller than byte_start.
• embedding_model and chunker_version are not empty, and embedding_dim is greater than 0.
• model_hash has the sha256: prefix and 64 hex digits.
• The blob and space shapes above.

It does not check:

• that embeddings are L2-normalized. The runtime normalizes queries, not stored vectors, so normalize them yourself.
• zero vectors. They are accepted and score 0 against every query.
• duplicate chunks. Use urna.chunk_id to find them first.
• that the spans match the text length.
• the zero placeholder model_hash, or uppercase hex in it.
• the HNSW knob ranges (hnsw_m=0 is accepted).

The messages for each failure are on Errors.

What changes content_hashCopy Anchor Link

content_hash is the first half of every citation, so anything that moves it changes every citation_id in the file. chunk_id depends only on text, source_uri, span and chunker_version, so it stays the same.

Writing and reproducibilityCopy Anchor Link

• The file is written in one call at the end, straight to output_path, with no temporary file. A crash during the write can leave a partial file. Do not rebuild over a file another process has open: write to a new path, then move it into place with os.replace.
• build holds the GIL for the whole build, HNSW included.
• The output is deterministic for the same inputs. With created=None nothing time-dependent is written, so two builds match byte for byte even without reproducible. reproducible=True writes the epoch as created, so its file_hash differs from a build without it. content_hash is the same in both, because created is a manifest-only field.
• with_graph and the HNSW build use hnsw_seed, so they are deterministic too.

ExampleCopy Anchor Link

import urna
from urna.embed_potion import potion_embedder

docs = [
    ("notes/offline.md", "The runtime never opens a socket. Queries are answered from the file."),
    ("notes/citations.md", "Every hit carries a urna:// citation that resolves to the stored text."),
    ("notes/format.md", "One .urna file holds chunks, embeddings, spans and indices."),
]

emb = potion_embedder()
vectors = emb.embed_texts([text for _, text in docs])

chunks = [
    {
        "canonical_text": text,
        "source_uri": uri,
        "byte_start": 0,
        "byte_end": len(text.encode("utf-8")),
        "embedding": vector,
    }
    for (uri, text), vector in zip(docs, vectors, strict=True)
]

urna.build(
    "notes.urna",
    emb.embedding_model,
    emb.embedding_dim,
    "notes/1",
    emb.model_hash(),
    chunks,
    title="notes",
    reproducible=True,
)

The same flow step by step, with the query side, is in Use urna from Python.