docsv0.5.1

[build] and [output]

Reference for [build] and [output] in a urna build spec: the engine preset, dtype, graph and mrl_dim of space 0, output modes, provenance and the cache.

[build] sets how the file's main vectors (space 0) and indexes are written. [output] sets which files come out, where they go, how much the manifest records, and where the embed cache lives. Both tables are optional.

[build]
preset = "hybrid"        # the default
with_graph = true

[output]
mode = "single"
dir = "out/handbook"
provenance = "standard"

[build]

The keys are passed to the Python writer urna.build for every output file. They apply to space 0 only; named spaces take their precision from [[models]] space_dtype.

KeyTypeDefaultMeaning
presetstring"hybrid"Storage preset for space 0: exact, compressed, tiny, nano or hybrid. See the table below
dtypestring""Overrides the preset's stored precision for space 0: float32, float16, int8 or int4. "" keeps the preset's
with_graphbooleantrueWrite the chunk-to-chunk graph section: an edge to the next chunk plus up to graph_top_m semantic neighbours per chunk. Note that urna.build called directly defaults to false
graph_top_minteger8Maximum semantic edges per chunk in the graph
graph_spacestring"default"Only "default" is accepted: build.graph_space: only "default" is supported
mrl_diminteger0Cut space 0 to its first mrl_dim components and L2-normalize again before storing. 0 keeps the native dimension. Must be greater than 0 and at most the model's dimension; with int4 it must be a multiple of 64
presetText codecSpace 0 dtypeHNSWBM25
exactrawfloat32nono
compressedzstdfloat16nono
tinyzstdint8yesno
nanozstdint4yesno
hybridzstdfloat32yesyes

See presets and stored precision for what each costs in size and recall, and the build presets reference.

preset, dtype and mrl_dim are not checked when the spec is validated. urna.build rejects a bad value when it writes the file, which is after rows are loaded and every model has embedded them. The build then fails with a Python ValueError and exit 1, for example unknown preset: fast (expected exact|compressed|tiny|nano|hybrid) or mrl_dim must satisfy 0 < mrl_dim <= embedding_dim (256), got 512. Run a small --sample build first when you change them. The vectors stay cached, so the rerun after a fix does not embed again.

preset, dtype and mrl_dim change the stored vectors or the search contract, so they change the file's content_hash and every citation. The graph is not part of the content_hash: with_graph and graph_top_m leave citations unchanged.

hybrid does not search with BM25

hybrid writes a BM25 index but declares index_type = "hnsw". urna ask, urna retrieve and urna search-text route by the declared index type, so they search with HNSW and the exact rerank and never read the BM25 index. Only a direct search_hybrid call from Python uses it. See search paths and Known limits.

Settings the build does not check or record

  • [build] mrl_dim cuts space 0 without checking the model's validated dimension ladder. potion has no ladder, yet mrl_dim = 128 on a potion corpus is accepted. The ladder check applies only to [[models]] dims.
  • [media.jxl_transcode] keep_metadata is read and never used.
  • The media profile name is not written to the manifest. It is recorded in the build lock under resolved_spec.media.profile.

See Known limits.

[output]

KeyTypeDefaultMeaning
modestring"single"single, per-model or both. See output modes
dirpath"out"Output directory, relative to the build's current directory. --out-dir overrides it. Holds the .urna files, the manifest, the lock, <name>.media/, .forge-state/ and .tmp/
provenancestring"standard"How much the manifest records: minimal, standard or full. The .urna files are identical in all three. See provenance
allow_remote_codelist of strings[]Presets allowed to run code from their model repository. Every jina-* and wemm-* preset in the spec must be listed
embed_mediabooleanfalseStore the encoded media inside the .urna as well, so one file serves text, vectors and images. <name>.media/ then stays as a build cache. Needs [media]
cache_dirpath""Root of the shared embed cache. See cache root

Validation errors (exit 2): output.mode: single|per-model|both, output.provenance: minimal|standard|full, and output.embed_media requires a [media] section (there is nothing to inline).

Output modes

ModeFiles written
single<name>.urna with space 0 and every named space
per-model<name>-<preset>.urna per model, each with space 0 plus that model's named spaces. The text = "default" model gets no file of its own when it has image = "none", since that file would repeat space 0 alone
bothThe single file and one file per model, the default model's included

Every file carries the same chunks, the same space 0 and the same media tables, so all of them cite identically. There is one manifest and one lock per build, named after the corpus, whatever the mode.

Provenance

provenance changes only <name>.manifest.json:

LevelSQLItemsPaths
fullThe [source] queryordinal, key, label, media_uri, image_pathAs given
standardEmpty stringSame as fullA home-directory prefix becomes ~, then the path is made relative to the spec's directory when possible
minimalAbsentkey and ordinal onlyBare file names

In standard, the home rewrite happens first, so a path under your home directory stays ~/... even when it lies inside the spec's directory. Relative paths and paths outside your home become relative to the spec's directory. The manifest fields are listed on build artifacts.

Cache root

Embeddings are cached outside the output directory, shared by every spec and output directory on the machine. The root is the first of:

  1. --cache-dir on the command line.
  2. [output] cache_dir.
  3. The URNA_CACHE_DIR environment variable.
  4. ${XDG_CACHE_HOME:-~/.cache}/urna.

A leading ~ expands. The location never enters the build lock, so builds that use different roots can still match. A root that cannot be created stops the build with spec error: output.cache_dir: cache root <path> is not a usable directory (...) (exit 2). The cache layout is on build artifacts.

The files a build writes are described on build artifacts.

On this page