Rust crates

urna ships two library crates on crates.io. urna-format is the v1 container: layout, manifest, sections, encodings, hashes, reader and writer. urna-runtime memory-maps a file and searches it: exact SIMD search, HNSW, BM25, graph and hybrid candidates, always finished by an exact cosine rerank. The urna CLI and the Python wheel are built on these two crates.

Which crate to depend onCopy Anchor Link

[dependencies]
urna-format = "0.5.1"
urna-runtime = "0.5.1"

Neither crate has Cargo features. Both need Rust 1.85 or newer (edition 2024). The urna CLI crate needs 1.88.

Three things are not in the Rust crates:

• An embedder. Queries are &[f32] vectors you produce with the model the file was built with. The offline embedders live in the CLI's Python payload and the wheel.
• Presets. exact, compressed, tiny, nano and hybrid are resolved by the Python bridge; in Rust you set the text encoding, the dtype and the indexes yourself (see below).
• retrieve. Python UrnaFile.retrieve and the CLI ask and retrieve combine a search call with canonical_texts() and chunk_ids().

How the crates meetCopy Anchor Link

urna-runtime exposes urna-format in two places: RuntimeError::Format(UrnaError) wraps every format-level failure, and MmapUrnaFile::blob_refs() returns urna_format::BlobRefRecord values. To attach an index to a file you build the payload with urna_runtime::ann::HnswIndex or urna_runtime::bm25::Bm25Index, serialize it with to_bytes(), and pass the bytes to urna_format::UrnaFileBuilder.

ExampleCopy Anchor Link

Build a three-chunk file, open it and run an exact search. This runs as is against the 0.5.1 crates.

use std::path::Path;

use urna_format::{ChunkInput, Manifest, UrnaFileBuilder};
use urna_runtime::MmapUrnaFile;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Build a three-chunk file with 4-dim unit vectors.
    let manifest = Manifest {
        embedding_model: "demo".into(),
        embedding_dim: 4,
        n_chunks: 3,
        chunker_version: "demo-chunker/1".into(),
        model_hash: format!("sha256:{}", "ab".repeat(32)),
        ..Default::default()
    };
    let mut builder = UrnaFileBuilder::new(manifest);
    for (i, text) in ["alpha", "beta", "gamma"].iter().enumerate() {
        let mut embedding = vec![0.0f32; 4];
        embedding[i] = 1.0;
        builder = builder.add_chunk(ChunkInput {
            canonical_text: text.to_string(),
            source_uri: "notes.txt".into(),
            byte_start: (i * 6) as u64,
            byte_end: (i * 6 + text.len()) as u64,
            embedding,
        });
    }
    builder.write_to_path("demo.urna")?;

    // Open it and run an exact search.
    let file = MmapUrnaFile::open(Path::new("demo.urna"))?;
    let result = file.search(&[0.0, 1.0, 0.0, 0.0], 2)?;
    for hit in &result.hits {
        println!("{:.4} {} {}", hit.score, hit.source_uri, hit.citation_id);
    }
    println!("route={} rerank={}", result.explain.route, result.explain.rerank_source.disclosure());
    Ok(())
}

1.0000 notes.txt urna://sha256:3e9dd79c37c1a1e0e8264a89cfc0df29372250d20bf2ba88bebc45b443950bc6/sha256:2999f54c8525f5e7cedc3c4d16ea7202582734aa25cdc7f33a91c01cadd7926d
0.0000 notes.txt urna://sha256:3e9dd79c37c1a1e0e8264a89cfc0df29372250d20bf2ba88bebc45b443950bc6/sha256:c4a06819ec4c55548dc8e6a8ce45e02377c77785349799f85a9b37303078c271
route=exact rerank=real cosine

To query a corpus someone else built, check the model first and route on the declared index type, the way the CLI does:

use std::path::Path;

use urna_runtime::{MmapUrnaFile, SearchResult};

fn query(path: &Path, model_hash: &str, query: &[f32], k: i32) -> Result<SearchResult, Box<dyn std::error::Error>> {
    let file = MmapUrnaFile::open(path)?;
    // The query vector must come from the model the corpus was built with.
    if file.model_hash() != model_hash {
        return Err(format!("corpus model_hash is {}", file.model_hash()).into());
    }
    let result = match file.declared_index_type() {
        "hnsw" => file.search_ann(query, k, 100)?,
        _ => file.search(query, k)?,
    };
    Ok(result)
}

The Rust crates never compare model hashes on their own, except search_space when you pass expected_model_hash. See The model gate.

urna-formatCopy Anchor Link

Crate rootCopy Anchor Link

The root re-exports what most callers need; the rest is under the public modules bytes, chunk, encoding, error, layout, manifest, reader, sections and writer.

CapabilitiesExt is at urna_format::manifest::CapabilitiesExt and REPRODUCIBLE_CREATED at urna_format::writer::REPRODUCIBLE_CREATED; neither is re-exported at the root.

UrnaFileBuilderCopy Anchor Link

A consuming builder: every method takes self and returns Self. UrnaFileBuilder::new(manifest) starts with provenance {}, reproducible = false, SectionEncoding::Raw and EmbeddingDType::Float32.

ChunkInput has five public fields: canonical_text: String, source_uri: String, byte_start: u64, byte_end: u64, embedding: Vec<f32>. The writer rejects a wrong vector length (DimensionMismatch), NaN or Inf (InvalidEmbeddingValue) and byte_end below byte_start (InvalidInput), and requires the chunk count to equal manifest.n_chunks. It expects L2-normalized vectors and does not check the norm.

Optional payloads are opaque to the builder. It does not decode them, so a malformed index or blob payload builds and fails at MmapUrnaFile::open.

Presets in RustCopy Anchor Link

The Python presets map to these builder calls:

The Python hybrid preset never calls .hybrid(), so its files declare index_type = "hnsw". Build the index payloads like this:

use urna_runtime::ann::{HnswIndex, DEFAULT_EF_CONSTRUCTION, DEFAULT_M};
use urna_runtime::bm25::{Bm25Index, DEFAULT_B, DEFAULT_K1};

// vectors: n * dim f32 values, row-major; texts: the canonical texts in chunk order.
let hnsw = HnswIndex::build(vectors, n, dim, DEFAULT_M, DEFAULT_EF_CONSTRUCTION, 42).to_bytes();
let bm25 = Bm25Index::build(&texts, DEFAULT_K1, DEFAULT_B).to_bytes();
let builder = builder.hnsw_index(hnsw).bm25_index(bm25);

DEFAULT_M is 16 and DEFAULT_EF_CONSTRUCTION 400; the Rust side has no default seed (Python uses 42). BM25 defaults are k1 = 1.5, b = 0.75. Build the HNSW index from float32 vectors before quantization, as the Python bridge does. Same inputs and seed give the same bytes.

UrnaViewCopy Anchor Link

UrnaView::from_bytes(&[u8]) parses and validates a file held in memory, in the order listed on Layout.

urna_format::reader::validate_slab_values(encoding, data, n, dim) runs the NaN and Inf walk on any vector slab.

urna-runtimeCopy Anchor Link

Crate rootCopy Anchor Link

MmapUrnaFileCopy Anchor Link

MmapUrnaFile::open(&Path) memory-maps the file, runs the full UrnaView validation and the NaN walk, decodes chunk ids, spans and every index or table the file carries, and computes file_hash and content_hash. The struct owns the map and has no lifetime parameter; dropping it unmaps the file.

Search functionsCopy Anchor Link

All take k: i32 and have no defaults. Every returned score is a real cosine recomputed by the exact rerank.

Queries are checked in this order: k at least 1 (InvalidK), not empty (EmptyQuery), length equals the dimension (DimensionMismatch), no NaN or Inf (InvalidQueryValue), nonzero norm (ZeroNormQuery). The runtime then L2-normalizes the query. search_space first checks the space name (SpaceNotFound) and, when you pass a hash, the space model_hash (SpaceModelMismatch), then runs the same checks against the space dimension.

search_graph returns the same hits as search: its seeds are the exact top max(ef, k) and the rerank uses the same scores. What differs is route, recall and the candidate counts.

Result typesCopy Anchor Link

SearchResult:

SearchHit:

SearchExplain (Copy) has route, exact_candidates, ann_candidates, bm25_candidates, graph_candidates, fusion_mode ("none" or "rrf"), rerank_source and recall_estimate.

RerankSourceKind says which vectors the rerank read: FullPrecision (disclosure() gives "real cosine", as_str() gives "full_precision") when the rerank read float32 vectors (the stored dtype, or a 0x09 full-precision slab when one is present), else StoredPrecision ("real cosine at stored precision", "stored_precision").

DType is Float32, Float16, Int8 or Int4, with name(). SimdBackend is Scalar, Avx2 or Neon, with name().

SIMDCopy Anchor Link

The runtime picks a kernel once per process: AVX2 when the CPU has both avx2 and fma, NEON on aarch64, scalar otherwise. Set URNA_FORCE_SCALAR to any value other than 0 to force the scalar path. float32, int8 and int4 have AVX2 and NEON kernels. float16 runs scalar on x86_64; on aarch64 it uses NEON only when the crate was compiled with rustc 1.94 or newer. All paths accumulate in f32, and the int4 kernel gives identical results on all three. See Environment variables.

Every error these crates return is on Typed errors.