Layout
Byte layout of a .urna v1 file: the 128-byte header, 32-byte section table entries, the manifest, 64-byte aligned payloads and the 40-byte footer.
A .urna file is one binary container: a fixed header, a table of sections, a JSON manifest, the section payloads and a fixed footer. This page gives the byte layout of format v1 as urna-format 0.5.1 writes and reads it. All integers are little-endian and unsigned.
File map
[0, 128) header (UrnaHeader, 128 bytes)
[128, 128 + 32 * count) section table (SectionEntry, 32 bytes each), sorted by id
[manifest_offset, +manifest_size) manifest JSON, directly after the table, not aligned
[align64(...), ...) section payloads, each at a 64-byte aligned offset
[file_size - 40, file_size) footer (UrnaFooter, 40 bytes), directly after the last payload- The writer sorts sections by id, places the manifest at
128 + 32 * count, starts the first payload at the manifest end rounded up to 64, and aligns every next payload to 64. - Padding between payloads is zero bytes. It is not part of any section checksum, but the footer hash covers it.
- The footer follows the last payload byte with no alignment.
The 64-byte alignment lets the runtime read embeddings straight from the memory map with SIMD loads.
Worked example
The quickstart corpus (examples/quickstart/out/quickstart.urna, 17942 bytes, 9 sections) lays out like this:
| Range | Content |
|---|---|
[0, 128) | header |
[128, 416) | section table, 9 entries |
[416, 1027) | manifest, 611 bytes |
[1027, 1088) | zero padding |
[1088, 17902) | payloads, first at 1088 (0x01 chunk_ids), last ends at 17902 (0x0C graph_adjacency) |
[17902, 17942) | footer |
Its first 128 bytes:
00000000: 5552 4e41 0100 0000 0000 0000 0001 0000 URNA............
00000010: 0c00 0000 0000 0000 0c00 0000 0000 0000 ................
00000020: 1646 0000 0000 0000 8000 0000 0000 0000 .F..............
00000030: 0900 0000 0000 0000 a001 0000 0000 0000 ................
00000040: 6302 0000 0000 0000 b885 9d96 efff 6caf c.............l.
00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000 ................Read it against the table below: magic URNA, version 1.0, flags 0, embedding_dim 256, n_chunks 12, n_embeddings 12, file_size 17942, table at 128 with 9 entries, manifest at 416 with 611 bytes, checksum b8859d96efff6caf, reserved zeros. Run urna inspect --json on any file to see the same values decoded (see urna inspect).
Header
128 bytes, Rust struct UrnaHeader (#[repr(C)], bytemuck::Pod, so the compiler rejects any padding).
| Offset | Size | Field | Type | Written value | Reader check |
|---|---|---|---|---|---|
| 0 | 4 | magic | [u8; 4] | URNA | URNA or the legacy NEST, else MagicMismatch |
| 4 | 2 | version_major | u16 | 1 | must equal 1 |
| 6 | 2 | version_minor | u16 | 0 | must be 0 or lower, so any minor bump is rejected |
| 8 | 4 | flags | u32 | 0 | not checked (covered by the header checksum) |
| 12 | 4 | embedding_dim | u32 | manifest embedding_dim | must equal the manifest value |
| 16 | 8 | n_chunks | u64 | manifest n_chunks | must equal the manifest value |
| 24 | 8 | n_embeddings | u64 | number of chunks | sets the expected embeddings size |
| 32 | 8 | file_size | u64 | total file bytes | must equal the file length, else FileSizeMismatch |
| 40 | 8 | section_table_offset | u64 | 128 | bounds only |
| 48 | 8 | section_table_count | u64 | number of sections | overflow-checked, bounds |
| 56 | 8 | manifest_offset | u64 | 128 + 32 * count | bounds |
| 64 | 8 | manifest_size | u64 | manifest JSON length | bounds |
| 72 | 8 | header_checksum | [u8; 8] | first 8 bytes of SHA-256 | InvalidHeaderChecksum |
| 80 | 48 | reserved | [u8; 48] | zeros | not checked |
The header checksum is the first 8 bytes of SHA-256 over header bytes [0, 72) followed by [80, 128): the checksum field is cut out of the preimage, not zeroed. See Hashes and citations.
UrnaHeader::new(embedding_dim, n_chunks, n_embeddings, file_size, section_table_offset, section_table_count, manifest_offset, manifest_size) fills magic and version and computes the checksum.
Section table entry
32 bytes per entry, Rust struct SectionEntry.
| Offset | Size | Field | Type | Meaning |
|---|---|---|---|---|
| 0 | 4 | section_id | u32 | id from the section map |
| 4 | 4 | encoding | u32 | wire encoding id, see Encodings |
| 8 | 8 | offset | u64 | absolute file offset, a multiple of 64 |
| 16 | 8 | size | u64 | payload length, padding excluded |
| 24 | 8 | checksum | [u8; 8] | first 8 bytes of SHA-256 over the payload bytes [offset, offset + size) |
The checksum covers the physical (encoded) bytes. A compressed section is checked as stored, before decoding.
Manifest
The manifest is UTF-8 JSON between the section table and the first payload. It is not aligned and has no checksum of its own: only the footer hash covers it. Its fields are on Manifest.
Footer
40 bytes, Rust struct UrnaFooter.
| Offset | Size | Field | Type | Meaning |
|---|---|---|---|---|
| 0 | 8 | footer_size | u64 | always 40 |
| 8 | 32 | file_hash | [u8; 32] | SHA-256 over [0, file_size - 40) |
The footer field hashes everything before the footer. The file_hash that urna validate, inspect and every search hit report is a different value: SHA-256 of the whole file, footer included, the same as sha256sum file.urna.
Constants
All are public in urna_format (pub use layout::*).
| Constant | Value | Meaning |
|---|---|---|
URNA_MAGIC | b"URNA" | magic the writer emits |
LEGACY_MAGIC | b"NEST" | magic of files from 0.4.0 and earlier; read, never written |
URNA_VERSION_MAJOR | 1 | header major version |
URNA_VERSION_MINOR | 0 | header minor version |
URNA_FORMAT_VERSION | 1 | manifest format_version; bumped when the binary container changes |
URNA_SCHEMA_VERSION | 1 | manifest schema_version; bumped when manifest fields or required section semantics change |
URNA_HEADER_SIZE | 128 | header bytes |
URNA_SECTION_ENTRY_SIZE | 32 | bytes per table entry |
URNA_FOOTER_SIZE | 40 | footer bytes |
SECTION_ALIGNMENT | 64 | payload offset alignment |
SECTION_PAYLOAD_PREFIX_SIZE | 12 | common payload prefix: u32 version + u64 count |
SECTION_PAYLOAD_VERSION | 1 | version in that prefix |
Read order
UrnaView::from_bytes checks a file in this order and stops at the first failure:
| Step | Check | Error |
|---|---|---|
| 1 | length is at least 168 bytes (header plus footer) | FileTruncated |
| 2 | magic is URNA or NEST | MagicMismatch |
| 3 | version_major == 1 and version_minor is 0 | UnsupportedVersion |
| 4 | header checksum | InvalidHeaderChecksum |
| 5 | file_size equals the file length | FileSizeMismatch |
| 6 | section table fits in the body | SectionOffsetOutOfBounds or FileTruncated |
| 7 | per entry: encoding legal for the section id | UnsupportedSectionEncoding |
| 8 | per entry: offset is a multiple of 64 | SectionMisaligned |
| 9 | per entry: payload fits in the body | SectionOffsetOutOfBounds |
| 10 | per entry: checksum | SectionChecksumMismatch |
| 11 | manifest range fits in the body | FileTruncated |
| 12 | manifest JSON parses | Json |
| 13 | manifest rules | manifest variants, see Manifest |
| 14 | footer hash over [0, file_size - 40) | FooterHashMismatch |
| 15 | manifest embedding_dim and n_chunks equal the header | ManifestInvalid |
| 16 | the six required sections are present | MissingRequiredSection |
| 17 | embeddings dtype matches the encoding, then the exact size | ManifestInvalid, UnsupportedDType, EmbeddingSizeMismatch |
| 18 | when space_table is present: every listed band present with the exact size | ManifestInvalid, UnsupportedDType, EmbeddingSizeMismatch |
| 19 | search_contract matches the manifest field by field | UnsupportedMetric and the other Unsupported* variants |
UrnaView::validate_embeddings_values() is a separate call that walks the embeddings for NaN and Inf. MmapUrnaFile::open in urna-runtime runs both, then decodes the index sections. Every error is listed on Typed errors.
What the reader does not enforce
- Section ids outside the known map load if their encoding is legal for their class and their checksum matches. There is no id allow-list.
- Duplicate section ids are not rejected.
UrnaView::entryreturns the first match. - Sections that overlap each other, the table or the manifest are not rejected. Only bounds and alignment are checked.
flags,reservedand asection_table_offsetother than 128 are accepted.- The reader does not compare
n_embeddingswithn_chunks. The runtime catches a mismatch at open asSectionCountMismatch.
The next page maps every section id: Sections.
Rust crates
The urna-format and urna-runtime crates: what to depend on, the builder, the reader, MmapUrnaFile, its search functions, result types and a runnable example.
Sections
Every section id of the .urna v1 format, which are required, who writes each one, the encodings a reader accepts and the byte layout of each payload.