Add data layout design

2026-08-01 16:45:46 +02:00
parent 67b1580eb4
commit 0537b44b05
1 changed files with 87 additions and 0 deletions
@@ -0,0 +1,87 @@
+# Sindri
+
+Asset packer.
+
+## Design
+
+The data layout design is very simple. It consists of data split into chunks
+with the first chunk holding the metadata for the structure and files.
+
+### Spec
+
+```
+                                   METADATA                                    DATA
+|---------------------------------------------------------------------------||------|
+
+[ MAGIC ][ VERSION ][ COMPRESSION ][ SIZE_PER_CHUNK ][ TOTAL_SIZE ][ #FILES ][ DATA ] # chunk 1
+[                                        DATA                                       ] # chunk 2
+[                                        DATA                                       ] # etc..
+[       DATA       ]                                                                  # last chunk
+```
+
+#### Magic string
+
+Magic string to detect if the read file is what we expect it is.
+Represented in 3 bytes.
+
+Value is `0x514D21`, the closest approximation of "Sindri".
+
+### Version
+
+Version of the specification.
+Represented in 2 bytes.
+
+Value is `1` currently.
+
+### Compression
+
+Wether LZ4 compression is enabled/disabled.
+Represented in 1 byte.
+
+Value is configurable, defaulting to `0`.
+
+`0` = off
+`1` = default LZ4 algorithm
+`2` = lowest LZ4-HC algorithm, faster and larger
+..
+`12` = highest LZ4-HC algorithm, slower and smaller
+
+### Size per chunks
+
+The size in bytes of each chunk.
+Represented in 8 bytes.
+
+Value is configurable, defaulting to `2GiB`.
+
+### Total size
+
+The total size in bytes of all chunks combined.
+Represented in 8 bytes.
+
+### Number of files
+
+The total number of files in the pack.
+Represented in 4 bytes.
+
+### A file
+
+Piece of data stored, consisting of 3 parts:
+
+- Path
+- Offset
+- Size
+
+#### Path
+
+Path of the file as if it were on the filesystem.
+Represented in 512 bytes.
+
+#### Offset
+
+The offset to the file, starting from the true 0th byte of the first chunk.
+Represented in 8 bytes.
+
+#### Size
+
+Size of the file.
+Represented in 8 bytes.