Releases
Changelog
Versions follow Semantic Versioning. Before 1.0, a minor version (0.x) may change on-disk or wire formats; docs/COMPATIBILITY.md says what each kind of release may change and how to upgrade.
0.3.0 (2026-11-17)#
Bulk imports that don't pause, and shorter stalls with many writers. Telling the store how many records are coming, from the embedded API, any client library, SQL or one server setting, on a single server or any cluster, cut the slowest 0.01% of writes during a load from 1.1 ms to 23–30 µs. The worst stall with four writers on four CPUs fell from 6.5–8.1 ms to 4.3–4.6 ms (docs/BENCHMARK_REPORT.md section 8.11).
This is a minor release because the Rust API gained a field: EngineConfig literals that list every field without ..Default::default() must add expected_records. Nothing else is incompatible.
Upgrading from 0.2.x: replace the binaries (install.sh / install.ps1 again) and restart. The on-disk formats are unchanged and both protocols only gained requests, so, unlike a usual minor release, a cluster can be upgraded one node at a time and 0.2.x and 0.3.0 nodes can run together. While they do, reserve sizes only 0.3 nodes (a range node skips peers that refuse its share), and a 0.2 server answers reserve with Invalid. Embedders: add expected_records to EngineConfig literals that need it.
Formats and protocols#
- On-disk formats: unchanged from 0.2.0.
- Client protocol: additive.
OP_DOC_ADMINsub-command12reserve (str collection | u64 count). The health report (OP_HEALTH) gainedindex_capacity. - Node-to-node protocol (range clusters): additive.
OP_RANGE_RESERVE(0x2F,u64 count) carries a node's share of areserve. - Rust API:
EngineConfig::expected_records(new field),StorageEngine::reserve_records,DocumentStore::reserve,PrimaryIndex::reserve.
Release checks#
CI on Linux x86_64 and arm64, macOS arm64 and x86_64, and Windows: the test suite, lints, binary sizes and the install scripts. Process chaos, 40 runs on x86_64 and on arm64. The power-loss suite (a crash at every I/O call) also runs with expected_records set. Benchmarks against 0.2.1, LMDB and RocksDB (section 8.11). reserve was checked from each of the eight client libraries against a running server, through SQL, and on a three-node range cluster (every node sized); STORAGE_EXPECTED_DOCUMENTS by starting the server with it. Not repeated: the soaks, full SDK suites and sanitizers, since replication and the unsafe code are unchanged since 0.2.0, and the clients and nodes only gained reserve.
Added#
STORAGE_EXPECTED_DOCUMENTS: about how many documents the database will hold (at most 1,000,000,000). The server sizes its index at start, so every client and ORM gets pause-free imports with no code change. Range nodes take the cluster's count and size for their share.- Reserve before an import, everywhere:
reserve(count)on a collection in all eight client libraries (ReserveAsyncin C#,Reserve(ctx, count)in Go), andSELECT storine_reserve('table', rows)in the SQL gateway, which any ORM can run as raw SQL. The server sizes its index for that many more documents, so the import doesn't pause each time the index doubles. A hint: at most 100,000,000 per call, no data changes. One call can take gigabytes of address space, so it needs the admin role (Unauthorizedfor a write token), andstorine_reserveneeds a SQL superuser and a gateway whose server token is admin. On a range cluster the node that receives it sizes every node for its share: the count times the copies kept (3), over the nodes. index_capacityin the health report: documents the index holds before it next grows.EngineConfig::expected_recordsandStorageEngine::reserve_records(n)size the primary index for a known number of records, so a bulk load doesn't pause each time the index doubles (fill p99.99 1.1 ms → 23–30 µs, level with RocksDB). The memory is what the index reaches at that size anyway, taken up front: 150–300 bytes per record. Default 0: grow as before.- A restart sizes the index from the checkpoint's record count instead of growing it while loading.
Fixed#
- 4–8 ms write stalls with several writers. A writer could rebuild the flush queue's duplicate check under the queue lock while every other writer waited. The background flusher now grows and pre-touches it outside the lock. The remaining 99.99th percentile with four writers on four CPUs (2.2–2.5 ms) comes from the scheduler, with more busy threads than CPUs; with a CPU to spare it is about 70 µs.
- A WAL test could fail on a slow machine before its truncations overlapped the writers (seen once on Windows CI). Test only.
- Release builds retry the compiler install when Linux package mirrors disagree (an arm64 build failed on 2026-11-17 for that alone). CI stops a hung test after 75 minutes instead of at the job's 2-hour limit, and the SQL access-control tests stop themselves after 10.
Known limitations#
- With four writers on four CPUs, p99.99 of memory-first overwrites stays at 2.2–2.5 ms, behind RocksDB without its WAL (0.26–0.30 ms) but ahead of it with one (2.7–2.9 ms). Planned for 0.4.0 (
TASK-164). - A range cluster's
reserveassumes an import spreads over every node, which holds once its ranges have split; a small collection sitting in one range gets less benefit.
0.2.1 (2026-11-16)#
A patch release: clearer command-line messages, and the documentation website. The engine, the protocols and the on-disk formats are those of 0.2.0.
Upgrading from 0.2.0: replace the binaries (install.sh / install.ps1 again) and restart. A cluster can be upgraded one node at a time, and 0.2.0 and 0.2.1 nodes can run together.
Formats and protocols#
- On-disk formats, client protocol, node-to-node protocol: unchanged from 0.2.0.
Release checks#
CI on Linux x86_64 and arm64, macOS arm64 and x86_64, and Windows: the test suite, lints, binary sizes and the install scripts. The release workflow installs the published binaries with the install scripts on each platform and checks their version. The checklist's chaos runs, soaks, SDK suites and sanitizers were not repeated: no code they cover changed since 0.2.0.
Added#
- A website with the whole documentation: guides, every option, each client's API, using an ORM through the SQL gateway, the protocol and the benchmarks, searchable. At https://storine.vercel.app and https://twilight-techy.github.io/storine/, rebuilt from
mainon every push (website/build.py, standard library only).
Fixed#
storage-cli --helpprinted an error ("invalid primary key: unknown CLI command"). It now prints the commands and exits 0,-V/--versionprint the version, and an unknown command is named, with exit status 2.- Configuration errors of
storage-serverandstorage-clino longer start with "invalid primary key:", and refusing a network address without tokens now namesSTORAGE_AUTH_TOKENS. - Documentation:
AsyncMemoryFirstcan lose the last flush interval of writes on a process crash (TECHNICAL.mdsaid nothing was lost); every SDK README says how to install the client from the repository (the C# and Java ones named packages that were never published); the Python README listsnearest, vector indexes and partial reads. - A cluster membership test could fail on a slow machine after a leader change (seen once on Windows CI); it retries on the new leader, as a client would. Test only.
0.2.0 (2026-11-16)#
Three features that existed only in the embedded engine are now in the server: encryption at rest, point-in-time recovery and multi-region placement (docs/TECHNICAL.md sections 7, 8 and 13). Single-group cluster writes are 5–8× faster, checkpoints end under any write load, the web console was redesigned, and install scripts set Storine up in one line.
Upgrading from 0.1.x: back up, stop every node, start 0.2.0 on the same data directories (docs/COMPATIBILITY.md). Data written by 0.1.1 is read as it is: tested for a single server (stopped cleanly and killed), a 3-node cluster and a 3-node range cluster. Mixed-version clusters are not supported, and 0.1.x cannot open data that 0.2.0 has written.
Formats and protocols#
- On-disk formats: read forward from 0.1.x. New only when encryption at rest is turned on: sealed Raft log frames and snapshot records.
- Client protocol: additive. A
NotLeaderreply's body is now always a leader address or empty (see Fixed). Health reports gain the active encryption key and, on range clusters, each node's ranges. - Node-to-node protocol: every node of a cluster must run 0.2.0.
Release checks#
Run on the release candidate (docs/RELEASE.md); the fixes they led to are listed under Fixed.
- Process chaos: 40 of 40 runs on x86_64 (Azure, 4 vCPU) and 40 of 40 on arm64 (Graviton2, 2 vCPU), then 10 more on each after the
NotLeaderfix. - Soaks, 2 hours each on 4-vCPU, 16 GB VMs (the checklist's 8 vCPU did not fit the cloud quota), with 125–141 of each fault (kill, stop, partition, half-open connection, delay): balances conserved and every key linearizable. Range cluster: at most 630–750 MiB per node, flat, disk 104–228 MiB, at most 87 descriptors. Single group: 850–940 MiB per node, nearly flat, disk 50–145 MiB, at most 36 descriptors. That is above the 334 MiB of the 0.1.0 soak, which ran on another machine: in 10-minute soaks on the same VM size, 0.1.1 peaked at 744–760 MiB for 5,191 writes and 0.2.0 at 813–846 MiB for 27,181.
- SDKs: all 8 suites against one server and against a range cluster, with TLS; the Python suite found the
NotLeaderbug, and all 16 runs pass with the fix. - Upgrade: data written by 0.1.1 (single server stopped cleanly and killed, 3-node cluster, 3-node range cluster) read by 0.2.0, then written to and restarted.
- Undefined behaviour: Miri and AddressSanitizer clean.
- Binary sizes (Linux x86_64 release build):
storage-server4.8 MB,storage-cli1.5 MB,storine-sql6.7 MB,storine-console0.83 MB, under the 5.5, 1.75, 8 and 1 MiB ceilings. - Benchmarks: no regression against 0.1.1 (
BENCHMARK_REPORT.mdsections 8.9 and 8.10).
Added#
- Install scripts:
curl -fsSL https://github.com/Twilight-Techy/storine/releases/latest/download/install.sh | shon Linux and macOS,irm https://github.com/Twilight-Techy/storine/releases/latest/download/install.ps1 | iexon Windows. They download the archive for the system, check it against the release'sSHA256SUMS, and put the four tools on the PATH (STORINE_VERSION,STORINE_INSTALL_DIR). CI runs them on five platforms, and every release is installed with them after it is published. - Encryption at rest in the server:
STORAGE_ENCRYPTION_KEYSnames a key file (storage-server gen-key <file>makes one; it must be readable by its owner only). Records, the WAL, index files, and now Raft logs and snapshots are sealed. Every start reseals whatever does not match the settings, so turning encryption on (withSTORAGE_ENCRYPTION_ALLOW_PLAINTEXT=1once), rotating keys and turning it off (STORAGE_ENCRYPTION_ACTIVE_KEY=none) each take one restart. Health reports show the active key. - Point-in-time recovery of a server:
STORAGE_WAL_ARCHIVE=<dir>archives the WAL at each checkpoint, kept forSTORAGE_WAL_ARCHIVE_RETAIN(7 days by default).storage-cli restore-pitrnow restores a server's document data, takes an RFC 3339 time, and reads the archive directory and a leftover WAL directly (--archive,--wal). Standalone and single-group servers; range clusters keep their cluster backups. - Multi-region placement in range clusters: localities in
--peers(id=host:port@region/zone) and placement rules in--zones <file>: replica counts, required or avoided regions and zones, preferred leaseholder regions, zone or region survival, follower-read staleness, global collections and regional-by-row collections. Health reports list each node's ranges with their voters and leaders. Each node reloads its rules file within 2 seconds of a change; rules that do not fit are refused and reported, and the previous ones stay. server-load(benches): throughput and latency of writes and reads against a running server or cluster (STORAGE_TOKENto log in), andtools/release/compare-versions.shto compare two source trees.EngineConfig::wal_archive_dir,restore_point_in_time_with,StorageEngine::encryption,RaftNode::reseal_storage, andStorageError::Configfor invalid settings.
Measured#
- Against v0.1.0, with the new features off: no regression in the engine, network or server benchmarks (
BENCHMARK_REPORT.mdsection 8.7). Found on the way, unchanged since 0.1.0: a single-group cluster commits a write in about 90 ms (fixed, see Changed). - v0.1.0, v0.1.1 and this code side by side on one arm64 machine: cluster writes 4–9× faster, everything else within noise (
BENCHMARK_REPORT.mdsection 8.9).
Fixed#
NotLeaderreplies could carry error text where clients expect an address: a write or read that lost leadership on its way through the cluster answered with the error's message in the address field, and clients tried to connect to it (the Python SDK failed withlabel too long). The body is now the leader's address, or empty when it is not known. Present since 0.1.0; found by the release SDK run.- Checkpoints that never ended under constant writes: a
flush_allcheckpoint kept flushing records that writers re-queued, and the WAL truncation kept chasing writers that logged faster than the disk could copy. Both now end. Writers that outpace the copy are slowed to about half its pace instead of being stopped: the worst write fell from 9–47 s to under 1 s (BENCHMARK_REPORT.mdsection 8.10). - Collections made only of documents were not listed:
collections()in every SDK, the CLI and the console returned only collections with an index or other metadata. Standalone and single-group servers now list every collection holding documents too; range clusters still list declared collections only. storage-server --helpstarted a server with a data directory named--help.--help/-hnow print the options and--version/-Vthe version. An unknown option is an error instead of being ignored.- Columnar snapshots could show half a transaction: a merge running during a multi-document write could store one document's old version while the snapshot read another's new one. Rare: 1 run in 30 of the concurrency test. A changed document is now recorded for the columnar view only once its new version is readable.
Changed#
- The web console was redesigned and keeps its state: each collection's filter, sort, columns and aggregates, recent and saved queries, pinned collections and recent documents are remembered per server in the browser, and unsaved edits survive a reload. New: a command palette (
Ctrl K/⌘K), keyboard navigation, a live overview with sparklines, a tree view of documents, sorting by column headers, a column chooser, compact rows, aggregate charts, downloading a page as JSON, and undo after deleting a document. Still three files with no libraries. - Single-group cluster writes are 5–8× faster: on three separate machines, 20 → 106 writes a second from one client (p50 47 → 8.3 ms) and 30 → 238 from 16. The leader writes its log outside the Raft state lock, so concurrent writes share one disk sync. The Raft meta file is rewritten in place. The commit index is no longer saved. Entries are applied in batches. Followers acknowledge before applying (
BENCHMARK_REPORT.mdsection 8.8). The durability guarantees are unchanged. - With encryption on, Raft log frames and snapshot records are sealed, so a node with encryption can only exchange snapshots with nodes holding the same keys. Data written without encryption is read as before.
0.1.1 (2026-11-13)#
Binaries for Linux, macOS and Windows, and a web console. On-disk formats, the client protocol and the node-to-node protocol are unchanged from 0.1.0: a rolling upgrade works, and 0.1.0 and 0.1.1 nodes can run in one cluster.
Added#
- Release binaries for Linux (x86_64, arm64; glibc 2.28 or newer), macOS (Apple silicon, Intel; macOS 11 or newer) and Windows (x86_64), with
SHA256SUMS. GitHub Actions builds, checks and smoke-tests them for each tagged release (.github/workflows/release.yml). storine-console: a web console in the browser. It shows the cluster and node health, browses, filters and edits documents with version checks, runs grouped aggregates and manages indexes. It listens on loopback, and only a browser opened with the link it prints (holding a per-run key) can use it.- macOS server support: a
kqueuereactor, likeepollon Linux and AFD poll on Windows. CI now runs the full test suite on macOS and Windows too. - Rust SDK:
Collection::page(one page of a query, with the total and the next page's key) andClient::health.
Changed#
- Release builds target each architecture's baseline CPU instead of the build machine's. Vector scoring (
dot_product_f32,cosine_similarity_f32) moved tostorage-codec, which picks an AVX2 + FMA copy at run time: a baseline build without it scored vectors 20 times slower. Other hot paths already picked hardware CRC32C, AES and AVX2 at run time. Measured against native builds on x86_64 and arm64:BENCHMARK_REPORT.mdsection 8.6. - The bench runner's checksum benchmark measured a loop the compiler had partly hoisted on native builds. It now reads its input through
black_box.
Fixed#
A standalone server (no
--peers) never cut its write-ahead log: nothing checkpointed its engine except an admin request, so the log grew with every write until a restart, which then replayed all of it. It now checkpoints every 10 s while it is written, as cluster nodes already did. New test:a_standalone_server_cuts_its_wal.Windows: the write-ahead log could not be synced. The WAL fsyncs through a second handle, which was opened read-only; Linux and macOS sync any descriptor, but Windows'
FlushFileBuffersneeds write access and refused, so every durable write failed. Append-mode files (Raft logs, index deltas) are now also opened for writing on Windows, with each write moved to the end. Windows had not been tested before 0.1.1; CI now runs the whole suite there.Binary-size ceilings (
docs/RULES.md) apply to the baseline build that ships, which is larger than a native build. Released sizes, Linux x86_64:storage-server4.6 MB,storage-cli1.4 MB,storine-sql6.7 MB,storine-console0.8 MB; macOS arm64: 3.6, 1.2, 5.2 and 0.7 MB. The ceilings are now 5.5, 1.75, 8 and 1 MiB.
Documentation#
- The README is now a short introduction;
docs/TECHNICAL.mdis a new technical reference written from the current code: every program, option, default and limit. - Corrected: the native port accepts TLS 1.2 as well as 1.3; range clusters have change feeds (
storage-cli changefeed); request body limits differ by opcode; the SQL gateway supports functions, procedures, triggers, cursors, temporary tables, materialized views andLISTEN/NOTIFY.
0.1.0 (2026-11-13)#
The first release. Everything below is new.
Engine#
- Embedded storage engine (
StorageEngine<T>,DocumentStore) with zero-copy memory-mapped reads, 64-byte record frames checked with CRC32C (hardware on x86_64 and arm64) and an XXH64-family hash, and zero third-party runtime crates. - Write-ahead log with three policies:
Immediate,GroupCommit(writers share onefsync) andAsyncMemoryFirst. Torn trailing frames are ignored on recovery. Recovery streams the log: memory stays bounded however long it is. - Storage layouts: arena segments with live compaction (the default for documents), one file per record, and ping-pong slots.
- Indexes: primary hash index (in memory, or a cold on-disk tier), tag bitmaps, numeric ranges (in memory, or paged), HNSW vector search, and columnar segments for analytics.
- MVCC snapshots, transactions, conditional and idempotent writes, atomic field operations, exact decimals, TTL, schema migrations.
- Encryption at rest (ChaCha20-Poly1305) with key rotation and per-record policy.
- Online backup and point-in-time recovery.
Server and clusters#
storage-server: a sharded non-blocking TCP server with a binary protocol (protocol/PROTOCOL.md), RBAC tokens and admission control.- TLS 1.3 on the native port:
STORAGE_TLS_CERT/STORAGE_TLS_KEY, with no third-party crates (storage-tls). Handshakes run on their own threads with a 10 s deadline, so a slow client never stalls a reactor. TLS-off performance is unchanged. - Encrypted cluster traffic: with
STORAGE_TLS_CA, nodes verify each other's certificates (by the host named in--peers) and Raft and range traffic is encrypted. Peers still authenticate withSTORAGE_CLUSTER_TOKEN, inside the encrypted channel. storage-server gen-ca <dir>andgen-cert <ca-dir> <out-dir> <names>issue an Ed25519 CA and node certificates.- Single-group Raft replication with membership changes, and range-partitioned clusters (
--ranges) with distributed transactions, rebalancing and multi-region placement.
Clients#
- SDKs for Python, TypeScript, Go, Java, C#, Rust, PHP and Ruby, checked against one set of protocol vectors. Each takes
storines://URLs and a CA file for TLS. storine-sql: a SQL gateway speaking the PostgreSQL, MySQL and SQL Server wire protocols.storage-cli, including over TLS (STORAGE_TLS_CA,STORAGE_TLS_SERVER_NAME).
Performance#
Measured on AWS c7i.xlarge against LMDB and RocksDB at equal durability (docs/BENCHMARK_REPORT.md section 8):
- Storine leads in every throughput row and every p99 row.
- Network: a TCP
GETround trip takes 8.6–9.0 µs, or 11.5–11.7 µs over TLS. - Release binaries (Linux x86_64, stripped, no shared libraries beyond libc):
storage-server3.9 MB,storage-cli1.3 MB,storine-sql5.6 MB.
Fixed before release#
A restarted range-cluster node could take minutes to come back. Range nodes stored one file per record, and recovery
fsynced one file per logged write. A node killed again before it finished started over each time. Range nodes now use the arena layout, asDocumentStoredoes. Recovery writes each record once per batch, and syncs record files in parallel. One stuck node's data now reopens in 2.3 s, where it had been at it for over 4 minutes.Range nodes never cut their WAL or compacted their storage while running: their periodic checkpoint only synced the log. In the soak, logs reached 374 MB in 25 minutes. Each node now checkpoints its engine every 5 s while written, on its own thread. In a 15-minute range soak, logs stayed under 3 MB.
A checkpoint under
SyncPolicy::AsyncMemoryFirstcut the WAL without syncing the records it had just flushed. Raft nodes sync the WAL themselves and then drop log entries, so a power loss (not a process crash) could have lost applied writes. Records are now synced before any WAL is cut.
Known limitations#
- Tail latency, against LMDB and RocksDB (
BENCHMARK_REPORT.mdsection 8.4):- Fill p99.99 is 0.74 ms, against RocksDB's 16–20 µs.
- 4-writer overwrite p99.99 is 2.3–3.0 ms, against 0.46–1.32 ms.
- Durable p99.9 and p99.99 trail LMDB.
- Arena segments can hold up to twice the live data until compaction reclaims them.
- TLS supports one suite per version: TLS 1.3 with
TLS_CHACHA20_POLY1305_SHA256, and TLS 1.2 withTLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256for clients without 1.3; x25519 and Ed25519 certificates. Certificates from other CAs must use Ed25519 keys. Clients are not asked for certificates; they authenticate with tokens. - Change feeds exist for range clusters only (
storage-cli changefeed), and the SDKs do not expose them. - Soak testing ran for 2 hours per mode under constant kills, pauses and partitions; both modes passed with balances conserved and every key linearizable. No multi-day soak has been run.
- A range-cluster node peaked at 1.12 GiB of resident memory in the soak, against 334 MiB for a single-group node.
- Upgrading from one 0.x minor version to another needs a stop of the whole cluster or a backup and restore. Rolling upgrades are supported only between patch releases of the same minor version (
docs/COMPATIBILITY.md).