Commit related keys together

Commit related writes

This complete console example moves an amount between two accounts. It reads both balances inside the transaction and writes both. Execute commits the transaction and retries the whole unit of work when it conflicts:

using ZoneTree;

using var tree = new ZoneTreeFactory<string, long>()
    .SetDataDirectory("data/transactions-example")
    .OpenOrCreateTransactional();
using var maintainer = tree.CreateMaintainer();

bool Transfer(string from, string to, long amount)
{
    return tree.Execute(tx =>
    {
        tx.TryGet(from, out var fromBalance);
        tx.TryGet(to, out var toBalance);
        if (fromBalance < amount)
            return false;
        tx.Upsert(from, fromBalance - amount);
        tx.Upsert(to, toBalance + amount);
        return true;
    });
}

Execute begins a transaction, runs the body, commits it, and returns the body's result. A conflict starts the body again with a new snapshot, so the body may run more than once:

• Keep the body free of effects outside the transaction, such as sending messages, and decide on them after Execute returns.
• Do not commit or roll back inside the body; Execute rejects a body that ends its transaction.
• Throw to abandon the work: the transaction is rolled back and the exception propagates.
• A long body can check tx.HasConflict to stop early; the attempt fails at its commit anyway.
• maxAttempts bounds the attempts; after the last one, the conflict is thrown. By default it retries until it commits. A conflict can also come from an earlier transaction whose commit record is appended but whose durable publication is still waiting for sync.

ExecuteDurableAsync retries conflicts in the same way, but syncs the commit record before making the successful transaction visible to readers.

Manage a transaction yourself

BeginTransaction() returns an ITransaction<TKey, TValue> when you need to control the commit. Reads and writes go through the transaction. Disposing an active transaction rolls it back; a durable commit already in progress continues even if its caller stops waiting:

using var tx = tree.BeginTransaction();
tx.Upsert("a", 1);
tx.Upsert("b", 2);
if (!tx.TryCommit())
    Console.WriteLine("Conflict: retry the transaction from its first read.");

Single-key operations

A transactional tree also offers operations that each run in a transaction of their own:

Use them for keys that need no coordination with other keys. For several keys, put the reads and writes in one transaction.

Single-key and multi-key coordination

Concurrent application threads alone do not require transactions. Use them for a multi-key invariant, and make the reads that decide the writes part of that transaction.

How a transaction works

1. BeginTransaction() takes a snapshot: the transaction reads the database as of the latest commit, plus its own writes.
2. Writes are stored at once, tagged with the transaction, and stay invisible to other transactions.
3. Commit checks for conflicts with transactions that committed or prepared a commit after the snapshot. Without a conflict, it appends a single commit record. Commit() and TryCommit() publish the writes without waiting for durability; the durable methods sync that record before publication.
4. On a conflict, or after Rollback(), the writes are discarded and the transaction ends.

Reads never block, and nothing locks a key. The first transaction to prepare a conflicting commit wins; the other one must retry from its first read. Commits publish in order, so a later commit can wait behind an earlier durable commit's sync. Appends and memory-segment rotation continue during that sync. Execute retries conflicts; a transaction you manage yourself reports each outcome like this:

Reads and writes never throw for a conflict; the commit reports it. A write that finds its key committed or reserved by a prepared commit after the snapshot is not stored and sets HasConflict. The transaction stays active, stores no more writes, and its commit fails.

Transactions in Committing, Committed, Aborted, or InDoubt cannot be used for further reads or writes. Retry conflicts with a new transaction and a new snapshot. An InDoubt transaction cannot be rolled back: its commit record may already be durable, and recovery may retain it.

Choose an isolation level

BeginTransaction takes an isolation level. Both levels read a consistent snapshot and never lose a concurrent write to the same key.

Use Serializable unless profiling shows that its read tracking matters. It keeps the keys a transaction read and the ranges it iterated until the commit, so a transaction that reads millions of keys holds that read set in memory. Snapshot keeps only the written keys and conflicts less often.

In a Snapshot transaction, read the keys a decision depends on with TryGetForUpdate. It protects those keys like writes, which prevents write skew on them without tracking every read. A transaction that writes nothing commits at its snapshot, so neither level validates its reads.

Read ranges

Range reads the records between two keys, inclusive, from the same snapshot as point reads, in ascending or descending key order:

using var tx = tree.BeginReadOnlyTransaction();
foreach (var (key, value) in tx.Range("order:0000", "order:9999"))
    Console.WriteLine($"{key} = {value}");

The records are read while the result is enumerated, before the transaction ends. In a serializable transaction the range joins the read set: the whole range when the enumeration runs to the end, or up to the last key read when it stops early. A commit after the snapshot that writes inside that range, even a brand new key, makes this transaction conflict.

For scans without an upper bound, use an iterator:

using var tx = tree.BeginTransaction();
using var iterator = tx.CreateIterator();
iterator.Seek("order:");
while (iterator.Next() && iterator.CurrentKey.StartsWith("order:"))
    Console.WriteLine($"{iterator.CurrentKey} = {iterator.CurrentValue}");

In a serializable transaction, an iterator adds the range from its seek key to the last key it read to the read set. Stop iterating when you have what you need: an iterator that runs to the end protects the rest of the key space too.

A range or an iterator sees the transaction's own writes made before the enumeration starts, or before each Seek, SeekFirst, or first Next call. Dispose iterators when the scan is done; they cannot be used after the transaction ends.

Read-only transactions

BeginReadOnlyTransaction() returns an IReadTransaction<TKey, TValue>: it reads a consistent snapshot without a transaction id or a read set, never conflicts, and has no write methods. Disposing it ends it. Use it for reports, exports, and any query that must see several keys or ranges at one point in time.

using var snapshot = tree.BeginReadOnlyTransaction();
var total = 0L;
using var iterator = snapshot.CreateIterator();
while (iterator.Next())
    total += iterator.CurrentValue;

ITransaction<TKey, TValue> extends IReadTransaction<TKey, TValue>, so code that only reads can accept either.

A long-running snapshot keeps the versions it can read: merges cannot drop a version that an open transaction may still read. End transactions when their work is done.

Durability

With WAL enabled, the durable APIs append the commit record, sync the WAL, and then publish the transaction. Readers cannot observe those commits before they are durable. Ordinary Commit() and TryCommit() publish without requiring a disk sync. Choose the boundary your application needs:

Concurrent durable commits and SyncWALAsync callers share disk syncs, so many committing threads pay for one sync together. A single thread that waits after every commit pays a full sync per commit.

The built-in transactional factories promote Sync and SyncCompressed to Shared, and AsyncCompressed to AsyncShared. Compression settings are retained; Sync does not turn compression off. Explicit shared modes and None are retained. Custom providers and provider factories supply their own recovery guarantees.

Shared hands appended records to the operating system, so completed ordinary commits survive a process crash. AsyncShared can lose queued records on a process crash. A power loss or operating-system crash can lose unsynced ordinary commits in either mode, but recovery keeps one prefix of commits across all segments, including every commit covered by a successful sync. Incomplete transactions stay invisible; a later commit cannot survive while an earlier commit in that prefix is lost.

If every writing transaction uses a durable commit API, readers see only commits that have already synced. Mixing in ordinary Commit() or TryCommit() allows readers to see those unsynced commits. There is no separate durable-only read option.

A cancellation token cancelled before the durable commit starts leaves the transaction active. Once its commit record is appended, cancellation stops only the caller's wait: the commit continues and becomes visible after a successful sync. Disposing a Committing transaction does not roll it back. A failed sync leaves the outcome InDoubt, and recovery may find the commit record even though the caller received an error.

AutoSyncInterval defaults to zero. It schedules SyncWALAsync, updates MVCC's durable bookkeeping, and requests syncs even when an async writer stays busy. WAL background jobs also sync automatically, but an async job that syncs only on an empty queue has no fixed persistence deadline. None of these syncs can make unlogged writes durable with mode None. See WAL modes for scheduling and storage details.

Transaction size

A transaction's writes are stored in the tree as it makes them, so they persist, move to disk, and merge like other records. Its size is limited by the disk, not the memory. The memory holds only the keys it wrote and, in serializable mode, the keys and ranges it read.

Rolling back a large transaction writes a deletion for every key it wrote, and merges remove its versions over time.

Maintenance

A maintainer merges the storage tree in the background, and merges also drop the versions no open transaction can read. Keep one running with the tree.