Disk Segment Tuning

Disk segment tuning controls the long-term shape of persistent data.

Start With The Workload

Before tuning, identify whether the workload is dominated by:

• point reads,
• range scans,
• large sequential scans,
• heavy writes,
• deletes/TTL,
• large values,
• backup/restore constraints,
• file-size limits.

Disk Segment Mode

MultiPartDiskSegmentis the default and is usually the right choice for large datasets. It partitions large disk segments into parts and helps keep very large storage shapes manageable.

SingleDiskSegmentcan be appropriate for smaller databases where one segment file is easier to manage.

using ZoneTree.Options;

using var zoneTree = new ZoneTreeFactory<int, string>()
    .SetDataDirectory("data/app")
    .ConfigureDiskSegmentOptions(options =>
    {
        options.DiskSegmentMode = DiskSegmentMode.MultiPartDiskSegment;
    })
    .OpenOrCreate();

Minimum And Maximum Record Count

For multipart disk segments:

• MinimumRecordCountcontrols the lower target size for a part,
• MaximumRecordCountcontrols the upper target size for a part.

The defaults are1_500_000and3_000_000records. The higher-levelDiskSegmentMaxItemCountdefault is20_000_000records.

Larger parts mean fewer files and potentially better sequential behavior. Smaller parts can reduce operational file-size pressure and make part-level reuse more flexible.

ZoneTree randomizes multipart part sizes between the configured minimum and maximum. This avoids rigid split boundaries and helps multipart merge reuse stay effective over time. See write amplification.

Sparse Array Step Size

DefaultSparseArrayStepSizecontrols how frequently ZoneTree records sparse index entries while creating disk segments.

Trade-off:

Use a lower step size when point lookups and seeks dominate. Use a higher step size when memory pressure matters more.

The default step size is1024.

using var zoneTree = new ZoneTreeFactory<int, string>()
    .SetDataDirectory("data/app")
    .ConfigureDiskSegmentOptions(options =>
    {
        options.DefaultSparseArrayStepSize = 512;
    })
    .OpenOrCreate();

Fixed-Size Layouts

When keys and values are small unmanaged structs, ZoneTree can use fixed-size disk segment layouts. This can reduce metadata overhead and simplify disk access.

For variable-length values such as strings and byte arrays, ZoneTree uses layouts with offsets and headers.

Compression Block Size

CompressionBlockSizeaffects disk compression and random-access behavior.

The default disk compression block size is4 MB, using LZ4 fastest compression.

Larger blocks often compress better but can make small random reads more expensive. Smaller blocks can improve random read granularity but may reduce compression ratio.

Cache Settings

Disk reads are mostly shaped by the decompressed block cache. Disk segments are compressed in blocks by default, and repeated reads can reuse decompressed blocks.

Block cache cleanup is controlled by the maintainer:

using var maintainer = zoneTree.CreateMaintainer();

maintainer.BlockCacheLifeTime = TimeSpan.FromMinutes(2);
maintainer.InactiveBlockCacheCleanupInterval = TimeSpan.FromSeconds(30);

Circular key/value caches are a smaller per-record layer:

• KeyCacheSize
• ValueCacheSize
• key/value cache lifetimes

The default key and value cache sizes are1024records each, with10 secondrecord lifetimes. Increase them when the same disk record indexes are read repeatedly.

using var zoneTree = new ZoneTreeFactory<int, string>()
    .SetDataDirectory("data/app")
    .ConfigureDiskSegmentOptions(options =>
    {
        options.KeyCacheSize = 4096;
        options.ValueCacheSize = 4096;
        options.KeyCacheRecordLifeTimeInMillisecond = 30_000;
        options.ValueCacheRecordLifeTimeInMillisecond = 30_000;
    })
    .OpenOrCreate();

Iterator scans do not contribute to the shared block cache by default. Enable iterator cache contribution only when the scan represents a useful working set.

See read-path caching.

Practical Defaults

Default disk segment settings are designed to be reasonable for general workloads. Tune only after you know what is limiting the system:

• memory,
• disk IO,
• file count,
• merge duration,
• read latency,
• backup/restore time.