Virtual File System
The Virtual File System layer
manages the repository on top of the OS File System. Odilon uses the underlying File System to store objects as encrypted files, or in some configurations to break objects into stripes, and each stripe into N shards.
It implements software RAID (
RAID 0,
RAID 1), and
Erasure Coding designed to withstand full disk failures without data loss.
Journaled atomic operations
Odilon implements atomic object updates using write-ahead journaling and versioned backups.
Each update is recorded in the journal before any changes are applied. The previous object state is preserved until the transaction commits, allowing automatic rollback after a failure. After a successful commit, the backup is discarded or retained as an earlier version if version control is enabled.
Erasure Coding
It is a method of encoding data into blocks (called shards) that can be distributed across multiple disks or nodes and then
reconstructed from a subset of those blocks. It has great flexibility since you can adjust the number and size of
the blocks and the minimum required for recovery. It uses less disk space than RAID 1 and can withstand multiple
full disk failures. Odilon implements this architecture using Reed Solomon
error-correction codes.
Volumes
Odilon organizes Erasure Coding storage into volumes. Each volume consists of N data directories (N=3, 6, 12, 24, 48). In a typical
deployment, the server starts with a single volume containing N directories.
Encoding
Odilon splits objects into 32 MB stripes, and each stripe is then converted into N Reed Solomon shards (which
include redundancy).
If encryption is enabled, the objects are encrypted and then split and encoded with Reed-Solomon. This whole
process is performed on the byte stream in one pass.
Full disk failures
Erasure Coding protects against complete disk failures without data loss. The number of disks that can fail depends
on the size of the volume. For example, a 3-disk volume can tolerate one disk failure, while a 6-disk volume can
tolerate two.
After a failed disk is replaced and the server is restarted, Odilon automatically rebuilds the lost data in
the background. The rebuild process is asynchronous, so the server is fully operational immediately and does not
need to wait for it to complete.
Erasure Coding. Adding capacity
The server can be expanded by adding new storage in groups of N data directories, where N = 3, 6, 12,
24, or 48.
Odilon organizes Erasure Coding storage into volumes. Each volume consists of N data directories. In a typical
deployment, the server starts with a single volume containing N directories.
When additional capacity is required, a new volume can be added: the administrator declares them as a new volume
(dataStorage.volume. and the volume number, which is also the the volume id)
and sets volume.active to the new volume id in odilon.properties. The new
volume must also contain N data directories. Once added, it becomes the active volume, while all previously
existing volumes are automatically switched to read-only mode.
Only one volume can be active at any given time. All new objects are written to the active volume, while
existing objects remain accessible in the older read-only volumes.
Data scrubber
Odilon runs a background data scrubber that periodically walks every object
in storage and verifies its integrity. The process runs in two phases:
Phase 1 — Detection. For each object the scrubber decodes the head version and
compares the SHA-256 hash of the decoded payload against the hash stored in the object’s
metadata. If the hashes match, the object is intact and no further action is taken.
Phase 2 — Shard identification and repair. If a mismatch is detected, the scrubber
verifies the SHA-256 checksum of every individual shard file
to identify the exact positions of the corrupt
shards. Those positions are then treated as erasures — locations whose index is
known — before the Reed-Solomon decoder is invoked. Converting errors of unknown position
into erasures of known position restores the full parity capacity: up to P simultaneous corrupt
shards can be recovered for a volume with P parity shards, the same tolerance as a full-disk
failure. The payload is then re-encoded with fresh Reed-Solomon shards and correct parity, and
the metadata (per-shard hashes, etag, integrity timestamp) is updated atomically via the journal.
This approach complements read-repair: while read-repair only triggers when an object is actively
requested, the scrubber detects and repairs silent corruption in objects that are rarely or never
accessed, preventing gradual data degradation from going unnoticed.
The scrubber is enabled by default and runs on a configurable schedule.
Read-repair strategy
Odilon can detect and repair silent data corruption caused by bad sectors, bit flips, or other storage
errors. At write time, the SHA-256 checksum of each individual Reed-Solomon shard is computed and stored in the
object's metadata (sha256Blocks). During every subsequent read, each shard is hashed in memory and
compared against its stored checksum. Any shard whose digest does not match is treated as an erasure and
reconstructed from the remaining healthy shards in a single RS decode pass. The repaired shard is then written
back to disk so future reads find it intact. This process is O(N) SHA-256 hashes over data already loaded in
memory and adds no extra disk I/O.
Because this verification runs on every read, it adds a small CPU cost proportional to the total shard data size.
On filesystems without built-in block checksums — such as ext4 or XFS — this is the only
application-level protection against silent corruption and should be enabled.
On filesystems that already provide end-to-end integrity guarantees — such as ZFS (with
checksum=sha256 or checksum=fletcher4) or Btrfs — the filesystem catches
corruption before Odilon sees the bytes, making the shard-level SHA-256 pass redundant. In those environments
it can be disabled by setting ec.shardChecksumVerify=false in
odilon.properties to eliminate the unnecessary CPU overhead.