Odilon can be configured to use software RAID for data replication. The supported configurations are RAID 0 (default, no data replication), RAID 1
(mirroring), ErasureCoding (Reed-Solomon erasure coding). Data storage mode and data directories are
configured in the configuration file (default location is: /etc/odilon/odilon.properties)
RAID 0
It is possible to use RAID 0 with only one disk, and also combine two or more disks for data storage. It is
not a configuration with data replication, its function is to provide greater storage and performance by allowing
access to the disks in parallel.
The variable redundancyLevel must be set to RAID 0 and the data directory or directories must be
added to the variable dataStorage in odilon.properties.
# default value is RAID 0
redundancyLevel=RAID 0
dataStorage=/opt/odilon-data/drive0
The configuration must include at least one data directory (in the example: /opt/odilon-data/drive0),
but you can specify two or more separated by commas. If these belong to different disks, they offer better
performance. Odilon distributes objects evenly across all directories.
Example with two data directories:
# default value is RAID 0
redundancyLevel=RAID 0
dataStorage=/opt/odilon-data/drive0, opt/odilon-data/drive1
RAID 0. Adding disks
Data directories can be added to the variable dataStorage, even if the Odilon server already contains
stored objects. When the server restarts, Odilon detects the existence of one or more new data directories and
runs a process to redistribute the objects across all data directories. This process is synchronous, and the
server will be unavailable until it is complete.
Erasure Coding
It is a method of encoding data into blocks (called shards) that can be distributed across multiple disks or nodes and then
reconstructed from a subset of those blocks. It has great flexibility since you can adjust the number and size of
the blocks and the minimum required for recovery. It uses less disk space than RAID 1 and can withstand multiple
full disk failures. Odilon implements this architecture using Reed Solomon
error-correction codes.
The supported configurations are:
- 3 disks (2 data and 1 parity, supports 1 full disk failure)
- 6 disks (4 data and 2 parity, supports up to 2 full disks failure)
- 12 disks (8 data and 4 parity, supports up to 4 full disk failure)
- 24 disks (16 data and 8 parity, supports up to 8 full disk failure)
- 48 disks (32 data and 16 parity, supports up to 16 full disk failure)
Volumes
Odilon organizes Erasure Coding storage into volumes. Each volume consists of N data directories (N=3, 6, 12, 24, 48). In a typical
deployment, the server starts with a single volume containing N directories.
Erasure Coding configuration
Set the variable redundancyLevel=ErasureCoding, the data directories
in dataStorage.volume.0= comma separated list of N directories (N=3, 6, 12,
24, 48) and volume.active=0. This configuration can be extended in the future
adding volumes.
Example Erasure Coding with 3 disks:
#
# ErasureCoding using 3 data directories (N=3)
#
redundancyLevel=ErasureCoding
dataStorage.volume.0=/data/d0,/data/d1,/data/d2
volume.active=0
Encoding
Odilon splits objects into 32 MB stripes, and each stripe is then converted into N Reed Solomon shards (which
include redundancy).
If encryption is enabled, the objects are encrypted and then split and encoded with Reed-Solomon. This whole
process is performed on the byte stream in one pass.
Full disk failures
Erasure Coding protects against complete disk failures without data loss. The number of disks that can fail depends
on the size of the volume. For example, a 3-disk volume can tolerate one disk failure, while a 6-disk volume can
tolerate two.
After a failed disk is replaced and the server is restarted, Odilon automatically rebuilds the lost data in
the background. The rebuild process is asynchronous, so the server is fully operational immediately and does not
need to wait for it to complete.
Erasure Coding. Adding capacity
The server can be expanded by adding new storage in groups of N data directories, where N = 3, 6, 12,
24, or 48.
Odilon organizes Erasure Coding storage into volumes. Each volume consists of N data directories. In a typical
deployment, the server starts with a single volume containing N directories.
When additional capacity is required, a new volume can be added: the administrator declares them as a new volume
(dataStorage.volume. and the volume number, which is also the the volume id)
and sets volume.active to the new volume id in odilon.properties. The new
volume must also contain N data directories. Once added, it becomes the active volume, while all previously
existing volumes are automatically switched to read-only mode.
Only one volume can be active at any given time. All new objects are written to the active volume, while
existing objects remain accessible in the older read-only volumes.
Example Erasure Coding with N=3, volume 0 is not active, just used to read objects, volume 1 is active
#
# ErasureCoding using 3 data directories (N=3)
# 2 volumes, volume 0 is read only, volume 1 is active
#
redundancyLevel=ErasureCoding
dataStorage.volume.0=/data/d0,/data/d1,/data/d2
dataStorage.volume.1=/data/d3,/data/d4,/data/d5
volume.active=1
Read-repair strategy
Odilon can detect and repair silent data corruption caused by bad sectors, bit flips, or other storage
errors. At write time, the SHA-256 checksum of each individual Reed-Solomon shard is computed and stored in the
object's metadata. During every subsequent read, each shard is hashed in memory and
compared against its stored checksum. Any shard whose digest does not match is treated as an erasure and
reconstructed from the remaining healthy shards in a single RS decode pass. The repaired shard is then written
back to disk so future reads find it intact. This process is O(N) SHA-256 hashes over data already loaded in
memory and adds no extra disk I/O.
Because this verification runs on every read, it adds a small CPU cost proportional to the total shard data
size. On filesystems without built-in block checksums — such as ext4 or XFS — this is the only
application-level protection against silent corruption and should be kept enabled. It is
disabled by default, to enable set the variable (ec.shardChecksumVerify=true).
On filesystems that already provide end-to-end integrity guarantees — such as ZFS (with
checksum=sha256 or checksum=fletcher4) or Btrfs — the filesystem catches
corruption before Odilon sees the bytes, making the shard-level SHA-256 pass redundant. In those environments
it can be disabled to eliminate the unnecessary CPU overhead by setting
ec.shardChecksumVerify=false in odilon.properties.
#
# ErasureCoding using 3 data directories (N=3)
# Shard checksum verification enabled (recommended for ext4 / XFS)
# Set to false on ZFS or Btrfs where the filesystem already guarantees integrity
#
redundancyLevel=ErasureCoding
dataStorage.volume.0=/data/d0,/data/d1,/data/d2
volume.active=0
ec.shardChecksumVerify=true
Data scrubber
Odilon runs a background data scrubber that periodically walks every object
in storage and verifies its integrity. The process runs in two phases:
Phase 1 — Detection. For each object the scrubber decodes the head version and
compares the SHA-256 hash of the decoded payload against the hash stored in the object’s
metadata. If the hashes match, the object is intact and no further action is taken.
Phase 2 — Shard identification and repair. If a mismatch is detected, the scrubber
verifies the SHA-256 checksum of every individual shard file
to identify the exact positions of the corrupt
shards. Those positions are then treated as erasures — locations whose index is
known — before the Reed-Solomon decoder is invoked. Converting errors of unknown position
into erasures of known position restores the full parity capacity: up to P simultaneous corrupt
shards can be recovered for a volume with P parity shards, the same tolerance as a full-disk
failure. The payload is then re-encoded with fresh Reed-Solomon shards and correct parity, and
the metadata (per-shard hashes, etag, integrity timestamp) is updated atomically via the journal.
On a successful repair the object’s integrityStatus field
is set to OK. If the number of corrupt shards exceeds the parity
capacity, the field is set to IRRECOVERABLE and persisted to disk
so that affected objects can be identified without re-running a full scrub pass. Irrecoverable
objects should be investigated and, where possible, restored from a standby replica or a backup.
This approach complements read-repair: while read-repair only triggers when an object is actively
requested, the scrubber detects and repairs silent corruption in objects that are rarely or never
accessed, preventing gradual data degradation from going unnoticed.
The scrubber is enabled by default and runs on a configurable schedule. The following properties
in odilon.properties control its behaviour:
# Enable or disable the background data scrubber (default: true)
integrityCheck=true
# Number of parallel threads used during a scrub pass (default: auto)
integrityCheckThreads=2
# Minimum age in days before an object is re-checked (default: 180)
# Objects whose last integrity check is more recent than this value are skipped.
integrityCheckDays=180
# Cron expression that controls when the scrub pass starts
# default vaule is every Sunday at 5:5:15 AM server time: 15 5 5 * * 7
#
# Other expressions:
#
# Last Sunday of every month at 5:15 AM server time: 15 5 5 * * 7L
# Every 4 months (apr, aug, dec) the first Saturday of the month at 5:15 AM server time: 15 5 5 ? 4,8,12 6#1
# see https://github.com/atolomei/odilon-server/blob/main/src/main/java/io/odilon/scheduler/CronExpressionJ8.java
#
integrityCheckCronExpression=15 15 5 * * *
At the end of each scrub pass Odilon logs a summary to the
dataIntegrityCheck log file, reporting the total number of objects
scanned, objects verified clean, objects successfully repaired, and objects that could not be
recovered. Irrecoverable objects should be investigated and, where possible, restored from a
standby replica or a backup.