Odilon can be configured to use software RAID for data replication. The supported configurations are RAID 0 (default, no data replication), RAID 1
(mirroring), ErasureCoding (Reed-Solomon erasure coding). Data storage mode and
data directories are configured in the configuration file (default location is:
/etc/odilon/odilon.properties)
RAID 0
It is possible to use RAID 0 with only one disk, and also combine two or more disks for data storage. It is
not a configuration with data replication, its function is to provide greater storage and performance by allowing
access to the disks in parallel.
The variable redundancyLevel must be set to RAID 0 and the data directory or directories must be
added to the variable dataStorage in odilon.properties.
# default value is RAID 0
redundancyLevel=RAID 0
dataStorage=/opt/odilon-data/drive0
The configuration must include at least one data directory (in the example: /opt/odilon-data/drive0),
but you can specify two or more separated by commas. If these belong to different disks, they offer better
performance. Odilon distributes objects evenly across all directories.
Example with two data directories:
# default value is RAID 0
redundancyLevel=RAID 0
dataStorage=/opt/odilon-data/drive0, opt/odilon-data/drive1
RAID 0. Adding disks
Data directories can be added to the variable dataStorage, even if the Odilon server already contains
stored objects. When the server restarts, Odilon detects the existence of one or more new data directories and
runs a process to redistribute the objects across all data directories. This process is synchronous, and the
server will be unavailable until it is complete.
Erasure Coding
It is a method of encoding data into blocks (called shards) that can be distributed across multiple disks or
nodes and then reconstructed from a subset of those blocks. It has great flexibility since you can adjust the
number and size of the blocks and the minimum required for recovery. It uses less disk space than RAID 1 and can
withstand multiple full disk failures. Odilon implements this architecture using Reed Solomon
error-correction codes.
The supported configurations are:
- 3 disks (2 data and 1 parity, supports 1 full disk failure)
- 6 disks (4 data and 2 parity, supports up to 2 full disks failure)
- 12 disks (8 data and 4 parity, supports up to 4 full disk failure)
- 24 disks (16 data and 8 parity, supports up to 8 full disk failure)
- 48 disks (32 data and 16 parity, supports up to 16 full disk failure)
Volumes
Odilon organizes Erasure Coding storage into volumes. Each volume consists of N data directories (N=3, 6,
12, 24, 48). In a typical deployment, the server starts with a single volume containing N directories.
Erasure Coding configuration
Set the variable redundancyLevel=ErasureCoding, the data directories in dataStorage.volume.0= comma separated list of N directories (N=3, 6, 12, 24, 48)
and volume.active=0. This configuration can be extended in the future adding
volumes.
Example Erasure Coding with 3 disks:
#
# ErasureCoding using 3 data directories (N=3)
#
redundancyLevel=ErasureCoding
dataStorage.volume.0=/data/d0,/data/d1,/data/d2
volume.active=0
Encoding
Odilon splits objects into 32 MB stripes, and each stripe is then converted into N Reed Solomon shards
(which include redundancy).
If encryption is enabled, the objects are encrypted and then split and encoded with Reed-Solomon. This
whole process is performed on the byte stream in one pass.
Full disk failures
Erasure Coding protects against complete disk failures without data loss. The number of disks that can fail
depends on the size of the volume. For example, a 3-disk volume can tolerate one disk failure, while a 6-disk
volume can tolerate two.
After a failed disk is replaced and the server is restarted, Odilon automatically rebuilds the lost data in
the background. The rebuild process is asynchronous, so the server is fully operational immediately and does not
need to wait for it to complete.
Erasure Coding. Adding capacity
The server can be expanded by adding new storage in groups of N data directories, where N = 3, 6, 12, 24,
or 48.
Odilon organizes Erasure Coding storage into volumes. Each volume consists of N data directories. In a
typical deployment, the server starts with a single volume containing N directories.
When additional capacity is required, a new volume can be added: the administrator declares them as a new volume
(dataStorage.volume. and the volume number, which is also the the volume id)
and sets volume.active to the new volume id in odilon.properties. The new
volume must also contain N data directories. Once added, it becomes the active volume, while all previously
existing volumes are automatically switched to read-only mode.
Only one volume can be active at any given time. All new objects are written to the active volume, while
existing objects remain accessible in the older read-only volumes.
Example Erasure Coding with N=3, volume 0 is not active, just used to read objects, volume 1 is active
#
# ErasureCoding using 3 data directories (N=3)
# 2 volumes, volume 0 is read only, volume 1 is active
#
redundancyLevel=ErasureCoding
dataStorage.volume.0=/data/d0,/data/d1,/data/d2
dataStorage.volume.1=/data/d3,/data/d4,/data/d5
volume.active=1
Read-repair strategy
Odilon can detect and repair silent data corruption caused by bad sectors, bit flips, or other storage
errors. At write time, the SHA-256 checksum of each individual Reed-Solomon shard is computed and stored in the
object's metadata. During every subsequent read, each shard is hashed in memory and compared against its stored
checksum. Any shard whose digest does not match is treated as an erasure and reconstructed from the remaining
healthy shards in a single RS decode pass. The repaired shard is then written back to disk so future reads find
it intact. This process is O(N) SHA-256 hashes over data already loaded in memory and adds no extra disk I/O.
Because this verification runs on every read, it adds a small CPU cost proportional to the total shard data size.
On filesystems without built-in block checksums — such as ext4 or XFS — this is the only
application-level protection against silent corruption and should be kept enabled. It is disabled by
default, to enable set the variable (ec.shardChecksumVerify=true).
On filesystems that already provide end-to-end integrity guarantees — such as ZFS (with
checksum=sha256
or
checksum=fletcher4
) or Btrfs — the filesystem catches corruption before Odilon sees the bytes, making the shard-level
SHA-256 pass redundant. In those environments it can be disabled to eliminate the unnecessary CPU overhead by
setting ec.shardChecksumVerify=false in odilon.properties.
#
# ErasureCoding using 3 data directories (N=3)
# Shard checksum verification enabled (recommended for ext4 / XFS)
# Set to false on ZFS or Btrfs where the filesystem already guarantees integrity
#
redundancyLevel=ErasureCoding
dataStorage.volume.0=/data/d0,/data/d1,/data/d2
volume.active=0
ec.shardChecksumVerify=true
Data scrubber
Odilon runs a background data scrubber that periodically walks every object in storage and verifies its
integrity. The process runs in two phases:
Phase 1 — Detection. For each object the scrubber decodes the head version and compares the SHA-256
hash of the decoded payload against the hash stored in the object’s metadata. If the hashes match, the
object is intact and no further action is taken.
Phase 2 — Shard identification and repair. If a mismatch is detected, the scrubber verifies the
SHA-256 checksum of every individual shard file to identify the exact positions of the corrupt shards. Those
positions are then treated as erasures — locations whose index is known — before the
Reed-Solomon decoder is invoked. Converting errors of unknown position into erasures of known position restores
the full parity capacity: up to P simultaneous corrupt shards can be recovered for a volume with P parity shards,
the same tolerance as a full-disk failure. The payload is then re-encoded with fresh Reed-Solomon shards and
correct parity, and the metadata (per-shard hashes, etag, integrity timestamp) is updated atomically via the
journal. On a successful repair the object’s integrityStatus field is set to
OK. If the number of corrupt shards exceeds the parity capacity, the field is set
to IRRECOVERABLE and persisted to disk so that affected objects can be identified
without re-running a full scrub pass. Irrecoverable objects should be investigated and, where possible, restored
from a standby replica or a backup.
This approach complements read-repair: while read-repair only triggers when an object is actively
requested, the scrubber detects and repairs silent corruption in objects that are rarely or never accessed,
preventing gradual data degradation from going unnoticed.
The scrubber is enabled by default and runs on a configurable schedule. The following properties in odilon.properties
control its behaviour:
# Enable or disable the background data scrubber (default: true)
integrityCheck=true
# Number of parallel threads used during a scrub pass (default: auto)
integrityCheckThreads=2
# Minimum age in days before an object is re-checked (default: 180)
# Objects whose last integrity check is more recent than this value are skipped.
integrityCheckDays=180
# Cron expression that controls when the scrub pass starts
# default vaule is every Sunday at 5:5:15 AM server time: 15 5 5 * * 7
#
# Other expressions:
#
# Last Sunday of every month at 5:15 AM server time: 15 5 5 * * 7L
# Every 4 months (apr, aug, dec) the first Saturday of the month at 5:15 AM server time: 15 5 5 ? 4,8,12 6#1
# see https://github.com/atolomei/odilon-server/blob/main/src/main/java/io/odilon/scheduler/CronExpressionJ8.java
#
integrityCheckCronExpression=15 15 5 * * *
At the end of each scrub pass Odilon logs a summary to the dataIntegrityCheck log
file, reporting the total number of objects scanned, objects verified clean, objects successfully repaired, and
objects that could not be recovered. Irrecoverable objects should be investigated and, where possible, restored
from a standby replica or a backup.