This article is about the general architecture of the Odilon server. It is of potential interest to developers but is not necessary to use the product.

Odilon has three hierarchical layers (API, Object Storage, Virtual File System) and about a dozen general services used by them (Scheduler, Lock, Journal, Cache, Encryption, etc.).

API

This is the HTTP/S interface used by client applications, like Odilon Java SDK, it is a RESTFul API implemented with Spring Boot: Bucket and Object operations, System info, and others. They interact directly with the Object Storage, they do not see the lower layers. See API Controllers Java source code

Object Storage

It is essentially an intermediary that downloads the requirements into the Virtual File System. Object Storage Java source code

Virtual File System

The Virtual File System layer manages the repository on top of the OS File System. Odilon uses the underlying File System to store objects as encrypted files, or in some configurations to break objects into chunks. It implements software RAID, which depending on the configuration can be RAID 0, RAID 1, RAID 6/Erasure Coding.
Odilon uses Reed Solomon encoding for Erasure Codes.

Scheduler

The Scheduler is a persistent job queue processor. It can manage multiple job queues, each with a specific execution policy (normally FIFO: First In, First Out) and semantics on job failure (ie. retry N times and continue, block and retry until the job can complete successfully, ...). The Scheduler persists jobs on the Virtual File System.

A Scheduler Worker is the main component of the Scheduler. It is a job queue with its' own policies and Thread pool to process the queue. It creates a dependency graph with the jobs that are to be executed in parallel in each batch in order to warrant that after the execution of the batch the end result will be equivalent to a sequential execution. The main scheduler workers are:

• Standard

  CRUD operations after the Transaction is committed by the Journal Service.
  

• CronJob

  Cron jobs that execute regularly based on a Cron expression, they are non blocking.
  

• Standby replication

  Executes asynchronous replication to the standby Odilon server. This worker will not be started if there is no Standby server connected. The semantics of the replica queue is strict order, If a job can not be completed the queue will block until it can be completed.
  

Journal

The default method by which Odilon implements atomic commit and rollback is a Rollback Journal. When a thread wants to execute an operation, it first records the original unchanged content in a rollback journal(Java source code).

Encryption

Odilon keeps objects encrypted (Encryption at Rest) using modern algorithms such as AES-256. Each object has a unique encryption key. In turn, the encryption key of the object can be generated by Odilon or, which is recommended for greater security, by a Key Management Server (KMS). (Java source code).

Lock

Odilon uses a lock-based resource protection on Read and Write provided by the Lock service (Java source code).

Replication

Service that asynchronously propagates operations already completed (after commit) to the Standby server (Java source code).

Other Services

Authentication, ObjectMetadata Cache and File Cache (both use the Caffeine library), Traffic Control to prevent resources overflow, Health Check and Monitoring (uses the Dropwizard Metrics library), Settings among others.