Architecture

ScaleDB provides an open source API which can be found here. License type: GPL V2.

ScaleDB provides a clustered, shared-disk storage engine. The shared-disk architecture enables multiple nodes to share the same physical data, eliminating the need to partition your data.

The diagram below shows a multi-node cluster. Each node runs the ScaleDB storage engine in conjunction with a DBMS like MySQL. The ScaleDB Cluster Manager then orchestrates the interaction of these nodes.

The Cache Accelerator Server (CAS) provides both storage and shared cache. One role of the CAS is to provide a high-speed cache for sharing data between database nodes. Another other role is to provide a highly-available low-cost set of storage devices. The final role of the CAS is to provide tight integration between storage and computing. If a database node requires some filtered or processed data, instead of moving the entire data set over the network to the node, the CAS can process it locally and only send the much more compact result set to the node. This sort of local processing at the storage tier is similar to what Oracle Exadata does.

All data sent to the CAS is mirrored into two or more CAS servers. This provides the redundancy of high-end storage, while running on commodity servers or cloud instances.ScaleDB CAS are designed to share the data, in conjunction with the Cluster Manager, amongst all of the database nodes. This eliminates the cost, complexity and overhead of a cluster file system.

Finally, as your data grows, or if the storage layer becomes a performance bottleneck, you can add additional pairs of CAS servers. ScaleDB dynamically redistributes the data across the new set of CAS servers, providing improved storage throughput. Basically, if your database application is CPU bound, add more database nodes, if it is I/O bound, add more CAS nodes.

ScaleDB Database Architecture

This architecture provides:

  • Dynamic Compute Scalability: Additional nodes can be added to the cluster without the need to repartition your data
  • Dynamic Storage Scalability: Add more CAS pairs, as needed, to expand your storage capacity and throughput
  • High-Availability: There is no single point of failure. If a node fails, the remaining nodes gracefully take-over uncommitted transactions. If the Cluster Manager fails, the standby Cluster Manager takes over. The data on disk and in the cache is mirrored to protect against disk failure
  • Cluster-Level Load Balancing: Since each node can address any database request in a master-only model, the load can be balanced across the entire cluster
  • Lower Total Cost of Ownership (TCO): The load-balanced, master-only architecture eliminates the complex process of data partitioning and the synchronization between masters and slaves. This significantly reduces total cost of ownership

The following diagram describes the various modules of the ScaleDB architecture:

read more

For Each Node:

  1. ScaleDB API – Exposes the ScaleDB functions to DBMS and applications that leverage ScaleDB.
  2. Transaction Manager – Ensures that the transactions are safe, guaranteeing completion or else rolling-back the uncommitted transaction in case of failure.
  3. Index Manager – This leverages our Multi-Table Indexing engine to facilitate rapid access to data.
  4. Data Manager – Coordinates reading and writing of data to files.
  5. Buffer Manager – Manages the machines' local cache on each node to improve efficiency and performance. It coordinates with the Global Buffer Manager on the Cluster Manager to ensure that each node is aware of changes made by other nodes.
  6. Log Manager – Maintains the local log, which is used for rolling back uncommitted transactions and also for failure recovery.
  7. Lock Manager – Manages local lock management and coordinates with global locking. This insures, among other things, that no two nodes are changing the same information at the same time.
  8. Recovery Manager – Coordinates with the Global Recovery Manager to ensure that upon failure of the node, it can be recovered gracefully.
  9. Storage Manager – This coordinates the flushing of the data to disk.

For Each Centralized Cluster Manager:

  1. Global Buffer Manager – Orchestrates the interactions of the various local Buffer Managers on each node in the cluster.
  2. Global Lock Manager – Ensures that the various nodes in the cluster are orchestrated in their efforts to establish and release locks the data. ScaleDB implements row-level locking.
  3. Global Recovery Manager – Orchestrates the recovery process in case of a node failure such that the data integrity is maintained throughout and after the recovery process.
  4. CAS Storage Expansion – Manages the addition of CAS pairs and the redistribution of data among the nodes

For Each Cache Accelerator Server (CAS):

  1. CAS Interface – Provides the interface between the Cluster Manager and the various database nodes in the cluster, without the need for a cluster file system
  2. CAS Cache – Provides the shared cache, which enables the database nodes to share data while minimizing disk utilization. Both the cache and storage in the CAS nodes are mirrored for high-availability
  3. CAS Storage – Handles the storage for the CAS and interfaces with the cluster manager for redistributing data when additional CAS pairs are added to the storage tier.
  4. CAS Processing – Provides local processing of data in a manner that is analogous to Oracle Exadata