Xline is a high-performance distributed key-value storage engine for multi-cluster. It can provide unified data management for multi-cluster scenarios, making mutual access, discovery, and modification easy and convenient. It also provides KV interface, multi-version concurrency control and is compatible with etcd and K8S.

Xline is the first geographically distributed consensus management service based on CURP (a WAN consensus protocol, read the paper for more details). It addresses the challenges of convergence and consistency across clouds.

It provides the following functions:

• etcd compatible API.
• Geographically distributed friendly deployment.
• Compatible with K8s.

The project is aimed at multi-data center scenarios, aiming to realize a high-performance multi-cloud metadata management solution, which is crucial for enterprises with geographically distributed and multi-active deployment requirements.

An Xline instance consists of the following layers:

• Client layer: Provide a simple and easy-to-use API for client use, which can greatly reduce the complexity of using Xline for business. Subsequent versions will implement Xline clients in different languages. Currently, Xline API is compatible with etcd, and you can use etcdctl to initiate requests.
• Access layer: The access layer mainly includes the communication protocol between client and server or server to server. The Xline API is based on the gRPC protocol.
• CURP protocol layer: The CURP protocol layer implements core algorithm functions such as Leader election, log replication, fast path and slow path, and is used to ensure data consistency between Xline multi-nodes and improve service availability. The CURP protocol is the cornerstone and highlight of Xline.
• Functional logic layer: This layer implements Xline business logic, including typical KV Server, Auth Server, Lease Server, and Watch Server. Client sends requests to Xline Server through the access layer, and Xline Server dispatches requests to specific servers for execution.
• Storage layer: This layer consists of two components, Index and DB, where Index is based on BTreeMap, and DB is mainly responsible for the persistent storage of data. At present, Xline is still in the early stage of development, so DB is mainly implemented based on memory. We will introduce persistent storage in the next version.

main feature:

• A distributed KV storage engine for multi-cluster.Unified management of data in multi-cluster scenarios, mutual access, discovery, and modification are simple and convenient.
• Unified metadata management system.By caching hotspot data in memory for cross-cloud data access, and providing unified data management, it realizes the automation of data migration and backup.
• High-performance multi-data center consensus protocol.It is the first geographically distributed consensus management service based on a WAN consensus protocol. It addresses the challenges of convergence and consistency across clouds.
• Compatible with ETCD interface.Provide KV interface, multi-version concurrency control, compatible with K8S.