Velox isA C++ vectorized database acceleration library designed to optimize query engines and data processing systems.Provides reusable, scalable and high-performance data processing components. These components can be reused to build compute engines focused on different analytical workloads, including batch, interactive, stream processing, and AI/ML. Velox was created by Facebook and is currently being developed in partnership with Intel, ByteDance and Ahana.

In common usage scenarios, Velox takes a fully optimized query plan as input and performs the described computation. Considering that Velox does not provide a SQL parser, data frame layer, or query optimizer, it is generally not intended to be used directly by end users; instead, it is primarily used by developers integrating and optimizing their computing engines.

Velox provides the following advanced components:

• Type: A generic type system that supports scalar, complex and nested types such as structs, maps, arrays, tensors, etc.
• Vector:and Arrow Compatible Columnar Memory Layout Modulein addition to the lazy implementation pattern and support for out-of-order writes, offers multiple encodings such as Flat, Dictionary, Constant, Sequence/RLE, and Bias.
• Expression Eval:OneFully vectorized expression evaluation engine allowing expressions to execute efficiently on top of vector/arrow-encoded data.
• Function Packages: A set of vectorized function implementations that follow Presto and Spark semantics.
• Operators : Implementation of common data processing operators, such as scan, projection, filter, groupBy, orderBy, shuffle,hash joinunnest, etc.
• I/O: Generic connector interface that allows the use of different file formats (ORC/DWRF and Parquet) and storage adapters (S3, HDFS, local files).
• Network Serializers: Interfaces that can implement different wired protocols for network communication, support PrestoPage and Spark’s UnsafeRow.
• Resource management: A collection of primitives for processing computational resources, such asMemoryRegion and buffer management, tasks, drivers, and thread pools for CPU and thread execution, overflow, and caching.

Velox is extensible, allowing developers to define their own engine-specific specializations, including:

1. custom type
2. Simple and vectorized functions
3. Aggregate function
4. operator
5. file format
6. storage adapter
7. network serializer