Arroyo is a distributed stream processing engine written in Rust designed to efficiently perform stateful computation on streams of data. Unlike traditional batch processing, streaming engines can operate on bounded and unbounded sources and emit results as soon as they are available.

Transform, filter, aggregate, and join Kafka streams using SQL and get results in sub-seconds. Automatically scales from 10 events per second to millions of events.

In short: Arroyo lets you ask complex questions of massive amounts of real-time data with sub-second results.

characteristic:

• SQL and Rust pipelines
• Scales to millions of events per second
• Stateful operations such as windows and connections
• State checkpointing for pipeline fault tolerance and recovery
• Through the data flow modelJust-in-time stream processing

Some example use cases include:

• Detect fraud and security incidents
• Real-time product and business analysis
• Real-time ingestion into your data warehouse or data lake
• Real-time machine learning feature generation