On December 22, 2022, the Apache Software Foundation (ASF) officially announced that Apache Kyuubi has officially graduated as a top-level project (TLP).

Apache Kyuubi is a distributed and multi-tenant gateway for serverless SQL on data warehouses and lake warehouses.

The project was originally developed by Netease Shufan and open-sourced in 2018. It donated to the Apache Foundation in June 2021. After more than a year of incubation, it was voted in November 2022. It graduated successfully in December and became a Apache Foundation Top Open Source Project!

Introduction to Apache Kyuubi

Apache Kyuubi builds distributed SQL query engines on various modern computing frameworks, such as Apache Spark™, ​​Apache Flink™, Apache Doris™, Apache Hive™ and Trino, etc., to queryquery distribution in heterogeneous dataLarge-scale datasets on the source machine farm.

Unified Gateway:Simplified, secure access to any cluster resource through a single point of entry to deploy diverse workloads to end users

• answerUser programming interface: supports various APIs, including Apache Thrift™, JDBC, ODBC, REST, etc., for easy access.

• Multi-tenancy: supports end-to-end multi-tenancy, which is beneficial to the security and concurrency of the cluster.

• High Availability: Ensuring its continuous, trouble-free operation for a specified period of time to meet agreed levels of operational performance.

Serverless SQL and more:Make it easier for end users to gain insights from the data universe and optimize data pipelines, regardless of their technical knowledge. It can use familiar SQL to provide the same user experience as RDBMS for various workloads, provide extensive and secure data access capabilities on different data sources, and provide high performance for large amounts of data through scalable computing resources.

• Ease of use: End users can have an optimized experience to explore their data universe in a serverless manner. The “superpowers” of corresponding engines such as Spark and Flink are no longer necessary.

• Run anywhere at any scale: All pre-programmed engines have distributed backends that can schedule tasks on single-node machines or across clusters.

• High performance: The most advanced query engine, global and continuous optimization on the server side ensure the performance improvement of the entire cluster.

community development

During the incubation process, the community ushered in more than a hundred contributions, thousands of submissions, and a total of nine successful releases. Developers and other types of contributors from different companies and countries have had a happy journey in the community.

Kyuubi is currently adopted by hundreds of enterprises around the world, involving multiple industries, such as cloud infrastructure, Internet, finance, medical care, taxi service, logistics, gaming, and ACG, among others. Companies like Alibaba, Bilibili, China Mobile, Ctrip, Dingxiangyuan, eBay, iQiyi, GF Securities, kt NexR, Netease, Tencent, T3, Womply, Xiaomi, Xueqiu, and Zhihu are all using Apache Kyuubi.

“This is an important milestone for Kyuubi, and the graduation also proves its great value in large companies. In the past year, we have successfully adopted Kyuubi as a unified gateway for Ad-hoc queries and batch ETL jobs. kyuubi not only supports Interactive sessions also support batch job submission, which is critical for enterprise users.”Jiaye Wang, eBay Hadoop Team Manager Said, “We look forward to more exciting features from the community in the future.”

“GF Securities has been paying attention to and adopting Kyuubi in the migration of big data platforms. We regard Apache Kyuubi as one of the flagship services of the next-generation unified data platform. Key to shared prosperity for other projects.”Liang Bowen, senior engineer of GF SecuritiesCommented, “As a contributor, it’s also a pleasure to keep making contributions in a supportive community. Excited to see Kyuubi graduate.”

“Graduation will bring confidence to more companies to introduce the Kyuubi framework,”Apache Kyuubi PMC, Yang Hua, Head of T3 Big Data Platform R&Dadded. “In the future, we will try to integrate more big data query/calculation engines with Kyuubi, and at the same time, connect it with internal data maps to enhance its authorization capabilities,Make it really become the largest number of our companyUnified gateway for data. “

Graduation message (Alphabetically by last name)

“very highIt’s great to see another Chinese-led Apache project graduate from the incubator. Kyuubi provides a Severless SQL accelerated analysis experience, which facilitates enterprises to easily access various data sources. As a brother project of Apache, Kyuubi also provides support for Apache Doris JDBC connector, welcome to use and give feedback. I also wish Kyuubi can write a new chapter in the open source arena! “

—Chen Mingyu,Apache Doris PMC

“Congratulations to Kyuubi for successfully graduating from the Apache incubator and becoming a top-level project. A new milestone has been born. As an excellent distributed multi-tenant gateway in the field of big data, we look forward to more prosperous future development!”

— Dai Lidong,Apache Dolphin Scheduler PMC Chair & Apache SeaTunnel PPMC

“We’re excited to see that Apache Kyuubi is becoming a popular choice for those who want to provide a serverless SQL engine on Big Data. It’s great to see it grow into a top-tier project with an open, collaborative, and diverse community.”

— Jiang Ning,Champion and Mentor of Apache Kyuubi

“Congratulations to Apache Kyuubi for graduating! It has been a pleasure to work with the Apache Kyuubi community in the incubator! The team is very responsive, responsive and open to new ideas. I believe they are now working as a top-level project Will continue to build this great project with even greater enthusiasm!”

—Martin Grigorov,Member of the Apache Software Foundation

“At the beginning of Kyuubi’s open source, Xiaomi paid attention to Kyuubi and began to use it in depth. Apache Kyuubi, as a unified gateway, serves Xiaomi’s new generation data platform and plays an important role in the evolution of the integrated architecture of lakes and warehouses. Xiaomi and Apache Kyuubi The community has also maintained in-depth cooperation, and we have participated in and witnessed the development of the Kyuubi project. Here, I sincerely wish the Apache Kyuubi project and the community to develop better and better!”

—Qin Zuoyan,Chairman of Xiaomi Group Open Source Committee, member of Open Atom Foundation TOC

“I am very glad to see that the Kyuubi project contributed by Netease Shufan to the Apache Foundation has officially graduated. Being able to graduate and become the top Apache project shows that the Kyuubi project has developed into an active and sustainable community governance project after the donation. It also shows that Kent The project sponsors are constantly learning and mastering the practice and essence of The Apache way. The Kyuubi project has a good idea, and the selected technical pain points are very precise. Now it has passed the test of Apache incubation and graduation, and the IP problem has been well verified. The community Health and persistence have also been well confirmed. I believe its graduation can be adopted by the industry more, congratulations to the Kyuubi project and community!”

— Tan Zhongyi,Deputy Secretary of China Open Source Promotion Alliance, Vice Chairman of TOC of Open Atom Open Source Foundation

“Kyuubi is a widely used SQL portal. Its serverless design supports common SQL drivers, which greatly improves the convenience of using big data technology. As a portal and gateway between clients and underlying technologies, Kyuubi also has great potential. Imagination space for functional extensions, such as SQL auditing, inspection, optimization based on query history, etc. This is a project with great potential.”

— Wang Yuan,Vice President of NetEase, Executive Director of NetEase Hangzhou Research Institute

“Congratulations to Apache Kyuubi for becoming Apache’s top open source project. Open source is like this. The joy of one party sets off the sadness of the other. The extreme emotions of human beings are fully reflected and released at this moment. This is Apache Kyuubi, which is why we are so The reason why I love the Apache Kyuubi project!”

— Jiang Xiaofeng,Senior Development Engineer, Infrastructure Department, bilibili

“It’s been fun and exciting to see the community grow and graduate from the incubator, which means we’ve successfully adopted the ‘Apache Way’ and are able to manage ourselves. As an Apache top-level project, Kyuubi is supported by people from all over the globe and organizations. Kyuubi helps them democratize data so that people of any skill level can comfortably work with data, regardless of their technical knowledge.”

— Yao Qin,VP of Apache Kyuubi

“Congratulations to Kyuubi for graduating and becoming the first Apache project incubated by Netease Shufan! Welcome more friends to join Kyuubi, the community is getting better and better.”

— Yu Lihua,General Manager of NetEase Shufan Big Data Product Line

Apache Kyuubi software is released under the Apache License v2.0 and actively contributed by a self-selected projectteam supervision.The Project Management Committee (PMC) guides the day-to-day operations of the project, including community development and product releases.
For downloads, documentation and participation in the Apache Kyuubi community please visit the following address:

【Official website address】
https://kyuubi.apache.org

【download】
https://kyuubi.apache.org/releases

【Documentation】
https://kyuubi.apache.org/docs/latest/index.html

【Github】
https://github.com/apache/kyuubi

See more Apache Kyuubi in action at future Kyuubi community events.

For more information and to register, please visit:

https://apachecon.com/?ref=kyuubi.apache.org

last of the last

Thanks to every community developer and user for every contribution!