A few days ago, the seventh ByteDance Technology Salon held by ByteTech, the ByteDance technology community, came to a successful conclusion. The theme of this salon is “Byte High-Performance Open Source Microservice Framework: CloudWeGo”.In the salon, senior R&D engineer of ByteDance ByteDance infrastructure service frameworkYang Rui, I shared with you “Unified Open Source Practices Inside and Out of High-Performance RPC Framework Kitex”, this article is organized according to the sharing. >

This article will introduce the practice and open source of CloudWeGo’s high-performance RPC framework Kitex from the following four aspects:

1. Inside-Out – Open Source Transition;

2. One-year change review of open source;

3. Community co-construction and improvement of ecology and enterprise landing;

4. Summary and Outlook.

Many students may have just learned about CloudWeGo. Let’s first introduce the relationship between CloudWeGo and Kitex.

CloudWeGo and Kitex

Kitex is the first microservice framework open sourced by CloudWeGo, it is aGolang RPC framework with multi-protocol supportfrom the network library, serialization library to the implementation of the frameworkBasically completely self-developedof.In particular, Kitex’s support for the gRPC protocol uses the official gRPC source code, but we doDeep and custom optimization, so the performance of the gRPC protocol supported by Kitex is better than the official gRPC framework. At the same time, this is also the main difference between Kitex and other Golang frameworks that have been open sourced and support the gRPC protocol. If users want to use gRPC and have high performance requirements, the Kitex framework will be a good choice.

Following the open source of Kitex, this year CloudWeGo has successively open sourced the Golang HTTP framework Hertz and the Rust RPC framework Volo. At the same time, we have also open sourced some high-performance basic libraries around these microservice frameworks and some general capabilities of microservices. For more CloudWeGo open source sub-projects, you can go to the CloudWeGo official website to learn more.

CloudWeGo official website: https://www.cloudwego.io/

According to the feedback from community students, in some open source groups, everyone will discuss whether Kitex will be an open source KPI project of ByteDance? Can its stability and continuity be guaranteed? I can say responsibly that Kitex is not a KPI project, it is a real project from large-scale practice within ByteDance. After Kitex is open source, it has always maintained internal and external unity. Based on the unification of internal and external codes, we ensured the continuous iteration of Kitex. In order to further eliminate everyone’s concerns, the following is a detailed introduction to the birth and open source process of Kitex.

Kitex development history

In 2014, ByteDance began to introduce Golang. In 2015, ByteDance’s internal serviceization started. In the RPC call scenario, the Thrift protocol is selected, and the RPC framework is supported internally. In 2016, the first Golang RPC framework Kite was officially released. Usually in the early stage of a company’s rapid development, the basic capabilities are all to quickly support the implementation of requirements, and the demand scenarios faced are relatively simple, and there will not be many considerations in the design. In fact, this is also reasonable, because the exploration stage is not completely clear. Which scenarios still need to be supported, too much consideration will lead to the problem of over-design.

However, with the complexity of business scenarios and the diversification of requirements, and the number of access services and calls increasing year by year, Kite is no longer enough to support subsequent iterations. After serving online for more than three years, we started a new project in 2019. Kitex, the official version was released in early 2020, and by the end of 2020, 1w+ services have been connected to Kitex.

From 2014 to 2020, Golang has been the main business development language within ByteDance, and it should be the company with the most Golang applications in the industry. Our service framework supports the reliable communication of tens of thousands of Golang microservices. After the verification of a large number of microservices and massive traffic, we already have relatively mature best practices for microservices, so we consider open source internal practices to enrich them The Golang product system of the cloud native community. In 2021, we officially open sourced Kitex, our first service framework, under the CloudWeGo brand.As of August this year, Kitex has provided internal services for ByteDance 6w+ services to provide support,Peak QPS reaches hundreds of millions.

You may still have questions. A complete microservice system is inseparable from the basic cloud ecosystem. Whether in public cloud or private cloud, additional services need to be built to support the governance of microservices, such as governance platform, registry, Configuration centers, monitoring, link tracking, service meshes, etc., and there are also some custom specifications. ByteDance naturally also has complete internal services to support the microservice system, but these services cannot be open sourced in the short term, so how does CloudWeGo maintain a set of codes inside and outside, and iterate uniformly?

Regarding this issue, let’s take a look at the module division of Kitex. The modules of Kitex are divided into three parts: the middle part is the main part of Kitex Kitex Corewhich defines the hierarchy of the framework, the implementation of the core logic of the interface, and the default implementation of the interface; the left Kitex Tool It is the implementation related to the generated code. Our generated code tool is obtained by compiling this package, including IDL parsing, verification, code generation, and plug-in support.However, in order to facilitate the use of users and provide more friendly extensions, the main capabilities have also been split as the base library to be independently open sourced, such as Thriftgo, Thrift-validator plugin, Fastpb; Kitex Byted It is an extended implementation of the integration of the internal basic capabilities of Bytes. We converged the internal capabilities as extensions into a package at the beginning.

In this way, we can open source the Kitex Core and Tool parts. We split the code, migrated the core code and tools of Kitex to the open source library, and integrated the internal extension module as an extension of Kitex and kept it in the internal library. At the same time, the internal library encapsulates a shell to ensure that internal users can upgrade without awareness.

So is the open source of Kitex just as simple as code splitting? Obviously not. In February 2021, we began to prepare for the open source of Kitex. Although based on the scalability of Kitex, we can decouple the ability to integrate with the internal infrastructure, but Kitex still relies on some internal basic libraries. If you want to open source, you must first open source the basic libraries. ability.Therefore, we first sorted out the dependent libraries, and cooperated with related students to open source first. bytedance/gopkg library. This library is maintained by CloudWeGo and the language team of ByteDance. It also includes enhancements to the Golang standard library capabilities. Interested students can pay attention to use.

bytedance/gopkg: https://github.com/bytedance/gopkg

After the gopkg library is open source, we adjust the code for open source adaptation. In July 2021, Kitex will be officially open sourced, and the open source library will be used in the internal release. However, Kitex supports tens of thousands of internal microservices after all. We must ensure that the internal services can transition smoothly after this change, so we did not officially announce it to the outside world at the beginning of the open source. After confirming the stability,In September 2021, Kitex officially announced the open source.

Introduced the history of Kitex’s birth and open source, hoping to relieve external students’ concerns about “will Kitex be a KPI project?”

The value of open source

At the end of the first part, briefly talk about the value that open source can bring us.Kitex is not implemented for open source, but its implementation is open source. Kitex itself is an internal large-scale implementation project. We hope that Kitex can help more users to quickly build microservices internally after open source. At the same time, open source can allow us to collect more feedback from communities and enterprises, and also attract external developers. Build, promote the evolution of Kitex for multi-scenario support, enrich product capabilities, and then be implemented in more scenarios and enterprises. This is a positive cycle and a process of mutual benefit and win-win.

Framework Metrics

Before introducing the one-year change of Kitex open source, let’s share the metrics of the framework, which is what everyone should consider when choosing a framework.

If a framework is strongly coupled with its internal capabilities, it cannot be ported to other platforms, or the framework’s support scenarios cannot be extended, making it difficult for such a framework to be used externally.

The ease of use of the framework is reflected in two aspects.The first is forbusiness developer, if a framework requires users to pay attention to many details of the framework during use, it may not be acceptable for teams that require high R&D efficiency.The second is toSecondary developer of the frameworkthey need to do some custom support for the framework. If the extension capabilities provided by the framework are too broad, the expansion cost is high, or the extensible capabilities are not enough, then the framework also has limitations.

Although the framework can be customized based on its extensibility, not all developers have enough energy to do customized development. If the framework itself provides different support for various extension capabilities, developers only need to base on their own foundation. Facilities can be combined to operate in their own environment.

The first three points are the indicators that need to be focused on in the initial selection of the framework, but as the service scale and resource consumption increase, performance becomes an issue that cannot be ignored. From a long-term perspective, you must pay attention to performance when choosing a framework, otherwise you will only face the problem of framework replacement in the future, or you will be forced to do custom maintenance for the framework.

Regarding the measurement indicators of the above four-point framework, although Kitex has not yet achieved the best results, these four elements have been taken into account in Kitex’s design and implementation, and we will not lose sight of one or the other.

Features

The following is an introduction to several important features of open source over the past year.

Proxyless Proxyless

It is the support provided by Kitex for open source scenarios. In the early days of Kitex open source, we internally discussed whether to support xDS to connect with Istio. For external users, using Istio can quickly build a basic micro-service architecture to solve problems such as service discovery, traffic routing, and configuration delivery, but if To use a complete Istio solution, it is necessary to introduce Envoy, which will increase the operation and maintenance cost, and directly using the official Envoy solution will degrade performance, introduce additional CPU overhead and increase latency.If Kitex can directly connect to Istio, users can not only enjoy some of Istio’s capabilities, but also avoid the performance loss and deployment and operation costs caused by Envoy. However, in the early days of open source, we did not see clear user demands, so we did not provide high-quality support for this.

Later, gRPC officials also released Proxyless support, and Istio officials also used Proxyless as a way to use Istio. Kitex has also completed support now. Currently, it is mainly for docking service discovery. The extensions supported by xDS are open sourced separately. kitex-contrib/xds In the library, it will be improved in the future. You can learn how to use Kitex to connect to Istio according to the README.

xDS Support: https://github.com/kitex-contrib/xds

JSON and Protobuf generalization call support

Previously, Kitex supported HTTP generalization applied in gateway scenarios, as well as Map and binary generalization applied in some general service scenarios. After the open source, the generalization of JSON and Protobuf has been added according to the feedback of users.

The generalization of Protobuf is also used in API gateway scenarios. The original data format of HTTP generalization transmission is JSON, but the serialization of JSON is bulky, inefficient, and has an impact on performance. Therefore, many mobile terminal interfaces choose to use Protobuf to transmit data, thus increasing the support for Protobuf generalization.

At present, the generalization of Kitex is mainly aimed at the back-end Thrift service. Whether it is Protobuf, Map or JSON, Kitex will combine IDL parsing on the calling end, and encode these data mappings into Thrift packets and send them to the back-end service.

So why put generalization on the caller side instead of the server side? The generalization that is widely known is that the server parses the generalization request, and of course the caller also provides the generalized Client accordingly. However, generalization is oriented to general services, and the cost of generalization is actually relatively high. It is not suitable for ordinary RPC scenarios, and general services are oriented to all back-end services, such as Golang/Java/C++/Python/Rust , if every language framework supports generalization, the cost is very high. Even if each language supports generalization, the convergence of the framework version is a long process. For general services, it is unrealistic to connect all services. For the above reasons, generalization is supported on the calling side.

Enhanced retry capability

Kitex already supported retry when it was open sourced last year. There are two types of retries previously supported, one is timeout retry, and the other is Backup Request.

For retrying overtime, we will only retry the timeout exception, but in order to further improve the request success rate, the user wants to retry other exceptions, or the user may define some user request status code, combined with the user status code to retry, in this case, it is obvious that we only support timeout retry, which does not meet user needs. Based on this background,Kitex added retry with specified resultthe user can specify other exceptions or specify a certain type of Response, and the framework will retry with the results specified by the user.

Secondly, when the user configures the retry, if the retry is set by code configuration, it will take effect on all RPC methods of the entire Client, but the user wants to apply different retry strategies for different RPC methods, even the same method. It is hoped that different retry strategies can be adopted, because requests for the same method initiated on different links also have different metric requirements. For example, some people want to use Backup Request to reduce delay, and some want to do abnormal retry to improve the success rate. In this case,The new version of Kitex supports request granular configuration retry.

The figure below is an example of usage.Take the request granularity retry configuration as an example. For example, the RPC method is Mockthen when we initiate an RPC call, we can configure a later callopt Specify a retry policy, which will be used for this request.

Thrift Validator

Thrift-gen-validator is a tool plug-in for Thriftgo, which can describe constraints according to the annotations defined in Thrift IDL to the corresponding struct generate IsValid() error method to check the validity of the value. Usually, when making RPC calls, users may verify the validity of some fields. If users directly write these verification codes, the investment cost will be very high. So we provide annotation support,As long as the user defines the annotation according to the specification in IDL, Kitex can help the user to generate the verification code.

The following figure is the code generation command and an example of IDL annotation definition. Specify the Thrift Validator plug-in when generating code, and our plug-in tool will parse the annotation and generate this set of validity verification code for the user. We are also currently contributing Thrift Validator functionality to Apache Thrift.

performance optimization

After introducing several important functional features, we will introduce some performance optimization features.

Thrift high performance codec

Frugal is a high-performance JIT-based dynamic Thrift codec without codec generation. Although we have optimized the official Thrift codec and supported FastThrift, which was also introduced in our optimization practice released before open source, but we hope to have further performance improvement, refer to the design of our open source high-performance JSON library Sonic , which implements the Thrift JIT codec. The table in the figure below is the performance comparison of Frugal combined with Kitex and FastThrift.

It can be seen that the RPC performance is better in most scenarios. In addition to the performance advantage, Frugal has another advantage in that it does not need to generate codecs to generate code. The generated code of Thrift is heavier than that of Protobuf. A complex IDL code generation file can reach tens of thousands of lines, and these codes do not need attention from users, but need to be maintained by users. Frugal only needs to generate structure code without generating codec code, which greatly solves this problem.

For how to use Frugal in Kitex, you can refer to the repository’s Readme. Of course, users can also use Frugal alone as the Thrift high-performance codec, and Kitex will consider using Frugal by default in the future.

Frugal: https://github.com/cloudwego/frugal#readme

Protobuf high performance codec

Although we mainly support Thrift internally, after open source, we found that external users will pay more attention to Protobuf or gRPC, so referring to the optimization ideas of Kitex FastThrift, we re-implemented the generated code of Protobuf. In version v0.4.0, if users use Kitex tools to generate Protobuf code, Fastpb codec code will be generated by default, and Kitex will also use Fastpb by default when initiating RPC calls.

The figure below shows the performance comparison between Fastpb and the official Protobuf serialization. It can be seen that whether it is encoding or decoding, Fastpb is far superior to the official Protobuf serialization library in terms of efficiency and memory allocation.

gRPC performance optimization

In the early days of open source, we paid less attention to the overall stability and performance of gRPC. Because there are not many scenes used internally. After the open source, we received a lot of feedback from external students, so we made a special problem management and performance optimization for gRPC. In the middle of this year, we have officially submitted the relevant optimization to the open source library and released it in the v0.4.0 version.

Kitex v0.4.0: https://mp.weixin.qq.com/s/ezifbQkHcZQP6MygmJABYA

The left side of the figure below is the pair of Kitex-gRPC and the official gRPC framework before optimization Unary requestCompared with the stress test throughput of 2000, Kitex’s throughput is not advantageous when the concurrency is relatively low. When using Fastpb, Kitex’s throughput performance will be better, but the low-concurrency throughput is still lower than the official gRPC. After optimization, the throughput comparison is shown on the right.Compared with before optimization, the throughput is increased by 46% – 70%, and compared with the official gRPC framework, the throughput is 51% – 70% higher.

The right side of the figure below is after optimization Unary requestThe latency comparison of Kitex is also significantly lower than the official gRPC when the throughput is much higher than the official gRPC. At the same time, as far as Kitex itself is concerned, the latency performance is much better after optimization.

Let’s look again Streaming requestsCompared with the pressure measurement performance of the previous optimization, the performance of the Streaming request before optimization is also in the case of low concurrency, which has no advantage over the gRPC framework. After optimization, the throughput of Kitex is significantly higher than the official gRPC, as shown in the figure below. At the same time, the throughput is high under low concurrency but the delay is flat. After increasing the concurrency, you can see the fork in the delay. Therefore, in terms of performance, the gRPC protocol supported by Kitex has obvious advantages over the official ones.

Although Kitex has not been fully aligned in some functions, it can meet the needs of most scenarios at present, and we will continue to align functions in the future.

The Kitex extension ecosystem built by the community

After the open source, we are very pleased to have received the attention of many developers. Frankly speaking, the internal team has limited energy and cannot quickly establish a Kitex extension ecosystem for external users. However, with the help of the community over the past year, Kitex has made a lot of additions in the expansion of service registration/discovery, observability, and service governance, especially the expansion of service registration/discovery. Currently, the mainstream open source registries have been completed. Docking, although we still need to strengthen the function richness, but combined with the existing support, we already have the ability to build a microservice architecture for external users.

Sincere thanks to the students who actively participated in the construction of the CloudWeGo community! For the ecological support related to Kitex, you can go to Kitex-contrib to learn more about open source repositories.

Kitex-contrib: https://github.com/kitex-contrib

Connect with external companies and assist in landing

The original intention of our open source is to help other external companies quickly build an enterprise-level cloud-native architecture. After the open source, Semir, Huaxing Securities, Ganwan Games, and Heduo Technology took the initiative to contact us successively, feedback usage problems and put forward requirements. Indeed, we found some problems that are different from the internal scene, and we need to pay attention, support and optimize. , we are very happy that Kitex can be used in these enterprises. In the CloudWeGo Meetup on June 25 this year, the R&D students of Semir and Huaxing Securities also shared their internal practice of using Kitex.

Semir: https://mp.weixin.qq.com/s/JAurW4P2E3NIduFaVY6jew

Huaxing Securities: https://mp.weixin.qq.com/s/QqGdzp-7rTdlxedy6bsXiw

In addition to the above companies, there are also some companies who have privately consulted us about usage issues. We are very grateful for the support of these corporate users and the feedback they gave us. As mentioned in the first part, collecting feedback from the community and enterprises can promote the evolution of Kitex to support multiple scenarios. Enterprise users are welcome to contact us if they have related needs.

How to use Kitex to integrate with internal infrastructure

Here is a brief introduction to how to use Kitex to integrate with your internal infrastructure. Taking byte internal as an example, there are extended implementations in open source libraries in the internal warehouse to integrate internal capabilities. In bytedSuite, we initialize Kitex for different scenarios. As shown in the code example below, the user only needs to add an option configuration when constructing the Client and Server to complete the integration. However, in order to let the user not need to pay attention to the integration of internal capabilities, we put this configuration in the generated scaffolding code. Regarding how the configuration is embedded in the generated code, we will also open it up in the future, so that secondary developers of external frameworks can provide integration capabilities for business development students in the same way.

Summarize

This sharing mainly introduces the following contents:

How Kitex maintains a unified internal and external transformation from a widely used internal framework to an open source framework;

What important features and performance optimizations have been released since the open source one year ago;

With the power of the community, what is Kitex’s surrounding ecology, how the company has landed, and how to use Kitex to elegantly integrate internal capabilities.

Outlook

Build with community classmates and continue to enrich the community ecology;

Combined with engineering practice, it provides more convenience for microservice developers;

Improve the BDThrift ecosystem and continue to optimize Protobuf/gRPC;

More feature support or open source, ShmIPC, QUIC, Protobuf generalization…

The above content is compiled from the seventh issue of ByteDance Technology Salon “Byte High-performance Open Source Microservice Framework: CloudWeGo”, obtainedInstructor PPT and playback video,Please pay attention CloudWeGo Official Accountand reply keywords in the background “anniversary”.

GitHub: https://github.com/cloudwego

Official website: www.cloudwego.io