Katalyst is ByteDance’s abstraction and summary of resource management capabilities in large-scale business cloud-native scenarios for many years. We expect to use Katalyst’s open source to directly or indirectly help users manage resources and achieve cost reduction and efficiency increase.

The current Internet application is based on the unit of day, and the resource usage of online business often changes with the fluctuation of the number of visits, which has obvioustidalcharacteristic. In order to ensure business stability, business parties often apply for resources with reference to resource usage during peak hours, but these resources are easy to be idle during off-peak hours.

If these idle resources can be temporarily transferred to low-priority services, and the resources are returned in time when the online business needs to be used, the formation ofMixing department in offline servicecan achieve the effect of cutting peaks and filling valleys and saving costs#.

Bytedance’s business scale is huge and its business types are diverse, covering a variety of business types including microservices, promotion and search services, machine learning and big data, and storage. Generally speaking, different business types have different resource management requirements for the underlying infrastructure. The traditional management model is to segment resource pools based on business lines or service types to achieve customized requirements.

However, the practice of dividing resource pools is easy to formresource islandit is impossible to realize flexible borrowing at the resource level, which is not conducive to the improvement of global resource utilization efficiency and the optimization of business costs, and increases the burden of cluster operation and maintenance.

In addition, due to the SLO requirements of different types of business and the characteristics of resource tides are complementary, the infrastructure team expects to make full use of these characteristics to optimize resource efficiency through scheduling, management and control, etc., to achieve the integration and unification of resource pools, and to help business teams obtain lower costs. Resource costs and greater resilience.

To achieve unified resource hosting, ByteDance has been building a unified infrastructure based on Kubernetes since 2016.

At this stage, Byte has basically completed the cloud-native transformation of full microservices, popular search services, and most machine learning and big data businesses. During this process, the infrastructure team continued to explore resource optimization methods under the unified resource pool, and gradually formed the “elastic expansion” and”Normal mix” A resource pool mix plan that cooperates with each other.

• elastic expansion: Realize time-sharing multiplexing of resources at the machine level and Numa level, combine business indicators and system indicators, and jointly guide the horizontal and vertical expansion and contraction strategies of business instances, and ultimately enable offline services to purchase more idle resources at a lower price , Online services purchase more peak-time resources at a higher price, and improve overall efficiency through market-oriented operation of resources.

• Hybrid deployment: Provide the ability to oversell resources, make full use of “resources that have been sold but not fully used” in the cluster to deploy more low-quality services, and improve the multi-dimensional resource isolation mechanism of CPU, memory, disk, and network on the system side, and Intelligently predict and perceive the load changes of various services, combined with the service classification mechanism, through minute-level index perception and control strategies, to ensure service stability.

The solution implements joint management and control based on the two systems of Kubernetes and Yarn on the link, runs the management and control components of Kubernetes and Yarn on a single machine at the same time, and cooperates with the central coordination component to allocate the amount of resources visible to the two systems. Based on the joint management and control system, the team realizes real-time resource estimation based on service resource portraits, and realizes more flexible and dynamic resource allocation under the premise of ensuring various service SLA requirements.

During the implementation of the resource pool mixed department solution, the infrastructure team completed the verification of the feasibility of pooling resources, completed the construction of the basic capabilities of the mixed department, and achieved a day-level utilization rate of the whole machine in some core business clusters from 23% to 60% improvement.

After undergoing large-scale tidal traffic business verification such as internal Douyin and Toutiao, ByteDance’s cloud-native hybrid practice has been perfected day by day.

In order to help more people understand the working principle of large-scale resource mixing practice, and facilitate more developers and users to experience this out-of-the-box, one-click deployment of resource management and control capabilities, the R&D team decided to give back to the community by adopting the Kubernetes Native approach Refactored and enhanced the implementation of the resource management and control system, and refined the resource management and control system Katalyst And officially open source.

Katalyst is derived from the word catalyst (sound [ˈkætəlɪst]), originally intended as a catalyst. The initial letter was changed to K, implying that the system can provide more powerful automatic resource management capabilities for all loads running in the Kubernetes system.

What is Katalyst

Katalyst was born out of ByteDance’s hybrid technology practice, and it also expands and supplements resource management and control capabilities from multiple dimensions such as resources, management and control, and scheduling. Its main features include:

• fully hatched atsuper large scaleMixed practice, and synchronously take over the resource management and control link in the process of cloud-native byte service, and truly realize the reuse of internal and external technical systems
• Enhanced Kubernetes, a Kubernetes distribution inside ByteDanceSynchronous open sourcegood compatibility, experience more core functions self-developed by Byte
• The system is built based on a plug-in model, and users can customize various module plug-ins such as scheduling, control, strategy, and data on the Katalyst Framework
• Provide one-click deployment templates and detailed operation and maintenance manuals to reduce external users’ understanding and access costs

How Katalyst Implements Resource Abstraction

At the resource layer, Kubernetes’ native QoS classification cannot meet the requirements of a large-scale production environment, and Katalyst further abstracts it on this basis.

Taking the CPU as the main dimension, Katalyst provides applications with system_core system cores, dedicated_core exclusive cores, shared_core shared cores, reclaimed_core recovery cores, and other different levels. Determine whether you need network card affinity or bandwidth restrictions, etc.), to achieveDifferentiationresource allocation and management strategies.

Through the abstract resource model, Katalyst provides users with a unified resource entry. According to actual business needs, users map business services to corresponding QoS and sales models to accurately express their own needs, and finally achieve resource acquisition from a unified resource pool. Don’t pay attention to the details of the underlying resource pool.

Architecture design of Katalyst

There were several problems in the early mixed deployment architecture: Although the joint management and control of the two systems of Kubernetes and Yarn realized the normal deployment of offline business, the complex system also made the maintenance cost higher.

In addition, this architecture also brings additional resource loss. On the one hand, these losses come from the resource occupation of the single-machine Agent component in the joint management and control mode, especially in ultra-large-scale clusters, this part of resources is very considerable. In addition, due to the high complexity of the system due to the two sets of controls, a multi-level resource funnel will be generated during the system interaction process, and any abnormality in any link will lead to resource loss.

In Katalyst, we optimized and refactored the overall mixed architecture:

At the management and control layer, we integrated Byte’s early fusion system based on Kubernetes and Yarn into a Kubernetes-based system.

Specifically, we keep the respective API entrances of Kubernetes and Yarn at the access layer, and implement the metadata management and resource control of the underlying system.uniform convergenceTo the Kubernetes-based management and control system Katalyst.

At the scheduling layer, Katalyst implements a resource scheduling mechanism that coordinates between the “center” and “stand-alone” on the basis of unified metadata.

existStand-alone schedulingSide: Katalyst is equipped with the extension module QoS Resource Manager (QRM) in Enhanced Kubernetes, which can realize pluggable micro-topology affinity allocation, and report the micro-topology to the center through a custom CRD to open up the scheduling process; during service operation, Katalyst will continue to observe the Pod runtime system indicators, gather business QoS requirements and business indicator feedback to estimate, determine the allocation of Pods in each resource dimension, and send them to CRI in real time through the QRM reconcile mechanism. The resource estimation model and QRM implementation in the above process can be customized through plug-in, so that the resource control strategy can better match the demands of different business scenarios.

existCentral dispatchSide: Katalyst expands richer scheduling capabilities based on the native Scheduler Framework. During the scheduling process, it also considers how the resource layer should be allocated and collaborated when different QoS services are running in the same cluster, and cooperates with stand-alone scheduling to achieve more fine-grained scheduling semantic requirements. ; At the same time, the central scheduling will also combine the real-time data and service portraits of the business container runtime to implement a dynamic Rebalance strategy within the entire cluster, reduce the cluster vacancy rate, and improve business stability.

By converging resource management and control, Katalyst effectively reduces resource consumption during system operation. In addition, under the joint control system, Katalyst switches the implementation of bypass asynchronous control to the implementation of synchronous control based on the standard CRI interface, so as to avoid service performance jitter caused by various asynchronous races or conflicts.

Finally, under a set of management and control system, we can also fully enjoy the advantages of Kubernetes’ API-oriented design, decouple the internal system by customizing CRD, and generalize the management and control strategy, so that the system can be more flexible through plug-in iterative, truly achievedInternal and external homology.

As a resource management system, Katalyst is one of its core application scenarios in offline mixing.In addition to abstracting the above core concepts, we also provide and plan for KatalystRich QoS Capabilities:

• Refined resource transfer strategy: Katalyst supports a variety of resource estimation strategies based on static heuristics, unsupervised algorithms, and QoS Aware, more accurately calculates and predicts the amount of resources that can be transferred by nodes, and further improves resource utilization.
• Multi-dimensional resource isolation capability: Based on cgroup, rdt, iocost, tc and other capabilities, the effective isolation of cpu, memory, disk, network and other resources in different mixed deployment scenarios can be realized to ensure that the QoS of online services will not be affected.
• Multi-level load eviction strategy: It supports multi-level eviction strategies based on multiple indicators, and improves the QoS of offline services as much as possible while ensuring the QoS of online services.

In addition to mixed deployment scenarios, Katalyst also provides some enhanced resource management capabilities:

• Resource Elasticity Management: Provide flexible and scalable HPA/VPA resource elasticity policies to help users improve deployment rate and resource utilization.
• Microtopology Aware Scheduling: Perceive the micro-topology of CPU, memory, and heterogeneous devices on the node, complete resource allocation based on finer-grained micro-topology information, and meet the QoS requirements of high-performance services.

For detailed function planning, please refer to the roadmap.

Although the hybrid technology has gone through several technical iterations within Byte, a common and standardized platform base must be polished in various scenarios. We are very much looking forward to more friends joining the Katalyst open source community!

Project address: github.com/kubewharf/katalyst-core