background

With the release of Dubbo 3.1, Dubbo has taken another important step on the road to cloud native. In this version, the new feature of Proxyless Mesh is added. Dubbo Proxyless Mesh directly implements xDS protocol analysis, realizes direct communication between Dubbo and Control Plane, and then realizes unified control of traffic control, service governance, observability, security, etc. , to avoid the performance loss and deployment architecture complexity brought by the Sidecar mode.

What is Service Mesh

Service Mesh is also translated as “service grid”, which serves as the infrastructure layer for inter-service communication. Buoyant CEO William Morgan explained what Service Mesh is and why cloud-native applications need Service Mesh in his article WHAT’S A Service Mesh? AND WHY DO I NEED ONE?.

The following is William Morgan’s explanation of Service Mesh.

A Service Mesh is a dedicated infrastructure layer for handling service-to-service communication. 
It’s responsible for the reliable delivery of requests through the complex topology of services 
that comprise a modern, cloud native application. In practice, the Service Mesh is typically implemented 
as an array of lightweight network proxies that are deployed alongside application code, without the 
application needing to be aware.

translate to Chinese

服务网格（Service Mesh）是处理服务间通信的基础设施层。它负责构成现代云原生应用程序的复杂服务拓扑来可靠地交付请求。
在实践中，Service Mesh 通常以轻量级网络代理阵列的形式实现，这些代理与应用程序代码部署在一起，对应用程序来说无需感知代理的存在。

When it comes to Service Mesh, it must be inseparable from the classic architecture of Sidecar. It injects the Sidecar container into the business Pod to take over the communication traffic of the business container. At the same time, the Sidecar container is connected to the control plane of the grid platform. The governance capability of the system is lowered to the Sidecar container, thereby realizing the sinking of the basic framework capabilities and decoupling from the business system.

The classic Sidecar Mesh deployment architecture has many advantages, such as smooth upgrades, multi-language, and less business intrusion, but it also brings some additional problems, such as:

• The performance loss caused by Proxy will become especially obvious in network calls with complex topologies
• Architectural complexity caused by traffic interception
• Sidecar lifecycle management
• The deployment environment is limited, not all environments meet the Sidecar traffic interception conditions

In response to the problem of the Sidecar Mesh model, the Dubbo community has conceived and thought about the direct connection of Dubbo to the control plane since a long time ago, and first proposed the concept of Proxyless Mesh in the domestic open source community. Of course, in terms of the concept of Proxyless, the most It was originally proposed by Google.

Dubbo Proxyless Mesh

The Dubbo Proxyless mode means that Dubbo directly communicates with Istiod, and realizes service discovery and service governance through the xDS protocol.

The Proxyless mode brings microservices back to the deployment architecture of the 2.x era, which is very similar to Dubbo’s classic service governance mode, so this mode is not new, and Dubbo has been such a design mode from the very beginning. Doing so can greatly improve application performance and reduce network latency. Some people say that this approach is back to the original SDK-based microservice model. In fact, it is not true. It still uses Envoy’s xDS API, but because it no longer needs to inject Sidecar agents into the application, it can reduce application performance. loss.

However, compared with the Mesh architecture, Dubbo’s classic service governance model does not emphasize the unified control of the control plane, which is exactly what Service Mesh emphasizes, emphasizing the standardized control and governance of traffic, observability, certificates, etc., which is also Mesh Where ideas are advanced.

In the Dubbo Proxyless architecture mode, the Dubbo process will directly communicate with the control plane, and the Dubbo process will continue to maintain a direct communication mode. We can see the advantages of the Proxyless architecture:

• There is no additional proxy loss, so it is more suitable for performance-sensitive applications
• More conducive to smooth migration of legacy systems
• Simple architecture, easy operation and maintenance deployment
• Suitable for almost any deployment environment

service discovery

The xDS access is docked in the registry mode, and the node discovery is the same as the service introspection model of other registry centers. The load balancing and routing configuration of xDS is transmitted through the dynamic runtime configuration of ServiceInstance, and the configuration parameters are passed in when building Invoker Configure the address.

certificate management

Under the zero-trust architecture, it is necessary to strictly distinguish the identification and trust of workloads, and issuing X.509 certificates is a recommended authentication method. In a Kubernetes cluster, services communicate with each other through DNS names, and network traffic may be hijacked by DNS spoofing, BGP/route hijacking, ARP spoofing, etc. In order to strongly associate service names (DNS names) with service identities, Istio Use a secure naming scheme built into X.509 certificates. SPIFFE is a security naming specification adopted by Istio, and it is also a standardized and portable workload identity specification defined by cloud native.

Secure Production Identity Framework For Everyone (SPIFFE) is a set of standards for identifying each other between services, mainly including the following:

• SPIFFE ID standard, SPIFFE ID is the unique identifier of the service, and the specific implementation uses the URI resource identifier
• The SPIFFE Verifiable Identity Document (SVID) standard, which encodes the SPIFFE ID into an encrypted verifiable data format
• API standard for issuing and revoking SVID (SVID is the identification certificate of SPIFFE ID)

SPIFFE ID specifies a spiffe://<trust domain>/<workload identifier> URI format, as the unique identifier of the workload (Workload). Istio, on the other hand, only uses SPIFFE ID as a security naming in its own ecology. Its data format is implemented by itself, and the communication format adopts the xDS protocol specification supported by CNCF (certificate authentication communication is more specifically the SDS of xDS).

Istio uses the form spiffe://<trust_domain>/ns/<namespace>/sa/<service_account> The SPIFFE ID in format is injected into the subjectAltName extension of the X.509 certificate as a security name. The “trust_domain” parameter is injected through the Istiod environment variable TRUST_DOMAIN for interaction in a multi-cluster environment.

The following is the process of Dubbo Proxyless Mesh certificate issuance

• Create an RSA private key (Istio does not yet support ECDSA private keys)
• Building a CSR (Certificate signing request) template
• Self-Signed CSR Generated Certificate
• Create Kubernetes Secret resource to store CA certificate and private key (CA Service processing)

Case Practice

Next, I will lead you through an example to quickly run existing projects in Proxyless Mesh mode.

Environmental preparation

install docker

Home

install minikube

Wall crack recommendation: https://kubernetes.io/zh-cn/docs/tutorials/hello-minikube/

Install istio

https://istio.io/latest/docs/setup/getting-started/

❗❗❗ When installing Istio, you need to enable first-party-jwt support (add the –set values.global.jwtPolicy=first-party-jwt parameter when using the istioctl tool to install), otherwise it will cause client authentication to fail . The reference command is as follows:

curl -L https://istio.io/downloadIstio | sh -
cd istio-1.xx.x
export PATH=$PWD/bin:$PATH
istioctl install --set profile=demo --set values.global.jwtPolicy=first-party-jwt -y

code preparation

xds-provider

define an interface

public interface GreetingService {

    String sayHello(String name);
}

Implement the corresponding interface

@DubboService(version = "1.0.0")
public class AnnotatedGreetingService implements GreetingService {

    @Override
    public String sayHello(String name) {
        System.out.println("greeting service received: " + name);
        return "hello, " + name + "! from host: " + NetUtils.getLocalHost();
    }
}

Write startup class

public class ProviderBootstrap {

    public static void main(String[] args) throws Exception {
        AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(ProviderConfiguration.class);
        context.start();
        System.out.println("dubbo service started");
        new CountDownLatch(1).await();
    }

    @Configuration
    @EnableDubbo(scanBasePackages = "org.apache.dubbo.samples.impl")
    @PropertySource("classpath:/spring/dubbo-provider.properties")
    static class ProviderConfiguration {
    }
}

Write configuration information

dubbo.application.name=dubbo-samples-xds-provider
# 由于 Dubbo 3 应用级服务发现的元数据无法从 istio 中获取，需要走服务自省模式。
# 这要求了 Dubbo MetadataService 的端口在全集群的是统一的。
dubbo.application.metadataServicePort=20885
# 走xds协议
dubbo.registry.address=xds://istiod.istio-system.svc:15012
dubbo.protocol.name=tri
dubbo.protocol.port=50051
# 对齐k8s pod生命周期，由于 Kubernetes probe 探活机制的工作原理限制，
# 探活请求的发起方不是 localhost，所以需要配置 qosAcceptForeignIp 参数开启允许全局访问
dubbo.application.qosEnable=true
dubbo.application.qosAcceptForeignIp=true

Write Deployment.yml and Service.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dubbo-samples-xds-provider
  namespace: dubbo-demo
spec:
  replicas: 3
  selector:
    matchLabels:
      app: dubbo-samples-xds-provider
  template:
    metadata:
      labels:
        app: dubbo-samples-xds-provider
    spec:
      containers:
        - name: server
          image: apache/dubbo-demo:dubbo-samples-xds-provider_0.0.1
          livenessProbe:
            httpGet:
              path: /live
              port: 22222
            initialDelaySeconds: 5
            periodSeconds: 5
          readinessProbe:
            httpGet:
              path: /ready
              port: 22222
            initialDelaySeconds: 5
            periodSeconds: 5
          startupProbe:
            httpGet:
              path: /startup
              port: 22222
            failureThreshold: 30
            periodSeconds: 10

apiVersion: v1
kind: Service
metadata:
  name: dubbo-samples-xds-provider
  namespace: dubbo-demo
spec:
  clusterIP: None
  selector:
    app: dubbo-samples-xds-provider
  ports:
    - name: grpc
      protocol: TCP
      port: 50051
      targetPort: 50051

Write Dockerfile

FROM openjdk:8-jdk
ADD ./target/dubbo-samples-xds-provider-1.0-SNAPSHOT.jar dubbo-samples-xds-provider-1.0-SNAPSHOT.jar
CMD java -jar -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=31000 /dubbo-samples-xds-provider-1.0-SNAPSHOT.jar

xds-consumer

define an interface

public interface GreetingService {

    String sayHello(String name);
}

Implement the corresponding interface

@Component("annotatedConsumer")
public class GreetingServiceConsumer {
  	// 这里特别注意的是、由于当前 Dubbo 版本受限于 istio 的通信模型无法获取接口所对应的应用名，
		// 因此需要配置 providedBy 参数来标记此服务来自哪个应用。
    @DubboReference(version = "1.0.0", providedBy = "dubbo-samples-xds-provider")
    private GreetingService greetingService;

    public String doSayHello(String name) {
        return greetingService.sayHello(name);
    }
}

Write startup class

public class ConsumerBootstrap {

    public static void main(String[] args) {
        AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(ConsumerConfiguration.class);
        context.start();
        GreetingServiceConsumer greetingServiceConsumer = context.getBean(GreetingServiceConsumer.class);
        while (true) {
            try {
                String hello = greetingServiceConsumer.doSayHello("xDS Consumer");
                System.out.println("result: " + hello);
                Thread.sleep(100);
            } catch (Throwable t) {
                t.printStackTrace();
            }
        }
    }

    @Configuration
    @EnableDubbo(scanBasePackages = "org.apache.dubbo.samples.action")
    @PropertySource("classpath:/spring/dubbo-consumer.properties")
    @ComponentScan(value = {"org.apache.dubbo.samples.action"})
    static class ConsumerConfiguration {

    }
}

Write configuration information

dubbo.application.name=dubbo-samples-xds-consumer
dubbo.application.metadataServicePort=20885
dubbo.registry.address=xds://istiod.istio-system.svc:15012
dubbo.consumer.timeout=3000
dubbo.consumer.check=false
dubbo.application.qosEnable=true
dubbo.application.qosAcceptForeignIp=true

Write Deployment.yml and Service.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: dubbo-samples-xds-consumer
  namespace: dubbo-demo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: dubbo-samples-xds-consumer
  template:
    metadata:
      labels:
        app: dubbo-samples-xds-consumer
    spec:
      containers:
        - name: server
          image: apache/dubbo-demo:dubbo-samples-xds-consumer_0.0.1
          livenessProbe:
            httpGet:
              path: /live
              port: 22222
            initialDelaySeconds: 5
            periodSeconds: 5
          readinessProbe:
            httpGet:
              path: /ready
              port: 22222
            initialDelaySeconds: 5
            periodSeconds: 5
          startupProbe:
            httpGet:
              path: /startup
              port: 22222
            failureThreshold: 30
            periodSeconds: 10

apiVersion: v1
kind: Service
metadata:
  name: dubbo-samples-xds-consumer
  namespace: dubbo-demo
spec:
  clusterIP: None
  selector:
    app: dubbo-samples-xds-consumer
  ports:
    - name: grpc
      protocol: TCP
      port: 50051
      targetPort: 50051

Write Dockerfile

FROM openjdk:8-jdk
ADD ./target/dubbo-samples-xds-consumer-1.0-SNAPSHOT.jar dubbo-samples-xds-consumer-1.0-SNAPSHOT.jar
CMD java -jar -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=31000 /dubbo-samples-xds-consumer-1.0-SNAPSHOT.jar

✅ So far our environment and code are all ready!

build image

1. Start docker

2. Start minikube

Because minikube is a local k8s, it needs a virtual engine to start. Here we use docker to manage it.We start with the following command

minikube start

We can see minikube in docker

3. Check the status of istio

4. Build a mirror image

Find the location of the code locally and execute the following commands in sequence

# 找到provider所在路径
cd ./dubbo-samples-xds-provider/
# 构建provider的镜像
docker build -t apache/dubbo-demo:dubbo-samples-xds-provider_0.0.1 .

# 找到consumer所在路径
cd ../dubbo-samples-xds-consumer/
# 构建consumer的镜像
docker build -t apache/dubbo-demo:dubbo-samples-xds-consumer_0.0.1 .

5. Check the local mirror

6. Create a namespace

# 初始化命名空间
kubectl apply -f https://raw.githubusercontent.com/apache/dubbo-samples/master/dubbo-samples-xds/deploy/Namespace.yml

# 切换命名空间
kubens dubbo-demo

If you do not create a namespace, you will see the following error

deploy container

# 找到provider所在路径
cd ./dubbo-samples-xds-provider/src/main/resources/k8s
# dubbo-samples-xds/dubbo-samples-xds-provider/src/main/resources/k8s/Deployment.yml
# dubbo-samples-xds/dubbo-samples-xds-provider/src/main/resources/k8s/Service.yml

# 部署provider的Deployment和Service
kubectl apply -f Deployment.yml
kubectl apply -f Service.yml

# 找到consumer所在路径
cd ../../../../../dubbo-samples-xds-consumer/src/main/resources/k8s
# dubbo-samples-xds/dubbo-samples-xds-consumer/src/main/resources/k8s/Deployment.yml

# 部署consumer的Deployment
kubectl apply -f Deployment.yml

See the pods we have deployed on the minikube dashboard

Observe consumer effect

kubectl logs xxx

result: hello, xDS Consumer! from host: 172.17.0.5
result: hello, xDS Consumer! from host: 172.17.0.5
result: hello, xDS Consumer! from host: 172.17.0.6
result: hello, xDS Consumer! from host: 172.17.0.6

Summary & Outlook

This article mainly analyzes the core processes of Dubbo Proxyless Mesh architecture, service discovery and certificate management, and finally demonstrates how to use Dubbo Proxyless through examples.

With the release of Dubbo 3.1, Dubbo has taken another important step on the road to cloud native. At the end of this year, Dubbo Mesh will release a version with service discovery capabilities, which will provide all Dubbo users with the ability to smoothly migrate from lower versions to the Mesh architecture; a version with governance capabilities will be released in the early spring of next year; next year A version with hot plug-in update capability will be released before the end of the year. I hope that students who are interested in witnessing Dubbo’s cloud-native journey can actively participate in community contributions!