[Shenzhen] Yuanchuanghui: 5.26pm, the party hall is waiting for you.”

Author: Wang Yuxuan (github: DMwangnima), Kitex Committer

1. Background

Kitex is a Go microservice RPC framework launched by the ByteDance infrastructure service framework team. It supports message protocols such as Thrift, Kitex Protobuf, and gRPC, and is characterized by high performance and strong scalability. After being officially open sourced in September 2021, it has been successfully implemented in many external companies, bringing them real cost, performance and stability benefits.

In the process of using Kitex transformation services, many enterprise users need Kitex to communicate with the services implemented by the existing Dubbo framework. This coincides with the goal of the CloudWeGo community to actively expand the ecosystem. Therefore, the Dubbo interoperability project codec-dubbo came into being. born.

With the enthusiastic help of community students, codec-dubbo is currently able to interoperate between Kitex and Dubbo-Java, Kitex and Dubbo-Go, and supports Dubbo users to migrate to Kitex.

This article will take Founder Securities' successful service transformation using Kitex and codec-dubbo as an example, explain the main functions of codec-dubbo used in the transformation process, and briefly analyze the implementation details.

2. Enterprise implementation cases

Founder Securities' original services are written in Java and Dubbo frameworks. Both are stable and have been verified in a large number of scenarios, meeting their production and development needs. Taking Xiaofang's individual stock details page, which has a large number of requests, as an example, the interface QPS during peak periods is 3-4k, and 16 16 Core 64G virtual machines are used for hosting.

With the rise of cloud-native architecture, Go has gradually become an important technology option for building enterprise services due to its advantages in memory usage and execution efficiency, as well as its natural adaptability to cloud-native. In order to better reduce costs and increase efficiency, after comprehensively considering factors such as cost, performance and stability, they decided to switch from Java to Go for new applications, introduced Kitex, Hertz and other CloudWeGo projects for service development and reconstruction, and migrated the entire Kubernetes environment.

During the reconstruction process, codec-dubbo relied on a user experience close to native Kitex + Thrift and good support for Dubbo concepts to reduce the cost of use and understanding. It successfully helped them solve the interoperability problem of Kitex <-> Dubbo and let Kitex serve Smoothly call the original Dubbo service.

目前，使用了 codec-dubbo 的 Kitex 服务已成功上线，稳定运行两个月。还是以小方个股详情页为例，Kitex 和 Hertz 承载了该页面一半左右的接口，在 QPS 不变的情况下，只需要提供 12 个 4 Core 4G Pod，降低资源占用效果显著。

3. codec-dubbo functional features

Dubbo protocol codec

Dubbo service mainly uses Dubbo protocol for communication. In order to support Kitex <-> Dubbo interoperability, we need to implement Dubbo protocol in Kitex. Thanks to the excellent scalability of Kitex, codec-dubbo implements DubboCodec, the core codec, based on the Codec interface provided by Kitex. You only need to inject DubboCodec during initialization to use the Dubbo protocol.

Type mapping and expansion

type mapping

Dubbo mainly uses the Hessian2 serialization protocol to encode and decode the payload. Its biggest feature is the self-describing serialization type, that is, it does not rely on external Schema or interface definitions. The serialization process relies on the mapping between programming language types and Hessian2 types. Taking the conversion of Go types into Java types as an example:

After analysis, we found that the basic type system of Hessian2 basically overlaps with Thrift. In order to ensure that the usage experience of Kitex + codec-dubbo is basically the same as Kitex + Thrift, we generate Kitex Dubbo-Hessian2 scaffolding code based on Thrift IDL. At this time, the type conversion process is as follows:

Refer to Dubbo's official dubbo-go-hessian2 type mapping. codec-dubbo provides the following type mapping (only part of the mapping is included here. For more precautions, please refer to codec-dubbo Readme):

According to the type mapping provided by codec-dubbo, we can easily convert the Dubbo interface definition into Thrift IDL, use the Kitex command line tool to generate project scaffolding code, and finally inject DubboCodec to complete Kitex -> Dubbo communication. Take the following Dubbo interface definition as an example:

package org.cloudwego.kitex.samples.api;

public interface GreetProvider {
    GreetResponse Greet(GreetRequest req) throws Exception;
}

public class GreetRequest implements Serializable {
    String req;

    public GreetRequest(String req) {
        this.req = req;
    }
}

public class GreetResponse implements Serializable {
    String resp;

    public GreetResponse(String resp) {
        this.resp = resp;
    }
}

The corresponding api.thrift file is as follows. It should be noted that the structure definitions in it need to be annotated with JavaClassName, which corresponds to the package + class name in the Dubbo interface definition.

struct GreetRequest {
    1: required string req,
} (JavaClassName="org.cloudwego.kitex.samples.api.GreetRequest")

struct GreetResponse {
    1: required string resp,
} (JavaClassName="org.cloudwego.kitex.samples.api.GreetResponse")

service GreetService {
    GreetResponse Greet(1: GreetRequest req)
}

Use the Kitex command line tool and specify the protocol as Hessian2:

kitex -module demo-client -protocol Hessian2 ./api.thrift

Then initialize DubboCodec and inject it into Kitex. Use the generated code to write the following client code to implement Kitex -> Dubbo call:

javaClass := "org.cloudwego.kitex.samples.api.GreetProvider"
cli, err := greetservice.NewClient("helloworld",
    // 指定想要访问的服务端地址，也支持 ZooKeeper 服务发现
    client.WithHostPorts("127.0.0.1:21000"),
    // 配置 DubboCodec
    client.WithCodec(
        // 指定想要调用的 Dubbo Interface
        dubbo.NewDubboCodec(dubbo.WithJavaClassName(javaClass))
    ),
)
if err != nil {
   panic(err)
}
    
resp, err := cli.Greet(context.Background(),
    &hello.GreetRequest{Req: "world"})
if err != nil {
    klog.Error(err)
    return
}
klog.Infof("resp: %s", resp.Resp)

Kitex + codec-dubbo Server-side process is basically similar to the Client-side process. For specific examples, please refer to the project homepage.

Type expansion

The schema-free feature of Hessian2 makes Dubbo's implementation "too flexible" and can use any type. In order to adapt to the type usage flexibility of Dubbo Hessian2, codec-dubbo supports type expansion, which mainly includes custom mapping and Java common type expansion.

Custom mapping

Java's basic types have corresponding wrapper types, such as booleanand java.lang.Boolean. The default boolmapping of Go types to Java java.lang.Booleantypes in type mapping does not cover the use booleanof . In order to unify the user experience and allow them to only use boolthe type on the Kitex side, we can add annotations after the Thrift method definition hessian.argsType="boolean", use the IDL reflection function of thriftgo to generate IDL meta information in advance and inject it into codec-dubbo, and then it can be run Dynamically java.lang.Booleanrewrite the default mapping type to boolean. The specific Thrift definition is as follows:

service EchoService {
   bool EchoBoolean(1: bool req) (hessian.argsType="boolean")
}

Similar to booleanand java.lang.Boolean, other Java basic types and wrapper types can also be customized in this way. At this time, the complete type mapping provided by codec-dubbo is as follows:

java common type expansion

Due to the limitations of Thrift types, we cannot directly use common types provided in Java class libraries. To this end, codec-dubbo maintains Java types that Thrift does not support (for example , ) and the corresponding java.thrift in the codec-dubbo/java package . At the same time, with the help of the idl-ref function provided by thriftgo, we can directly use Thrift These types are referenced in IDL and corresponding code is generated. The current java.thrift looks like this:java.lang.Objectjava.util.Date

struct Object {} (JavaClassName="java.lang.Object")

struct Date {} (JavaClassName="java.util.Date")

struct Exception {} (JavaClassName="java.lang.Exception")

In order to enable these types, we need to import them in Thrift IDL include "java.thrift"and add parameters when generating code using the Kitex command line tool -hessian2 java_extensionto pull the expansion package.

The Kitex command line tool will automatically download java.thrift. You can also download it manually and put it in the root directory of the project. Thrift IDL example referencing types in java.thrift:

include "java.thrift"

service EchoService {
    // java.lang.Object
    i64 EchoString2ObjectMap(1: map<string, java.Object> req)
    // java.util.Date
    i64 EchoDate(1: java.Date req)
}

Method overloading

Go does not natively support method overloading. You can only achieve overloading-like effects by defining multiple methods. In order to map multiple methods in Go to overloaded methods in Java, similar to the custom mapping section, we add the JavaMethodName tag after the method definition in Thrift, and use the IDL reflection function of thriftgo to dynamically map Go at runtime. The original method name is rewritten to the overloaded method in Java specified by JavaMethodName.

Take EchoMethod on the Java side as an example:

String EchoMethod(Boolean req);
String EchoMethod(Integer req);
String EchoMethod(int req);
String EchoMethod(Boolean req1, Integer req2);

We write the following Thrift definition to complete the overloaded method mapping between Go and Java. Note JavaMethodNamethat and hessian.argsTypecan be used at the same time:

service EchoService {
    string EchoMethodA(1: bool req) (JavaMethodName="EchoMethod")
    string EchoMethodB(1: i32 req) (JavaMethodName="EchoMethod")
    string EchoMethodC(1: i32 req) (JavaMethodName="EchoMethod", hessian.argsType="int")
    string EchoMethodD(1: bool req1, 2: i32 req2) (JavaMethodName="EchoMethod")
 }

Exception handling

codec-dubbo maps exceptions in Java to errors in Go. These errors uniformly implement the following interface:

type Throwabler interface {
    Error() string
    JavaClassName() string
    GetStackTrace() []StackTraceElement
}

Based on the exception handling practices officially recommended by Dubbo and the current needs of enterprise users, we divide exceptions into common exceptions and custom exceptions, taking into account the basic needs and scalability needs of users.

Common exceptions

codec-dubbo provides common Java exceptions in the pkg/hessian2/exception package, currently supporting java.lang.Exception.

Common exceptions do not require support from the Kitex command line tool and can be quoted directly. The following are examples of exceptions extracted by the client and exceptions returned by the server.

Client extraction exception

resp, err := cli.Greet(context.Background(),
    &hello.GreetRequest{Req: "world"})
if err != nil {
    // FromError 返回 Throwabler
    exceptionRaw, ok := hessian2_exception.FromError(err)
    if !ok {
        // 视作常规错误处理        
    } else {
        // 若不关心 exceptionRaw 的具体类型，直接调用 Throwabler 提供的方法即可
        klog.Errorf("get %s type Exception", exceptionRaw.JavaClassName())                
        // 若想获得 exceptionRaw 的具体类型，需要进行类型转换，但前提是已知该具体类型
        exception := exceptionRaw.(*hessian2_exception.Exception)
    }
}

Server side returns exception

func (s *GreetServiceImpl) Greet(ctx context.Context, req *hello.GreetRequest) (resp *hello.GreetResponse, err error) {
    return nil, hessian2_exception.NewException("Your detailed message")
}

Custom exception

Custom exceptions in Java often inherit a basic exception. Here, taking as CustomizedExceptionan example , CustomizedExceptionit inherits java.lang.Exception:

public class CustomizedException extends Exception {
    private final String customizedMessage;
    
    public CustomizedException(String customizedMessage) {
        super();
        this.customizedMessage = customizedMessage;
    }
}

Thanks to thriftgo's support for generating nested structures, in order to define corresponding exceptions on the Kitex side, we write the following definition in Thrift:

exception CustomizedException {
    // thrift.nested=“true” 注解让 thriftgo 生成嵌套结构体
    1: required java.Exception exception (thrift.nested="true")
    2: required string customizedMessage
}(JavaClassName="org.cloudwego.kitex.samples.api.CustomizedException")

Pay attention exceptionto the field annotations thrift.nested="true", which allow thriftgo to generate nested structures to achieve an effect similar to inheritance.

Like Java common type extensions, you need to add parameters when using the kitex scaffolding tool to generate code -hessian2 java_extensionto pull the expansion package. The generated code is as follows:

type EchoCustomizedException struct {
        java.Exception `thrift:"exception,1,required" frugal:"1,required,java.Exception" json:"exception"`

        CustomizedMessage string `thrift:"customizedMessage,2,required" frugal:"2,required,string" json:"customizedMessage"`
}

The usage method is consistent with common exceptions and will not be repeated here.

Service registration and discovery

Dubbo provides both interface-level and application-level service registration and discovery models. Based on the current production environment needs of enterprise users, we choose to prioritize the interface-level model based on zookeeper: Dubbo registry-zookeeper.

Different from the application-level model we are familiar with, the interface-level model needs to maintain the mapping relationship of interface name => service (different from microservices, closer to Handler). One interface name will be mapped to multiple services, and these services may exist in the same in a process.

Considering that Dubbo's interface-level service model is quite different from Kitex's service model, and Dubbo registry-zookeeper should be bound to codec-dubbo for use, it is not considered to modify the original registry-zookeeper in kitex-contrib to make dubbo registry-zookeeper A sub-go module of codec-dubbo is developed and maintained uniformly.

Taking into account the Dubbo interface-level service model, Kitex API and user experience, we provide the following configuration levels:

The WithServers and WithRegistryGroup functions in registry/options.go and resolver/options.go provide registry-level configuration, respectively specifying the address of the zookeeper and the group to which these zookeepers belong. Users use these functions to generate Kitex registry.Registryand discovery.Resolverinstances.
Service-level configuration is passed client.WithTagwith and , registries/common.go provides Tag keys, which correspond to service metadata in Dubbo one-to-one.server.WithRegistryInfo

resolver example

intfName := "org.cloudwego.kitex.samples.api.GreetProvider"
res, err := resolver.NewZookeeperResolver(
    // 指定 zookeeper 服务器的地址，可指定多个，请至少指定一个 
    resolver.WithServers("127.0.0.1:2181"),
)
if err != nil {
    panic(err)
}
cli, err := greetservice.NewClient("helloworld",
    // 配置 ZookeeperResolver
    client.WithResolver(res),
    // 指定想要调用的 dubbo Interface
    client.WithTag(registries.DubboServiceInterfaceKey, intfName),
)
if err != nil {
    panic(err)
}
// 使用 cli 进行 RPC 调用

registry example

intfName := "org.cloudwego.kitex.samples.api.GreetProvider"
reg, err := registry.NewZookeeperRegistry(
    // 指定 zookeeper 服务器的地址，可指定多个，请至少指定一个 
    registry.WithServers("127.0.0.1:2181"),
)
if err != nil {
    panic(err)
}

svr := greetservice.NewServer(new(GreetServiceImpl),
    server.WithRegistry(reg),
    // 配置dubbo URL元数据
    server.WithRegistryInfo(&kitex_registry.Info{
        Tags: map[string]string{
            registries.DubboServiceInterfaceKey: intfName,
            // application请与dubbo所设置的ApplicationConfig保持一致，此处仅为示例
            registries.DubboServiceApplicationKey: "application-name",
        }
    }),
)
// 启动 svr

Summarize

Kitex supports the Dubbo protocol, which is a big step for CloudWeGo to help integrate multi-language cloud-native ecology. It solves the pain points of many enterprise users in converting Java to Go, and coexisting between Java and Go. Everyone is welcome to try and access it; if you encounter any problems during use, If you have any questions, you can join our Feishu user group or give us feedback on Github.

Kitex supports Dubbo protocol: facilitating the integration of multi-language cloud-native ecosystems