持续创作，加速成长！这是我参与「掘金日新计划 · 6 月更文挑战」的第5天，点击查看活动详情
RocketMQ的通信模块及通信协议
rocketmq-remoting 模块是 RocketMQ消息队列中负责网络通信的模块，它几乎被其他所有需要网络通信的模块所依赖和引用。为了实现客户端与服务器之间高效的数据请求与接收，RocketMQ消息队列自定义了通信协议并在Netty的基础之上扩展了通信模块。
在Client和Server之间完成一次消息发送时，需要对发送的消息进行一个协议约定，因此就有必要自定义RocketMQ的消息协议。同时，为了高效地在网络中传输消息和对收到的消息读取，就需要对消息进行编解码。在RocketMQ中，RemotingCommand这个类在消息传输过程中对所有数据内容的封装，不但包含了所有的数据结构，还包含了编码解码操作。
通信模块实现
通信模块的顶层接口 RemotingService，实现类为NettyRemotingClient/NettyRemotingServer，分为客户端和服务端。发送接收消息时各有不同实现。
消息传递的实体是RemotingCommand,部分变量为：
private int code; // 操作码，标识不同的业务
private LanguageCode language = LanguageCode.JAVA;// 实现语言
private int version = 0;// 版本
private int opaque = requestId.getAndIncrement(); // 自增的requestId
private int flag = 0; // 标记RPC/onewayRPC
private String remark; // 备注信息
private HashMap<String, String> extFields; // 扩展属性
private transient CommandCustomHeader customHeader; // 业务中的CustomHeader
private SerializeType serializeTypeCurrentRPC = serializeTypeConfigInThisServer;
private transient byte[] body;
复制代码传输数据格式
传输数据使用encode()方法转码，传输数据格式的encode()方法如下， 其中发送的几个关键数据为：消息长度(标记4)、标记序列化类型和消息头的长度(标记5)、消息头数据(标记6)、消息体数据(标记7)
public ByteBuffer encode() {
// 1> header length size
int length = 4;
// 2> header data length
byte[] headerData = this.headerEncode();
length += headerData.length;
// 3> body data length
if (this.body != null) {
length += body.length;
}
ByteBuffer result = ByteBuffer.allocate(4 + length);
// 4 length
result.putInt(length);
// 5 header length
result.putInt(markProtocolType(headerData.length, serializeTypeCurrentRPC));
// 6 header data
result.put(headerData);
// 7 body data;
if (this.body != null) {
result.put(this.body);
}
result.flip();
return result;
}
复制代码消息通信方式说明
- 构建请求信息，包括
AccessKey、Signature - 开始发送消息。获取
opaque值； - 构建
ResponseFuture用于接收channel写入返回值； responseTable设置opaquechannel.writeAndFlush开始写入；- 注册监听器，操作完成移除
opaque,responseTable.remove(opaque) - 获取返回值
RemotingCommand； NettyClientHandler.channelRead0读发送的消息- 进入
processRequestCommand方法 - 获取
opaque值； - 构建
RequestTask的Runnable方法； pair.getObject2().submit提交执行Runnable；- 发送
processor.processRequest请求执行具体业务操作 - 回调判断是否为单向
OnewayRPC，不是单向则将opaque与对应信息返回response org.apache.rocketmq.remoting.netty
流程图
NettyRemotingClient关键代码
这部分代码就是上面消息通信方式说明中的构建请求信息，包括AccessKey、Signature，invokeSyncImpl是同步发送实现。
public RemotingCommand invokeSyncImpl(final Channel channel, final RemotingCommand request,
final long timeoutMillis)
throws InterruptedException, RemotingSendRequestException, RemotingTimeoutException {
final int opaque = request.getOpaque();
try {
final ResponseFuture responseFuture = new ResponseFuture(channel, opaque, timeoutMillis, null, null);
this.responseTable.put(opaque, responseFuture);
final SocketAddress addr = channel.remoteAddress();
channel.writeAndFlush(request).addListener(new ChannelFutureListener() {
@Override
public void operationComplete(ChannelFuture f) throws Exception {
if (f.isSuccess()) {
responseFuture.setSendRequestOK(true);
return;
} else {
responseFuture.setSendRequestOK(false);
}
responseTable.remove(opaque);
responseFuture.setCause(f.cause());
responseFuture.putResponse(null);
log.warn("send a request command to channel <" + addr + "> failed.");
}
});
RemotingCommand responseCommand = responseFuture.waitResponse(timeoutMillis);
if (null == responseCommand) {
if (responseFuture.isSendRequestOK()) {
throw new RemotingTimeoutException(RemotingHelper.parseSocketAddressAddr(addr), timeoutMillis,
responseFuture.getCause());
} else {
throw new RemotingSendRequestException(RemotingHelper.parseSocketAddressAddr(addr), responseFuture.getCause());
}
}
return responseCommand;
} finally {
this.responseTable.remove(opaque);
}
}
复制代码NettyRemotingServer关键代码
根据code与唯一标识opaque，执行命令任务pair.getObject2().submit(requestTask)
public void processRequestCommand(final ChannelHandlerContext ctx, final RemotingCommand cmd) {
final Pair<NettyRequestProcessor, ExecutorService> matched = this.processorTable.get(cmd.getCode());
// 找到处理器，找不到则使用默认处理器
final Pair<NettyRequestProcessor, ExecutorService> pair = null == matched ? this.defaultRequestProcessor : matched;
// 获取唯一标识opaque
final int opaque = cmd.getOpaque();
if (pair != null) {
Runnable run = new Runnable() {
@Override
public void run() {
try {
// 获取地址remoteAddr
String remoteAddr = RemotingHelper.parseChannelRemoteAddr(ctx.channel());
doBeforeRpcHooks(remoteAddr, cmd);
// callback
final RemotingResponseCallback callback = new RemotingResponseCallback() {
@Override
public void callback(RemotingCommand response) {
doAfterRpcHooks(remoteAddr, cmd, response);
if (!cmd.isOnewayRPC()) {
if (response != null) {
response.setOpaque(opaque);
response.markResponseType();
response.setSerializeTypeCurrentRPC(cmd.getSerializeTypeCurrentRPC());
try {
ctx.writeAndFlush(response);
} catch (Throwable e) {
log.error("process request over, but response failed", e);
log.error(cmd.toString());
log.error(response.toString());
}
} else {
}
}
}
};
// 同步处理
if (pair.getObject1() instanceof AsyncNettyRequestProcessor) {
AsyncNettyRequestProcessor processor = (AsyncNettyRequestProcessor)pair.getObject1();
processor.asyncProcessRequest(ctx, cmd, callback);
} else {
// 异步处理
NettyRequestProcessor processor = pair.getObject1();
RemotingCommand response = processor.processRequest(ctx, cmd);
callback.callback(response);
}
} catch (Throwable e) {
log.error("process request exception", e);
log.error(cmd.toString());
if (!cmd.isOnewayRPC()) {
final RemotingCommand response = RemotingCommand.createResponseCommand(RemotingSysResponseCode.SYSTEM_ERROR,
RemotingHelper.exceptionSimpleDesc(e));
response.setOpaque(opaque);
ctx.writeAndFlush(response);
}
}
}
};
if (pair.getObject1().rejectRequest()) {
final RemotingCommand response = RemotingCommand.createResponseCommand(RemotingSysResponseCode.SYSTEM_BUSY,
"[REJECTREQUEST]system busy, start flow control for a while");
response.setOpaque(opaque);
ctx.writeAndFlush(response);
return;
}
try {
// 构建requestTask
final RequestTask requestTask = new RequestTask(run, ctx.channel(), cmd);
pair.getObject2().submit(requestTask);
} catch (RejectedExecutionException e) {
if ((System.currentTimeMillis() % 10000) == 0) {
log.warn(RemotingHelper.parseChannelRemoteAddr(ctx.channel())
+ ", too many requests and system thread pool busy, RejectedExecutionException "
+ pair.getObject2().toString()
+ " request code: " + cmd.getCode());
}
// 非单向命令，需要获取返回值并写入ChannelHandlerContext
if (!cmd.isOnewayRPC()) {
final RemotingCommand response = RemotingCommand.createResponseCommand(RemotingSysResponseCode.SYSTEM_BUSY,
"[OVERLOAD]system busy, start flow control for a while");
response.setOpaque(opaque);
ctx.writeAndFlush(response);
}
}
} else {
String error = " request type " + cmd.getCode() + " not supported";
// 获取命令结果response
final RemotingCommand response =
RemotingCommand.createResponseCommand(RemotingSysResponseCode.REQUEST_CODE_NOT_SUPPORTED, error);
// response设置唯一标识
response.setOpaque(opaque);
ctx.writeAndFlush(response);
log.error(RemotingHelper.parseChannelRemoteAddr(ctx.channel()) + error);
}
}
复制代码小结
以上，列出了RocketMQ通信的关键代码，RocketMQ通信的基础是Netty底层通信库，通过定义协议，封装数据结构，再由通用类进行通信逻辑处理。