Netty粘包/拆包解决方案

其他 2018-08-07 17:16:16 阅读次数: 0

TCP粘包/拆包问题

 TCP粘包和拆包示意图如下：

TCP粘包/拆包

假设客户端分别发送了两个数据包D1和D2给服务端，由于服务端一次读到的字节数是不确定的，故可能存在以下4中情况.
 1. 服务端分两次读取到了两个独立的数据包，分别是D1和D2,没有粘包和拆包
 2. 服务端一次收到了连个数据包，D1和D2粘合在一起，被称为TCP粘包
 3. 服务端分别两次读取到了两个数据包，第一次读取到了完整的D1包和D2包的部分内容，第二次读取到了D2包的剩余内容，称之为TCP拆包。
 4. 服务端分两次读取到了两个数据包，第一次读取到了D1包的部分内容D1_1,第二次读取到了D1包的剩余内容D1_2和D2包的整包。如果此时服务端TCP接受滑窗非常小，而数据包D1和D2比较大，很有可能会发生第五种可能，即服务端分多次才能将D1和D2包接收完全，期间分生多次拆包。

粘包问题的解决策略

由于底层的TCP 无法理解上层的的业务数据，所以在底层是无法保证数据包不被拆分和重组的，这个问题只能通过上层的应用协议栈设计来解决，根据业界的主流的协议的解决方案，可以归纳如下：
 1. 消息定长，例如每个报文的大小为固定长度200字节，如果不够，空位补空格；
 2. 在包尾增加回车换行符进行分割，例如FTP协议；
 3. 将消息分为消息头和消息体，消息头中包含表示消息总长度的字段，通常设计思路为消息头的第一个字段使用int32来表示消息的总长度；
 4. 更复杂的应用层协议。

未考虑TCP粘包导致功能异常的案例

 未考虑TCP粘包下的客户端启动，至添加自己实现的普通的写数据的TimeClientHandler。

/**
 * @desc 客户端启动
 */
public class TimeClient {

    public void connect(int port, String host) {
        EventLoopGroup workGroup = new NioEventLoopGroup();
        Bootstrap bootstrap = new Bootstrap();
        bootstrap.group(workGroup)
                .channel(NioSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ch.pipeline()
                                .addLast(new TimeClientHandler());
                    }
                });

        ChannelFuture future = bootstrap.connect(host, port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeClient().connect(8080, "127.0.0.1");
    }
}

 TimeClientHandler实现如下：


/**
 * @desc 客户端handler
 */
public class TimeClientHandler extends ChannelInboundHandlerAdapter {

    private int count;

    private byte[] request;

    public TimeClientHandler() {
        request = ("QUERY TIME ORDER" + System.getProperty("line.separator")).getBytes();
    }
    /**
     * @desc 接受的服务端返回数据
     */
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {

        ByteBuf byteBuf = (ByteBuf) msg;
        request = new byte[byteBuf.readableBytes()];
        String body = new String(request, "UTF-8");

        System.out.println("now is:" + body.toString() + "; the counter is" + ++count);

    }
    /**
     * @desc 客户端与服务端channel开启的时候，写数据
     */
    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        ByteBuf message = null;
        for (int i = 0; i < 100; i++) {
            message = Unpooled.buffer(request.length);
            message.writeBytes(request);
            ctx.writeAndFlush(message);
        }
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        cause.printStackTrace();
        super.exceptionCaught(ctx, cause);
    }
}

 服务端启动类：


/**
 * @desc 服务端启动
 */
public class TimeServer {

    public void bind(int port) {

        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workGroup = new NioEventLoopGroup();
        ServerBootstrap bootstrap = new ServerBootstrap();
        bootstrap.group(bossGroup, workGroup)
                .channel(NioServerSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .childHandler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ch.pipeline()
                                .addLast(new TimeServerHandler());
                    }
                });

        ChannelFuture future = bootstrap.bind(port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            bossGroup.shutdownGracefully();
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeServer().bind(8080);
    }
}

 服务端TimeServerHandler实现，接收数据之后的处理逻辑：


/**
 * @desc 服务端处理handler
 */
public class TimeServerHandler extends ChannelInboundHandlerAdapter {

    private int count;

    /**
     * @desc 服务端接受到客户端数据的处理逻辑
     */
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {

        ByteBuf byteBuf = (ByteBuf) msg;
        byte[]  request = new byte[byteBuf.readableBytes()];
        byteBuf.readBytes(request);
        String body = new String(request, "UTF-8")
                .substring(0, request.length - System.getProperty("line.separator").length());
        System.out.println("time server receive order:" + body + ";the count is :" + ++ count);

        String currentTime = "QUERY TIME ORDER".equalsIgnoreCase(body) ? new Date(System.currentTimeMillis()).toString() : "BAD ORDER";

        currentTime = currentTime + System.getProperty("line.separator");
        ByteBuf response = Unpooled.copiedBuffer(currentTime.getBytes());
        ctx.writeAndFlush(response);
    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("client connect !!!!!!");
        super.channelActive(ctx);
    }
}

运行结果如下：

QUERY TIME ORDER
QUERY TIME ORDER
QUERY TIME ORDER
QUERY TIME ORDER
QUE;the count is :1
time server receive order:Y TIME ORDER
QUERY TIME ORDER
QUERY TIME ORDER

 发生粘包现象。”QUERY TIME ORDER”会出现不完整的数据包。

利用LineBasedFrameDecoder解决TCP粘包问题

 LineBasedFrameDecoder其实就是按照换行符分隔字符创的解码器。


/**
 * @desc 基于LineBasedFrameDecoder服务端
 */
public class TimeServer {

    public void bind(int port) {

        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workGroup = new NioEventLoopGroup();
        ServerBootstrap bootstrap = new ServerBootstrap();
        bootstrap.group(bossGroup, workGroup)
                .channel(NioServerSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .childHandler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ch.pipeline()
                                .addLast(new LineBasedFrameDecoder(1024))//添加LineBasedFrameDecoder
                                .addLast(new TimeServerHandler());
                    }
                });

        ChannelFuture future = bootstrap.bind(port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            bossGroup.shutdownGracefully();
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeServer().bind(8080);
    }
}

 客户端实现，与前面的几乎相同：


/**
 * @desc 基于LineBasedFrameDecoder客户端
 */
public class TimeClient {

    public void connect(int port, String host) {
        EventLoopGroup workGroup = new NioEventLoopGroup();
        Bootstrap bootstrap = new Bootstrap();
        bootstrap.group(workGroup)
                .channel(NioSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ch.pipeline()
                                .addLast(new LineBasedFrameDecoder(1024))
                                .addLast(new TimeClientHandler());
                    }
                });

        ChannelFuture future = bootstrap.connect(host, port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeClient().connect(8080, "127.0.0.1");
    }
}

 运行结果如下：

time server receive order:QUERY TIME ORDE;the count is :1
time server receive order:QUERY TIME ORDE;the count is :2
time server receive order:QUERY TIME ORDE;the count is :3
.......
time server receive order:QUERY TIME ORDE;the count is :50

 完美解决粘包现象。LineBasedFrameDecoder的工作原理是依次遍历ByteBuf中的可读字节，判断是否有”“\n”或者”“\r\n”,如果有，就依次位置为结束位置，从可读索引到结束位置区间的字节组成一行。它是以换行符为结束标志的解码器，支持携带结束符或者不携带结束符两种解码方式，同时支持配置单行的最大长度。如果连读读取到最大长度后仍然没有发现换行符，就会抛出异常，同时忽略掉之前读取到的异常码流。
 StringDecoder的功能非常简单，就是将接受到的对象转换成字符串，然后继续调用后面的handler,LineBasedFrameDecoder + StringDecoder组合就是按行切换的文本解码器，它被设计用来支持TCP的粘包和拆包

利用DelimiterBasedFrameDecoder解决TCP粘包问题

 DelimiterBasedFrameDecoder是一种基于分隔符的编码器，用以处理粘包问题。

public class TimeClient {

    public void connect(int port, String host) {
        EventLoopGroup workGroup = new NioEventLoopGroup();
        Bootstrap bootstrap = new Bootstrap();
        bootstrap.group(workGroup)
                .channel(NioSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .handler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ByteBuf delimiter = Unpooled.copiedBuffer("$_".getBytes());//定义使用的分隔符

                        ch.pipeline()
                                .addLast(new DelimiterBasedFrameDecoder(1024, delimiter))//添加DelimiterBasedFrameDecoder
                                .addLast(new StringDecoder())
                                .addLast(new TimeClientHandler());
                    }
                });

        ChannelFuture future = bootstrap.connect(host, port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeClient().connect(8080, "127.0.0.1");
    }
}

 客户端实现。

public class TimeClientHandler extends ChannelInboundHandlerAdapter {

    private int count;


    static final String echo = "hi,welcome to netty.$_";


    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {

        System.out.println("this is" + ++count + " times receive client : {" + msg + "}");


    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        IntStream.range(0,20).forEach(i -> {
            ctx.writeAndFlush(Unpooled.copiedBuffer((echo+i).getBytes()));
        });

    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        cause.printStackTrace();
        super.exceptionCaught(ctx, cause);
    }
}

 服务端实现。

public class TimeServer {

    public void bind(int port) {

        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workGroup = new NioEventLoopGroup();
        ServerBootstrap bootstrap = new ServerBootstrap();
        bootstrap.group(bossGroup, workGroup)
                .channel(NioServerSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .childHandler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ByteBuf delimiter = Unpooled.copiedBuffer("$_".getBytes());
                        ch.pipeline()
                                .addLast(new DelimiterBasedFrameDecoder(1024, delimiter))
                                .addLast(new StringDecoder())
                                .addLast(new TimeServerHandler());
                    }
                });

        ChannelFuture future = bootstrap.bind(port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            bossGroup.shutdownGracefully();
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeServer().bind(8080);
    }
}

public class TimeServerHandler extends ChannelInboundHandlerAdapter {

    private int count;

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {

        String body = (String) msg;
        System.out.println("this is" + ++count + " times receive client : {" + body + "}");
        body += "$_";

        ByteBuf response = Unpooled.copiedBuffer(body.getBytes());
        ctx.writeAndFlush(response);
    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("client connect !!!!!!");
        super.channelActive(ctx);
    }
}

client connect !!!!!!
this is1 times receive client : {hi,welcome to netty.}
this is2 times receive client : {0hi,welcome to netty.}
this is3 times receive client : {1hi,welcome to netty.}
this is4 times receive client : {2hi,welcome to netty.}
this is5 times receive client : {3hi,welcome to netty.}
this is6 times receive client : {4hi,welcome to netty.}
this is7 times receive client : {5hi,welcome to netty.}
this is8 times receive client : {6hi,welcome to netty.}
this is9 times receive client : {7hi,welcome to netty.}
this is10 times receive client : {8hi,welcome to netty.}
this is11 times receive client : {9hi,welcome to netty.}
this is12 times receive client : {10hi,welcome to netty.}
this is13 times receive client : {11hi,welcome to netty.}
this is14 times receive client : {12hi,welcome to netty.}
this is15 times receive client : {13hi,welcome to netty.}
this is16 times receive client : {14hi,welcome to netty.}
this is17 times receive client : {15hi,welcome to netty.}
this is18 times receive client : {16hi,welcome to netty.}
this is19 times receive client : {17hi,welcome to netty.}
this is20 times receive client : {18hi,welcome to netty.}

 没有发生粘包现象。

利用FixedLengthFrameDecoder解决TCP粘包问题


public class TimeServer {

    public void bind(int port) {

        EventLoopGroup bossGroup = new NioEventLoopGroup();
        EventLoopGroup workGroup = new NioEventLoopGroup();
        ServerBootstrap bootstrap = new ServerBootstrap();
        bootstrap.group(bossGroup, workGroup)
                .channel(NioServerSocketChannel.class)
                .option(ChannelOption.SO_BACKLOG, 1024)
                .childHandler(new ChannelInitializer<Channel>() {
                    @Override
                    protected void initChannel(Channel ch) throws Exception {
                        ch.pipeline()
                                .addLast(new FixedLengthFrameDecoder(20))
                                .addLast(new StringDecoder())
                                .addLast(new TimeServerHandler());
                    }
                });

        ChannelFuture future = bootstrap.bind(port);
        try {
            future.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            bossGroup.shutdownGracefully();
            workGroup.shutdownGracefully();
        }
    }

    public static void main(String[] args) {
        new TimeServer().bind(8080);
    }
}

public class TimeServerHandler extends ChannelInboundHandlerAdapter {

    private int count;

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        System.out.println("receive client : {" + msg + "}");
    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        System.out.println("client connect !!!!!!");
        super.channelActive(ctx);
    }
}

 此项测试不做客户端实现，直接telnet localhost 8080

pjx@pjxdeMacBook-Pro:~/project/netty-learning$  telnet localhost 8080
Trying ::1...
Connected to localhost.
Escape character is '^]'.
netty fixedLengthFrameDecoder

服务端接受到固定长度数据

receive client : {netty fixedLengthFra}

总结

 DelimiterBasedFrameDecoder用于使用分隔符结尾的消息进行自动解码，FixedLengthFrameDecoder用于对固定长度的消息进行解码。有了上述两种解码器，再结合其他解码器，可以完成对很多消息的自动解码，而且不需要需要再考虑tcp粘包/拆包导致的读半包问题。

猜你喜欢

转载自blog.csdn.net/Pengjx2014/article/details/80966276
今日推荐
基于大语言模型的开源知识库问答系统 MaxKB GitHub Star 数量突破 5,000 个！
美国拟限制 AI 大模型出口中国和俄罗斯
苹果将与 OpenAI 达成协议，将 ChatGPT 应用于 iPhone
openKylin 社区生态委员会第六次会议圆满召开
阿里云正式发布通义千问 2.5
Python 3.13 发布首个 Beta：实验性自由线程模式和 JIT、改进交互式解释器
Stack Overflow 拿我的代码去训练 AI 大模型，还封了我的账号
Pop!_OS 的 COSMIC 桌面完成 App Store 上架工作
《2024 年一季度互联网投融资运行情况》研究报告
报告：Django 仍然是 74% 开发者的首选
15 年前上了“FFmpeg 耻辱柱”，今天他还得谢谢咱——腾讯QQPlayer一雪前耻？
TIOBE 5 月榜单：Fortran “复活”进入 Top 10
周排行
记一下去大梅沙的准备（2018-05-26）
Spring 注解 事务
基于HTTP协议的客户端缓存
阿里云rds 备份和还原
[PHP] 几个拖慢 PHP 程序/API 运行速度的点
python 代码风格------------PEP8规则
js控制json生成菜单——自制菜单（一）
将字符串: 'k:1|k1:2|k2:3|k3:4 ' ,处理成 python 字典: {'k':1, 'k1':2, ...}
微信小程序转支付宝小程序
Qt551.窗口滚动条
每日归档
更多
2024-05-13(18)
2024-05-12(0)
2024-05-11(38)
2024-05-10(38)
2024-05-09(35)
2024-05-08(42)
2024-05-07(14)
2024-05-06(40)
2024-05-05(0)
2024-05-04(7)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022