Netty实现TCP私有协议栈的开发

为什么使用TCP通信？
因为在进行对接机器C语言开发的直播主机)的时候，TCP是一种更安全更高效率的连接，它是一种会确认通信对方，保持通信状态，并且能检查报文完整性的连接，而HTTP则是在TCP服务层上面的应用层，当然TCP通信稍微要复杂一点。

为什么要使用Netty?
先说一下BIO,NIO,AIO的关系：

1. BIO：是一种同步阻塞的io，当io建立连接等待应答的时候，当前的线程就被挂起不能做其他的事情。

2. NIO：是一种同步非阻塞的io，当io建立连接的时候，当前线程不会被挂起，可以去做其他的事情，它会不断去询问内核处理完消息没，当内核返回消息的时候他就可以拿着消息进行读写，其间线程并没有北挂起。

3. AIO：是一种异步非阻塞的io，当io建立连接的时候，当前线程不会被挂起，可以去做其他的事情，然而它也不会去询问消息是否出来完成，内核消息处理完成后会自动通知它，这才叫异步。

   举个栗子：小明去食堂排队打饭的时候，他只能排在那里不能干其他的事情这就是BIO，当食堂实行叫号取餐的时候，小明这个时候完全不用一直等在那里，今天先去外面买瓶可乐打一把王者，他只需要时不时去看看取餐到他没有，这就是NIO，小明在寝室打吃鸡，于是叫了个外卖，餐好了会有外卖小哥送到寝室，他都不用去食堂了也不需要时不时去看是否做好了餐食，这就是AIO。

阻塞与非阻塞是看线程是否被挂起，同步与异步是看是否需要主动去询问消息已准备好

如果使用BIO这样的方式，一个线程只能应答一个请求同时还会被阻塞，不过可以采用线程池的方式来使用BIO,但是这样会造成线程的极大开销，导致系统资源匮乏，请求量过大会引起系统宕机，这时候基于NIO的Netty出来了，它首先不会北阻塞挂起，其次Netty是使用的是IO多路复用的技术，通过selector这个轮询器去轮询当前的请求，只需要一个线程就可以处理多个连接，这样能极大的提高效率

直接上需求把！！！

1，私有协议栈需求

私有协议为40字节的消息头+Json格式的消息体，如下：

2，消息封装

NettyMessage.class用于消息头+消息体的封装

public final class NettyMessage {

    private Header header;

    private Object body;

    /**
     * @return the header
     */
    public final Header getHeader() {
	return header;
    }

    /**
     * @param header
     *            the header to set
     */
    public final void setHeader(Header header) {
	this.header = header;
    }

    /**
     * @return the body
     */
    public final Object getBody() {
	return body;
    }

    /**
     * @param body
     *            the body to set
     */
    public final void setBody(Object body) {
	this.body = body;
    }

    @Override
    public String toString() {
        return "NettyMessage{" +
                "header=" + header +
                ", body=" + body +
                '}';
    }
}

Header.class用于消息头的封装，因为消息头里面里面数据是占固定字节的，可以使用Java的不同数据类型定义来占用固定字节

public final class Header {

    /**
     * 版本号占2个字节
     */
    private short sVersion;

    /**
     * 命令类型占2个字节
     */
    private short sCommand;

    /**
     * 包体长度占4个字节
     */
    private int nPacketLen;

    /**
     * 命令标识，建议获取当前时间的毫秒时间+4位随机数
     * 占8个字节
     */
    private long uIdentity;

    /**
     * 具体命令，详见具体命令章节
     * 占2个字节
     */
    private short pCmd;

    /**
     * 账户名要求64位的数字
     * 占8个字节
     */
    private long account;

    /**
     * 密码要求64位的数字
     * 占8个字节
     */
    private long password;

    /**
     * szSession占六个字节
     */
    private byte[] szSession;


    /**
     * 本地命令为0，集控命令为1
     */
    private int flag;


    public short getsVersion() {
        return sVersion;
    }

    public void setsVersion(short sVersion) {
        this.sVersion = sVersion;
    }

    public short getsCommand() {
        return sCommand;
    }

    public void setsCommand(short sCommand) {
        this.sCommand = sCommand;
    }

    public int getnPacketLen() {
        return nPacketLen;
    }

    public void setnPacketLen(int nPacketLen) {
        this.nPacketLen = nPacketLen;
    }

    public long getuIdentity() {
        return uIdentity;
    }

    public void setuIdentity(long uIdentity) {
        this.uIdentity = uIdentity;
    }

    public short getpCmd() {
        return pCmd;
    }

    public void setpCmd(short pCmd) {
        this.pCmd = pCmd;
    }

    public long getAccount() {
        return account;
    }

    public void setAccount(long account) {
        this.account = account;
    }

    public long getPassword() {
        return password;
    }

    public void setPassword(long password) {
        this.password = password;
    }

    public byte[] getSzSession() {
        return szSession;
    }

    public void setSzSession(byte[] szSession) {
        this.szSession = szSession;
    }

    public int getFlag() {
        return flag;
    }

    public void setFlag(int flag) {
        this.flag = flag;
    }

    @Override
    public String toString() {
        return "Header{" +
                "sVersion=" + sVersion +
                ", sCommand=" + sCommand +
                ", nPacketLen=" + nPacketLen +
                ", uIdentity=" + uIdentity +
                ", pCmd=" + pCmd +
                ", account=" + account +
                ", password=" + password +
                ", szSession=" + Arrays.toString(szSession) +
                '}';
    }
}

3，序列化数据

Java默认提供的序列化机制，需要序列化的Java对象只需要实现 Serializable / Externalizable 接口并生成序列化ID，这个类就能够通过 ObjectInput 和 ObjectOutput 序列化和反序列化，不过Java的序列化效率低，而且不支持跨语言比如C,C++,目前主流的可跨语言的序列化方式有谷歌的Protobuf和Jboss的Marshalling，这里我们使用Marshalling

(1)MarshallingCodecFactory.class

public final class MarshallingCodecFactory {

    /**
     * 创建Jboss Marshaller
     * 
     * @return
     * @throws IOException
     */
    protected static Marshaller buildMarshalling() throws IOException {
	final MarshallerFactory marshallerFactory = Marshalling
		.getProvidedMarshallerFactory("serial");
	final MarshallingConfiguration configuration = new MarshallingConfiguration();
	configuration.setVersion(5);
	Marshaller marshaller = marshallerFactory
		.createMarshaller(configuration);
	return marshaller;
    }

    /**
     * 创建Jboss Unmarshaller
     * 
     * @return
     * @throws IOException
     */
    protected static Unmarshaller buildUnMarshalling() throws IOException {
	final MarshallerFactory marshallerFactory = Marshalling
		.getProvidedMarshallerFactory("serial");
	final MarshallingConfiguration configuration = new MarshallingConfiguration();
	configuration.setVersion(5);
	final Unmarshaller unmarshaller = marshallerFactory
		.createUnmarshaller(configuration);
	return unmarshaller;
    }
}

(2)ByteBuf的数据读写封装类

class ChannelBufferByteInput implements ByteInput {

    private final ByteBuf buffer;

    public ChannelBufferByteInput(ByteBuf buffer) {
        this.buffer = buffer;
    }

    @Override
    public void close() throws IOException {
        // nothing to do
    }

    @Override
    public int available() throws IOException {
        return buffer.readableBytes();
    }

    @Override
    public int read() throws IOException {
        if (buffer.isReadable()) {
            return buffer.readByte() & 0xff;
        }
        return -1;
    }

    @Override
    public int read(byte[] array) throws IOException {
        return read(array, 0, array.length);
    }

    @Override
    public int read(byte[] dst, int dstIndex, int length) throws IOException {
        int available = available();
        if (available == 0) {
            return -1;
        }

        length = Math.min(available, length);
        buffer.readBytes(dst, dstIndex, length);
        return length;
    }

    @Override
    public long skip(long bytes) throws IOException {
        int readable = buffer.readableBytes();
        if (readable < bytes) {
            bytes = readable;
        }
        buffer.readerIndex((int) (buffer.readerIndex() + bytes));
        return bytes;
    }

}

class ChannelBufferByteOutput implements ByteOutput {

    private final ByteBuf buffer;

    /**
     * Create a new instance which use the given {@link ByteBuf}
     */
    public ChannelBufferByteOutput(ByteBuf buffer) {
        this.buffer = buffer;
    }

    @Override
    public void close() throws IOException {
        // Nothing to do
    }

    @Override
    public void flush() throws IOException {
        // nothing to do
    }

    @Override
    public void write(int b) throws IOException {
        buffer.writeByte(b);
    }

    @Override
    public void write(byte[] bytes) throws IOException {
        buffer.writeBytes(bytes);
    }

    @Override
    public void write(byte[] bytes, int srcIndex, int length) throws IOException {
        buffer.writeBytes(bytes, srcIndex, length);
    }

    /**
     * Return the {@link ByteBuf} which contains the written content
     *
     */
    ByteBuf getBuffer() {
        return buffer;
    }
}

(3)MarshallingEncoder.class序列化消息体

@Sharable
public class MarshallingEncoder {

    private static final byte[] LENGTH_PLACEHOLDER = new byte[4];
    Marshaller marshaller;

    public MarshallingEncoder() throws IOException {
	marshaller = MarshallingCodecFactory.buildMarshalling();
    }

    protected void encode(Object msg, ByteBuf out) throws Exception {
		System.out.println("-----------------编码时进行序列化--------------------");
	try {
	    int lengthPos = out.writerIndex();
	    out.writeBytes(LENGTH_PLACEHOLDER);
	    ChannelBufferByteOutput output = new ChannelBufferByteOutput(out);
	    marshaller.start(output);
	    marshaller.writeObject(msg);
	    marshaller.finish();
	    out.setInt(lengthPos, out.writerIndex() - lengthPos - 4);
	} finally {
	    marshaller.close();
	}
    }
}

(4)MarshallingDncoder.class反序列化消息体

public class MarshallingDecoder {

    private final Unmarshaller unmarshaller;

    /**
     * Creates a new decoder whose maximum object size is {@code 1048576} bytes.
     * If the size of the received object is greater than {@code 1048576} bytes,
     * a {@link StreamCorruptedException} will be raised.
     * 
     * @throws IOException
     * 
     */
    public MarshallingDecoder() throws IOException {
	unmarshaller = MarshallingCodecFactory.buildUnMarshalling();
    }

    protected Object decode(ByteBuf in) throws Exception {
		System.out.println("-----------------解码时进行反序列化--------------------");
	int objectSize = in.readInt();
	ByteBuf buf = in.slice(in.readerIndex(), objectSize);
	ByteInput input = new ChannelBufferByteInput(buf);
	try {
	    unmarshaller.start(input);
	    Object obj = unmarshaller.readObject();
	    unmarshaller.finish();
	    in.readerIndex(in.readerIndex() + objectSize);
	    return obj;
	} finally {
	    unmarshaller.close();
	}
    }
}

4，编码解码器

(1)NettyMessageEncoder.class消息编码器，对消息头每个参数固定字节，对消息体进行序列化

public final class NettyMessageEncoder extends MessageToByteEncoder<NettyMessage> {

    MarshallingEncoder marshallingEncoder;

    public NettyMessageEncoder() throws IOException {
	this.marshallingEncoder = new MarshallingEncoder();
    }

    @Override
    protected void encode(ChannelHandlerContext ctx, NettyMessage msg,ByteBuf sendBuf) throws Exception {
		System.out.println("-----------------开始进行编码--------------------");
	if (msg == null || msg.getHeader() == null){
		throw new Exception("The encode message is null");
	}else {
		// 信息写入到流里面并且固定字节
		sendBuf.writeShort(msg.getHeader().getsVersion());
		sendBuf.writeShort(msg.getHeader().getsCommand());
		sendBuf.writeInt(NettyUtil.getByteNum(msg.getBody()));
		sendBuf.writeLong(msg.getHeader().getuIdentity());
		sendBuf.writeShort(msg.getHeader().getpCmd());
		if (msg.getHeader().getFlag()==0) {
			// 本地tcp方式
			msg.getHeader().setSzSession(new byte[22]) ;
			sendBuf.writeBytes(msg.getHeader().getSzSession());
		}else {
			// 集控tcp方式
			sendBuf.writeLong(msg.getHeader().getAccount());
			sendBuf.writeLong(msg.getHeader().getPassword());
			msg.getHeader().setSzSession(new byte[6]);
			sendBuf.writeBytes(msg.getHeader().getSzSession());
		}

	}

	if (msg.getBody() != null) {
		Object body = msg.getBody();
		String jsonString = JSON.toJSONString(body);
		marshallingEncoder.encode(jsonString, sendBuf);
	}else {
		sendBuf.writeInt(0);
	}

	}

(2)NettyMessageDecoder.class消息编码器，对消息头每个参数进行读取，对消息体进行反序列化

public class NettyMessageDecoder extends LengthFieldBasedFrameDecoder {


    MarshallingDecoder marshallingDecoder;

    public NettyMessageDecoder(int maxFrameLength, int lengthFieldOffset,
	    int lengthFieldLength) throws IOException {
	super(maxFrameLength, lengthFieldOffset, lengthFieldLength);
	marshallingDecoder = new MarshallingDecoder();
    }

    @Override
    protected Object decode(ChannelHandlerContext ctx, ByteBuf frame) throws Exception {
	System.out.println("-----------------开始进行解码--------------------");
	System.err.println("frame.readableBytes():"+frame.readableBytes());
	NettyMessage message = new NettyMessage();
	Header header = new Header();
	header.setsVersion(frame.readShort());
	header.setsCommand(frame.readShort());
	header.setnPacketLen(frame.readInt());
	header.setuIdentity(frame.readLong());
	header.setpCmd(frame.readShort());
	// 获取22字节的szSession
	frame.readLong();
	frame.readLong();
	frame.readInt();
	frame.readShort();

	// 开始对消息包进行解码
	if (frame.readableBytes() > 4) {
	    message.setBody(marshallingDecoder.decode(frame));
	}
	message.setHeader(header);
	return message;
    }
}

5，服务端代码

NettyServer.class用于监听端口，处理客户端请求

public class NettyServer {

    public void bind(int port) throws Exception {
		//bossGroup就是parentGroup，是负责处理TCP/IP连接的
        EventLoopGroup bossGroup = new NioEventLoopGroup(); 
        //workerGroup就是childGroup,是负责处理Channel(通道)的I/O事件
        EventLoopGroup workerGroup = new NioEventLoopGroup(); 

        ServerBootstrap sb = new ServerBootstrap();
        sb.group(bossGroup, workerGroup)
                .channel(NioServerSocketChannel.class)
                 //初始化服务端可连接队列,指定了队列的大小128
                .option(ChannelOption.SO_BACKLOG, 128)
                //保持长连接
                .childOption(ChannelOption.SO_KEEPALIVE, true) 
                //channel通道的处理器
                .childHandler(new ServerChannelInitializer());
        //绑定监听端口，调用sync同步阻塞方法等待绑定操作完
        ChannelFuture future = sb.bind(port).sync();

        if (future.isSuccess()) {
            System.out.println("服务端启动成功");
        } else {
            System.out.println("服务端启动失败");
            future.cause().printStackTrace();
            bossGroup.shutdownGracefully(); //关闭线程组
            workerGroup.shutdownGracefully();
        }

        //成功绑定到端口之后,给channel增加一个 管道关闭的监听器并同步阻塞,直到channel关闭,线程才会往下执行,结束进程。
        future.channel().closeFuture().sync();

    }
}

ServerChannelInitializer.class用于初始化channel通道，包括编码器，解码器，处理器等等

/**
 * @author David
 * @className ServerChannelInitializer
 * @date 2020/3/23 21:36
 */
public class ServerChannelInitializer extends ChannelInitializer<SocketChannel> {
    @Override
    protected void initChannel(SocketChannel channel) throws Exception {
        // 传输数据的解码器
        channel.pipeline().addLast(new NettyMessageDecoder(1024 * 1024, 4, 4));
        // 传输数据的编码器
        channel.pipeline().addLast( new NettyMessageEncoder());
        // 服务端收据接收处理器
        channel.pipeline().addLast(new ServerHandler());

    }
}

ServerHandler.class用于接收处理客户端的消息

public class ServerHandler extends ChannelInboundHandlerAdapter{

    /**
     * 接受client发送的消息
     * @param ctx
     * @param msg
     * @throws Exception
     */
    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
        System.out.println("-----------------服务器开始输出消息--------------------");
        NettyMessage nettyMessage = (NettyMessage) msg;
        System.out.println("接收到客户端信息:" + nettyMessage.toString());
        //返回的数据结构
        JsonBody jsonBody = new JsonBody();
        jsonBody.setUuid(UUID.randomUUID().toString());
        jsonBody.setData("hello nettyclient，这是给你的回复!");
        // nettymessage
        NettyMessage sendNettyMessage = new NettyMessage();
        Header header = new Header();
        header.setsVersion((short) 0x6635);
        header.setsCommand((short) 0x2802);;
        header.setuIdentity((long)1232221);
        header.setnPacketLen(0);
        header.setpCmd((short)0x2901);
        header.setSzSession(null);
        header.setAccount((long) 42343244);
        header.setPassword((long) 31231213);
        header.setFlag(1);
        sendNettyMessage.setHeader(header);
        sendNettyMessage.setBody(jsonBody);
        ctx.writeAndFlush(sendNettyMessage);
    }

    /**
     * 通知处理器最后的channelRead()是当前批处理中的最后一条消息时调用
     * @param ctx
     * @throws Exception
     */
    @Override
    public void channelReadComplete(ChannelHandlerContext ctx) throws Exception {
        System.out.println("服务端接收数据完毕..");
        ctx.flush();
    }

    /**
     * 读操作时捕获到异常时调用
     * @param ctx
     * @param cause
     */
    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) {
        ctx.close();
    }

    /**
     * 客户端去和服务端连接成功时触发
     * @param ctx
     * @throws Exception
     */
    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
//        ctx.writeAndFlush("hello client");
    }
}

6，客户端代码

NettyClient.class用于建立连接，发送数据，配置处理器

public class NettyClient {

    private final String host;
    private final int port;
    private Channel channel;

    //连接服务端的端口号地址和端口号
    public NettyClient(String host, int port) {
        this.host = host;
        this.port = port;
    }

    public void start() throws Exception {

        final EventLoopGroup group = new NioEventLoopGroup();
        Bootstrap b = new Bootstrap();
        // 使用NioSocketChannel来作为连接用的channel类
        b.group(group).channel(NioSocketChannel.class)
                .handler(new ClientChannelInitializer());
        //发起异步连接请求，绑定连接端口和host信息
        final ChannelFuture future = b.connect(host, port).sync();

        future.addListener(new ChannelFutureListener() {

            @Override
            public void operationComplete(ChannelFuture arg0) throws Exception {
                if (future.isSuccess()) {
                    System.out.println("连接服务器成功");

                } else {
                    System.out.println("连接服务器失败");
                    future.cause().printStackTrace();
                    group.shutdownGracefully(); //关闭线程组
                }
            }
        });

        this.channel = future.channel();
    }

    public Channel getChannel() {
        return channel;
    }

ClientChannelInitializer.class用于初始化channel通道，包括编码器，解码器，处理器等等

public class ClientChannelInitializer extends ChannelInitializer<SocketChannel> {

    @Override
    protected void initChannel(SocketChannel channel) throws Exception {
        // 解码器
        channel.pipeline().addLast(new NettyMessageDecoder(1024 * 1024, 4, 4));
        // 编码器
        channel.pipeline().addLast( new NettyMessageEncoder());
        // 处理器
        channel.pipeline().addLast(new ClientHandler());
}
}

ClientHandler .class用于客户端消息处理，处理服务端返回的数据

/**
 * @author david
 * client消息处理类
 */
public class ClientHandler extends SimpleChannelInboundHandler<NettyMessage> {

    /**
     * 处理服务端返回的数据
     * @param ctx
     * @param msg
     * @throws Exception
     */
    @Override
    protected void channelRead0(ChannelHandlerContext ctx, NettyMessage msg) throws Exception {
        System.out.println("接受到server响应数据: " + msg.toString());
    }

    @Override
    public void channelActive(ChannelHandlerContext ctx) throws Exception {
        super.channelActive(ctx);
    }

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
        ctx.close();
    }



}

7，测试

启动服务端

public class NettyServerStart {

    public static void main(String[] args) throws Exception {

        new NettyServer().bind(8080);
    }
}

启动客户端

public class NettyClientStart {

    public static void main(String[] args) throws Exception {

        NettyClient client = new NettyClient(NettyConstant.LOCAL_IP, NettyConstant.LOCAL_PORT);
        //启动client服务
        client.start();
        Channel channel = client.getChannel();
        
        JsonBody jsonBody = new JsonBody();
        jsonBody.setUuid("qwertyuiopuyiyiyiyiyit沪深 三分毒");
        jsonBody.setData("hello nettyserver!");
        // nettymessage
        NettyMessage nettyMessage = new NettyMessage();
        Header header = new Header();
        header.setsVersion((short) 0x6635);
        header.setsCommand((short) 0x2802);;
        header.setuIdentity((long)1232221);
        header.setpCmd((short)0x2901);
        header.setSzSession(null);
        header.setAccount((long) 121212151);
        header.setPassword((long) 784514544);
        header.setFlag(1);
        nettyMessage.setHeader(header);
        nettyMessage.setBody(jsonBody);

        //channel对象可保存在map中，供其它地方发送消息
        channel.writeAndFlush(nettyMessage);
    }

}

服务端启动成功接收到消息，并返回消息，40字节的消息头没有错位

客户端发送消息，并接收服务器消息成功

总结：如上结束基于Netty的私有协议栈开发，其中还有登录模块，心跳模块，超时模块会在后面给出