距离上一篇Netty的文章已经一个星期了,哈哈哈。周末学习了一下关于序列化的,先做个笔记先。
Java序列化,就那么两个目的,第一是进行网络传输,第二就是对象持久化到磁盘。虽然在Netty传输中我们可以继续使用Java进行对象序列化,但是Java序列化的硬伤太多了。比如无法跨语言,序列后码流太大,序列化性能太低等等。
现在主流的编解码框架有:
JBoss的Marshalling包
google的Protobuf
基于Protobuf的Kyro
MessagePack框架
下面呢,我们将用到的是JBoss的Marshalling包。Jboss Marshalling是一个Java对象序列化包,但是呢,对JDK默认的序列化框架进行了优化,但又保持跟java.io.Serializable接口的兼容,同时增加了一些可调的参数和附加特性。
类库:jboss-marshalling-1.3.0、jboss-marshalling-serial-1.3.0
下载地址:https://www.jboss.org/jbossmarshalling/downloads
JbossMarshalling与Netty结合后进行序列化对象的代码编写非常的简单,我们下面看一下例子代码:
其实代码和之前的例子是差不多的,只是添加的编解码器变了,发送和接收数据的逻辑也变了,变为直接将msg转为POJO。最重要的是自己创建的根据Jboss Marshalling编写编解码器。
下面直接上例子:
Server:
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelOption;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.logging.LoggingHandler;
public class Server{
public static void main(String[] args) throws InterruptedException {
//1、创建一个线程组来处理服务器接收客户端连接
EventLoopGroup bossGroup = new NioEventLoopGroup();
//2、创建一个线程组来进行网络通信
EventLoopGroup workGroup = new NioEventLoopGroup();
//3、创建辅助工具类Bootstrap,用于服务端通道的配置
ServerBootstrap sb = new ServerBootstrap();
sb.group(bossGroup, workGroup)
.handler(new LoggingHandler()) //设置日志
.channel(NioServerSocketChannel.class) //指定NIO模式
.option(ChannelOption.SO_BACKLOG, 1014) //设置tcp缓冲区大小
.option(ChannelOption.SO_SNDBUF, 32*1024) //设置发送缓冲区大小
.option(ChannelOption.SO_RCVBUF,32*1024) //设置接收缓冲区大小
.option(ChannelOption.SO_KEEPALIVE, true) //保持连接(长连接)
.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
//添加mashalling的编解码
ch.pipeline().addLast(MarshallingCodeFactory.buildMarshallingEncoder());
ch.pipeline().addLast(MarshallingCodeFactory.buildMarshallingDecoder());
ch.pipeline().addLast(new ServerHandler()); //配置具体的数据处理类
}
});
ChannelFuture f = sb.bind(9876).sync(); //绑定端口,客户端连接需要知道端口
//异步等待关闭(记得不是close()方法)
f.channel().closeFuture().sync();
//当通道关闭后,将线程组也关闭
bossGroup.shutdownGracefully();
workGroup.shutdownGracefully();
}
}
ServerHandler:继承ChannelHandlerAdapter
import io.netty.channel.ChannelHandlerAdapter;
import io.netty.channel.ChannelHandlerContext;
public class ServerHandler extends ChannelHandlerAdapter{
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
//因为有marshalling编解码器,所以可以直接将msg强转为RequestEntity
RequestEntity entity = (RequestEntity)msg;
System.out.println(entity.getId()+" "+entity.getName());
ResponseEntity response = new ResponseEntity(entity.getId(), "server"+entity.getId());
ctx.writeAndFlush(response);
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception {
cause.printStackTrace();
ctx.close(); //出现异常的话就关闭
}
}
Client:
import io.netty.bootstrap.Bootstrap;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
public class Client {
public static void main(String[] args) throws InterruptedException {
//和服务端不一样,客户端只需要一个线程组来处理网络通信即可
EventLoopGroup workGroup = new NioEventLoopGroup();
//创建辅助类,和服务器的不一样,服务端的是ServerBootStrap,而客户端的是BootStrap
Bootstrap bs = new Bootstrap();
bs.group(workGroup)
.channel(NioSocketChannel.class) //设置tcp缓冲区大小
.handler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
//添加mashalling的编解码
ch.pipeline().addLast(MarshallingCodeFactory.buildMarshallingEncoder());
ch.pipeline().addLast(MarshallingCodeFactory.buildMarshallingDecoder());
ch.pipeline().addLast(new ClientHandler());
}
});
//连接服务端
ChannelFuture cf = bs.connect("127.0.0.1", 9876).sync();
//给服务端写数据
for(int i=0;i<10;i++){
RequestEntity entity = new RequestEntity(Integer.toString(i), "Client"+i);
cf.channel().writeAndFlush(entity);
}
//异步监听管道的关闭,如果关闭了就往下执行
cf.channel().closeFuture().sync();
workGroup.shutdownGracefully();
}
}
ClientHandler:
import io.netty.channel.ChannelHandlerAdapter;
import io.netty.channel.ChannelHandlerContext;
import io.netty.util.ReferenceCountUtil;
public class ClientHandler extends ChannelHandlerAdapter{
@Override
public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
//直接将msg强转为ResponseEntity
ResponseEntity entity = (ResponseEntity) msg;
System.out.println(entity.getId()+" "+entity.getName());
//最后如果没有回写记得释放
ReferenceCountUtil.release(msg);
}
@Override
public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception{
cause.printStackTrace();
ctx.close();
}
}
Marshlling工厂:可以创建Marshalling编解码器(这个是重点,以后都可以直接当工具类拿来用)
import org.jboss.marshalling.MarshallerFactory;
import org.jboss.marshalling.Marshalling;
import org.jboss.marshalling.MarshallingConfiguration;
import io.netty.handler.codec.marshalling.DefaultMarshallerProvider;
import io.netty.handler.codec.marshalling.DefaultUnmarshallerProvider;
import io.netty.handler.codec.marshalling.MarshallerProvider;
import io.netty.handler.codec.marshalling.MarshallingDecoder;
import io.netty.handler.codec.marshalling.MarshallingEncoder;
import io.netty.handler.codec.marshalling.UnmarshallerProvider;
public class MarshallingCodeFactory {
/**
* 创建Jboss Marshalling解码器MarshallingDecoder
* @return MarshallingDecoder
*/
public static MarshallingDecoder buildMarshallingDecoder(){
//首先通过Marshalling工具类的静态方法获取Marshalling实例对象 参数为serial标识创建的是java序列化工厂对象
final MarshallerFactory marshallerFactory = Marshalling.getProvidedMarshallerFactory("serial");
//创建MarshallingConfiguration对象,配置版本号为5
final MarshallingConfiguration configuration = new MarshallingConfiguration();
configuration.setVersion(5);
//根据marshallerFactory和configuration创建provider
UnmarshallerProvider provider = new DefaultUnmarshallerProvider(marshallerFactory, configuration);
//构建Netty的marshallingDecoder对象,两个参数分别为provider和单个消息序列化后的最大长度
MarshallingDecoder decoder = new MarshallingDecoder(provider, 1024*1024*1);
return decoder;
}
/**
* 创建Jboss Marshalling编码器MarshallingEncoder
* @return MarshallingDecoder
*/
public static MarshallingEncoder buildMarshallingEncoder(){
//首先通过Marshalling工具类的静态方法获取Marshalling实例对象 参数为serial标识创建的是java序列化工厂对象
final MarshallerFactory marshallerFactory = Marshalling.getProvidedMarshallerFactory("serial");
//创建MarshallingConfiguration对象,配置版本号为5
final MarshallingConfiguration configuration = new MarshallingConfiguration();
configuration.setVersion(5);
//根据marshallerFactory和configuration创建provider
MarshallerProvider provider = new DefaultMarshallerProvider(marshallerFactory, configuration);
//构建Netty的marshallingDecoder对象,用于将实现序列化接口的POJO对象序列化成二进制数组
MarshallingEncoder encoder = new MarshallingEncoder(provider);
return encoder;
}
}
请求实体类和响应实体类:记得一定要实现serializable接口
import java.io.Serializable;
public class RequestEntity implements Serializable{
private static final long serialVersionUID = 1L;
private String id;
private String name;
public RequestEntity(String id, String name) {
super();
this.id = id;
this.name = name;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
package com.demo.testNetty.tcp.serial;
import java.io.Serializable;
public class ResponseEntity implements Serializable{
private static final long serialVersionUID = 1L;
private String id;
private String name;
public ResponseEntity(String id, String name) {
super();
this.id = id;
this.name = name;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
不过需要注意的是,现在是服务端和客户端都是同一个项目,正常来说可能客户端和服务端是同的项目,不同的服务器,那么我们需要知道,请求和响应实体类在两个端的名字和包路径一定要一模一样,不然是会通信出错的。这就好像自定义消息体一样,双方要协定好,双方都要遵循这个自定义协议才能通信成功。
上面的POJO只是简单的基本类型属性,那么如果要传输文件呢。这时候我们可以加上字节数组的属性,但是需要注意的是,传输之前,发送的一方一定要压缩,然后接收的那一方记得要解压缩。
那么我们先看一下解压缩的工具类:它用到的是JDK自带的解压缩流。
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.util.zip.GZIPInputStream;
import java.util.zip.GZIPOutputStream;
public class GzipUtil {
public static byte[] gzip(byte[] data) throws Exception{
ByteArrayOutputStream bos = new ByteArrayOutputStream();
GZIPOutputStream gzip = new GZIPOutputStream(bos);
gzip.write(data); //将解压后的数据写到bos里面
gzip.flush();
gzip.close();
byte[] result = bos.toByteArray(); //将bos里面的数据转为字节数组
bos.close();
return result;
}
public static byte[] ungzip(byte[] data) throws Exception{
ByteArrayInputStream bis = new ByteArrayInputStream(data);
GZIPInputStream gzip = new GZIPInputStream(bis); //将数据放进gzip
byte[] buf = new byte[1024];
int num = -1;
ByteArrayOutputStream bos = new ByteArrayOutputStream();
while((num = gzip.read(buf, 0, buf.length)) != -1){ //每次从gzip中读一个字节的数据
bos.write(buf,0,num); //将读取的数据放进bos中
}
gzip.close();
bis.close();
byte[] result = bos.toByteArray(); //将bos里面的数据转为字节数组
bos.flush();
bos.close();
return result;
}
public static void main(String[] args) throws Exception {
//读取文件
//System.getProperty可以获取当前系统信息,这里的user.dir表示当前路径
//File.separatorChar表示文件的分隔符,兼容各种操作系统
String filepath = System.getProperty("user.dir")+File.separatorChar+"lib"+File.separatorChar+"sigar.jar";
File file = new File(filepath);
FileInputStream fis = new FileInputStream(file);
byte[] data = new byte[fis.available()];
fis.read(data); //将数据读到字节数组
fis.close();
System.out.println("文件的原始的大小为:"+data.length);
//进行压缩
byte[] result1 = gzip(data);
System.out.println("压缩后的大小:"+result1.length);
//解压缩
byte[] result2 = ungzip(result1);
System.out.println("解压缩后的大小"+result2.length);
//写出文件
String path = System.getProperty("user.dir")+File.separatorChar+"lib"+File.separatorChar+"sigar2.jar";
FileOutputStream fos = new FileOutputStream(path);
fos.write(result2);
fos.close();
}
}
好了,现在我们将进行测试一下。
第一,在RequestEntity加上字节数组来存放文件字节数据
private byte[] attachment;
第二,修改客户端给服务端写数据的代码:
String id = "1";
String name = "客户端发送的数据";
//将压缩后的字节数组也放进实体中
String filePath = System.getProperty("user.dir")+File.separatorChar+"lib"+File.separatorChar+"sigar.jar";
FileInputStream fis = new FileInputStream(new File(filePath));
byte[] data = new byte[fis.available()];
fis.read(data);
fis.close();
//记得进行压缩
byte[] result = GzipUtil.gzip(data);
RequestEntity entity = new RequestEntity(id, name);
entity.setAttachment(result);
cf.channel().writeAndFlush(entity); //往服务端里头写
第三,修改服务端Handler接收数据的逻辑:
//因为有marshalling编解码器,所以可以直接将msg强转为RequestEntity
RequestEntity entity = (RequestEntity)msg;
System.out.println(entity.getId()+" "+entity.getName());
//将文件的字节数组拿出来,然后写出来
byte[] data = entity.getAttachment();
//记得进行解压缩
byte[] result = GzipUtil.ungzip(data);
String filePath = System.getProperty("user.dir")+File.separatorChar+"recive"+File.separatorChar+"test.jar";
//将文件写回项目中
FileOutputStream fos = new FileOutputStream(new File(filePath));
fos.write(result);
fos.close();
ResponseEntity response = new ResponseEntity(entity.getId(), "server"+entity.getId());
ctx.writeAndFlush(response);
结果:最后确实成功了。