数据序列化框架——Kryo
1. 概览
-
简介
Kryo是一款专用于Java的快速、高效的数据序列化的框架,同时也支持Scala、Clojure、Objective-C等。目前已被多种大型项目使用,包括:Apache Hive、Apache Spark 、Storm、Apache Dubbo 等。 -
特点
- 快速、高效、占用空间小
- 使用简单(不用像protobuf一样编写协议文件)
- 支持浅拷贝、深拷贝功能
- 专用于Java(如果你的服务器全是Java端,那么采用Kryo是一个非常好的选择)
-
GitHub地址:https://github.com/EsotericSoftware/kryo
注: 如果你觉得下载依赖包麻烦,也可以使用本博客提供的jar包
2. Java中使用Kryo
- 添加maven依赖
<dependency> <groupId>com.esotericsoftware</groupId> <artifactId>kryo</artifactId> <version>3.0.3</version> </dependency>
- 导包
import com.esotericsoftware.kryo.Kryo; import com.esotericsoftware.kryo.io.Input; import com.esotericsoftware.kryo.io.Output; import java.io.FileInputStream; import java.io.FileNotFoundException; import java.io.FileOutputStream;
- 待使用的对象Person
public class Person { private String name; private int age; private String address; private Person parent; public Person() { } public Person(String name, int age, String address) { this.name = name; this.age = age; this.address = address; } public Person(String name, int age, String address, Person parent) { this.name = name; this.age = age; this.address = address; this.parent = parent; } public String getName() { return name; } public void setName(String name) { this.name = name; } public Person getParent() { return parent; } public void setParent(Person parent) { this.parent = parent; } @Override public String toString() { return "Person{" + "name='" + name + '\'' + ", age=" + age + ", address='" + address + '\'' + ", parent=" + parent + '}'; } }
- 文件序列化与反序列化
Kryo kryo = new Kryo(); kryo.register(Person.class); Person parent = new Person("老王", 38, "BeiJing"); System.out.println("--------> 序列化到文件中"); Output output = new Output(new FileOutputStream("file.bin")); kryo.writeObject(output, parent); output.close(); Input input = new Input(new FileInputStream("file.bin")); Person object1 = kryo.readObject(input, Person.class); input.close();
- byte序列化与反序列化
Kryo kryo = new Kryo(); kryo.register(Person.class); Person parent = new Person("老王", 38, "BeiJing"); System.out.println("--------> byte方式序列化"); // -1 表示maxBufferSize可以无限制,源码中是给的Integer.MAX_VALUE Output output2 = new Output(1024, -1); kryo.writeObject(output2, parent); byte[] bytes = output2.getBuffer(); Input input2 = new Input(bytes); Person object2 = kryo.readObject(input2, Person.class); System.out.println("object2 = " + object2);
- 深拷贝与浅拷贝
Kryo kryo = new Kryo(); kryo.register(Person.class); Person parent = new Person("老王", 38, "BeiJing"); Person person = new Person("小明", 18, "BeiJing", parent); System.out.println("--------> 深拷贝"); Person copyPerson2 = kryo.copy(person); copyPerson2.getParent().setName("老李"); System.out.println("copyPerson2 = " + copyPerson2); System.out.println("person = " + person); System.out.println("--------> 浅拷贝"); Person copyPerson = kryo.copyShallow(person); copyPerson.getParent().setName("老李"); System.out.println("copyPerson = " + copyPerson); System.out.println("person = " + person);
3. Scala中使用Kryo
注: 我所使用的scala是2.11.8版本
- Scala中使用Kryo非常简单,但还需要额外再引入几个Twitter的工具包(文章顶部有下载)
- twitter/chill : chill_2.11-0.8.0.jar, chill-java-0.8.0.jar
- twitter/bijection: bijection-core_2.11-0.9.1.jar, chill-bijection_2.11-0.9.1.jar
- jar包也可以从maven仓库下载,pom.xml依赖如下
<dependency> <groupId>com.twitter</groupId> <artifactId>chill_2.11</artifactId> <version>0.8.0</version> </dependency> <dependency> <groupId>com.twitter</groupId> <artifactId>chill-java</artifactId> <version>0.8.0</version> </dependency> <dependency> <groupId>com.twitter</groupId> <artifactId>bijection-core_2.11</artifactId> <version>0.9.1</version> </dependency> <dependency> <groupId>com.twitter</groupId> <artifactId>chill-bijection_2.11</artifactId> <version>0.9.1</version> </dependency>
- 使用示例
import com.twitter.chill.KryoInjection /** * Descr: Scala Kryo序列化的Injection示例 * Date: 2019/8/21 14:50 * * @author A Lion~ */ object ScalaDemo { def main(args: Array[String]): Unit = { // 构建对象 case class Person(name: String, age: Int, address: String) val person = Person("小明", 18, "BeiJing") // 将对象转为byte数组 val bytes: Array[Byte] = KryoInjection(person) // 解析byte数组,转为对象 val tryDecode: scala.util.Try[Any] = KryoInjection.invert(bytes) println(s"tryDecode = ${tryDecode.get}") } }