流行的序列化框架有很多，常见的 Avro、Hessian、Protostuff、Thrift等，致力解决各种场景的数据序列化提高程序运行效率，基于现有的序列化行为不同的序列化协议不同程度的实现了消息协议进而提供了基础的RPC程序调用。通常应用程序的开发都是基于RPC层面进行，大大简化了跨进程通信的难度(IPC)。
Apache Avro不仅仅是一个数据序列化协议，更提供一个良好定义的IPC消息协议，同时提供更高级别的RPC封装，使得基于Apache Avro的RPC调用变得更快捷。Apache Avro 提供了三种不同的文件格式分别用来定义数据序列化协议(Schema ),接口协议(Protocol ),接口定义语言(IDL)分别以.avsc,.avdr,.avdl作为文件标识。可使用Avro提供的工具生成对应的平台代码文件。 eclipse开发环境下可使用maven插件自动生成代码。具体文件格式请移步 Avro官网 http://avro.apache.org/docs/current/index.html 。
下面以Apache Avro实现一个简单的RPC调用
定义协议接口
Avro使用.avsc 文件定义IPC接口文件,文件“user.avsc” 定义了一个序列化对象User，其位于com.xiaofen.record包下类名为User有三个字段分别为id，name，password。
{
"namespace":"com.xiaofen.record",
"type":"record",
"name":"User",
"fields": [
{"name": "id", "type": "string"},
{"name": "name", "type": "string"},
{"name": "password", "type": "string"}
]
}
使用maven avro插件生成的Java代码如下,生成的代码除定义Bean属性外大部分都是在定义序列化及反序列化行为。
/**
* Autogenerated by Avro
*
* DO NOT EDIT DIRECTLY
*/
package com.xiaofen.record;
import org.apache.avro.specific.SpecificData;
import org.apache.avro.message.BinaryMessageEncoder;
import org.apache.avro.message.BinaryMessageDecoder;
import org.apache.avro.message.SchemaStore;
@SuppressWarnings("all")
@org.apache.avro.specific.AvroGenerated
public class User extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {
private static final long serialVersionUID = 3366767355238404410L;
public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"User\",\"namespace\":\"com.xiaofen.record\",\"fields\":[{\"name\":\"id\",\"type\":\"string\"},{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"password\",\"type\":\"string\"}]}");
public static org.apache.avro.Schema getClassSchema() { return SCHEMA$; }
private static SpecificData MODEL$ = new SpecificData();
private static final BinaryMessageEncoder<User> ENCODER =
new BinaryMessageEncoder<User>(MODEL$, SCHEMA$);
private static final BinaryMessageDecoder<User> DECODER =
new BinaryMessageDecoder<User>(MODEL$, SCHEMA$);
/**
* Return the BinaryMessageDecoder instance used by this class.
*/
public static BinaryMessageDecoder<User> getDecoder() {
return DECODER;
}
/**
* Create a new BinaryMessageDecoder instance for this class that uses the specified {@link SchemaStore}.
* @param resolver a {@link SchemaStore} used to find schemas by fingerprint
*/
public static BinaryMessageDecoder<User> createDecoder(SchemaStore resolver) {
return new BinaryMessageDecoder<User>(MODEL$, SCHEMA$, resolver);
}
/** Serializes this User to a ByteBuffer. */
public java.nio.ByteBuffer toByteBuffer() throws java.io.IOException {
return ENCODER.encode(this);
}
/** Deserializes a User from a ByteBuffer. */
public static User fromByteBuffer(
java.nio.ByteBuffer b) throws java.io.IOException {
return DECODER.decode(b);
}
@Deprecated public java.lang.CharSequence id;
@Deprecated public java.lang.CharSequence name;
@Deprecated public java.lang.CharSequence password;
/**
* Default constructor. Note that this does not initialize fields
* to their default values from the schema. If that is desired then
* one should use <code>newBuilder()</code>.
*/
public User() {}
/**
* All-args constructor.
* @param id The new value for id
* @param name The new value for name
* @param password The new value for password
*/
public User(java.lang.CharSequence id, java.lang.CharSequence name, java.lang.CharSequence password) {
this.id = id;
this.name = name;
this.password = password;
}
public org.apache.avro.Schema getSchema() { return SCHEMA$; }
// Used by DatumWriter. Applications should not call.
public java.lang.Object get(int field$) {
switch (field$) {
case 0: return id;
case 1: return name;
case 2: return password;
default: throw new org.apache.avro.AvroRuntimeException("Bad index");
}
}
// Used by DatumReader. Applications should not call.
@SuppressWarnings(value="unchecked")
public void put(int field$, java.lang.Object value$) {
switch (field$) {
case 0: id = (java.lang.CharSequence)value$; break;
case 1: name = (java.lang.CharSequence)value$; break;
case 2: password = (java.lang.CharSequence)value$; break;
default: throw new org.apache.avro.AvroRuntimeException("Bad index");
}
}
/**
* Gets the value of the 'id' field.
* @return The value of the 'id' field.
*/
public java.lang.CharSequence getId() {
return id;
}
/**
* Sets the value of the 'id' field.
* @param value the value to set.
*/
public void setId(java.lang.CharSequence value) {
this.id = value;
}
/**
* Gets the value of the 'name' field.
* @return The value of the 'name' field.
*/
public java.lang.CharSequence getName() {
return name;
}
/**
* Sets the value of the 'name' field.
* @param value the value to set.
*/
public void setName(java.lang.CharSequence value) {
this.name = value;
}
/**
* Gets the value of the 'password' field.
* @return The value of the 'password' field.
*/
public java.lang.CharSequence getPassword() {
return password;
}
/**
* Sets the value of the 'password' field.
* @param value the value to set.
*/
public void setPassword(java.lang.CharSequence value) {
this.password = value;
}
/**
* Creates a new User RecordBuilder.
* @return A new User RecordBuilder
*/
public static com.xiaofen.record.User.Builder newBuilder() {
return new com.xiaofen.record.User.Builder();
}
/**
* Creates a new User RecordBuilder by copying an existing Builder.
* @param other The existing builder to copy.
* @return A new User RecordBuilder
*/
public static com.xiaofen.record.User.Builder newBuilder(com.xiaofen.record.User.Builder other) {
return new com.xiaofen.record.User.Builder(other);
}
/**
* Creates a new User RecordBuilder by copying an existing User instance.
* @param other The existing instance to copy.
* @return A new User RecordBuilder
*/
public static com.xiaofen.record.User.Builder newBuilder(com.xiaofen.record.User other) {
return new com.xiaofen.record.User.Builder(other);
}
/**
* RecordBuilder for User instances.
*/
public static class Builder extends org.apache.avro.specific.SpecificRecordBuilderBase<User>
implements org.apache.avro.data.RecordBuilder<User> {
private java.lang.CharSequence id;
private java.lang.CharSequence name;
private java.lang.CharSequence password;
/** Creates a new Builder */
private Builder() {
super(SCHEMA$);
}
/**
* Creates a Builder by copying an existing Builder.
* @param other The existing Builder to copy.
*/
private Builder(com.xiaofen.record.User.Builder other) {
super(other);
if (isValidValue(fields()[0], other.id)) {
this.id = data().deepCopy(fields()[0].schema(), other.id);
fieldSetFlags()[0] = true;
}
if (isValidValue(fields()[1], other.name)) {
this.name = data().deepCopy(fields()[1].schema(), other.name);
fieldSetFlags()[1] = true;
}
if (isValidValue(fields()[2], other.password)) {
this.password = data().deepCopy(fields()[2].schema(), other.password);
fieldSetFlags()[2] = true;
}
}
/**
* Creates a Builder by copying an existing User instance
* @param other The existing instance to copy.
*/
private Builder(com.xiaofen.record.User other) {
super(SCHEMA$);
if (isValidValue(fields()[0], other.id)) {
this.id = data().deepCopy(fields()[0].schema(), other.id);
fieldSetFlags()[0] = true;
}
if (isValidValue(fields()[1], other.name)) {
this.name = data().deepCopy(fields()[1].schema(), other.name);
fieldSetFlags()[1] = true;
}
if (isValidValue(fields()[2], other.password)) {
this.password = data().deepCopy(fields()[2].schema(), other.password);
fieldSetFlags()[2] = true;
}
}
/**
* Gets the value of the 'id' field.
* @return The value.
*/
public java.lang.CharSequence getId() {
return id;
}
/**
* Sets the value of the 'id' field.
* @param value The value of 'id'.
* @return This builder.
*/
public com.xiaofen.record.User.Builder setId(java.lang.CharSequence value) {
validate(fields()[0], value);
this.id = value;
fieldSetFlags()[0] = true;
return this;
}
/**
* Checks whether the 'id' field has been set.
* @return True if the 'id' field has been set, false otherwise.
*/
public boolean hasId() {
return fieldSetFlags()[0];
}
/**
* Clears the value of the 'id' field.
* @return This builder.
*/
public com.xiaofen.record.User.Builder clearId() {
id = null;
fieldSetFlags()[0] = false;
return this;
}
/**
* Gets the value of the 'name' field.
* @return The value.
*/
public java.lang.CharSequence getName() {
return name;
}
/**
* Sets the value of the 'name' field.
* @param value The value of 'name'.
* @return This builder.
*/
public com.xiaofen.record.User.Builder setName(java.lang.CharSequence value) {
validate(fields()[1], value);
this.name = value;
fieldSetFlags()[1] = true;
return this;
}
/**
* Checks whether the 'name' field has been set.
* @return True if the 'name' field has been set, false otherwise.
*/
public boolean hasName() {
return fieldSetFlags()[1];
}
/**
* Clears the value of the 'name' field.
* @return This builder.
*/
public com.xiaofen.record.User.Builder clearName() {
name = null;
fieldSetFlags()[1] = false;
return this;
}
/**
* Gets the value of the 'password' field.
* @return The value.
*/
public java.lang.CharSequence getPassword() {
return password;
}
/**
* Sets the value of the 'password' field.
* @param value The value of 'password'.
* @return This builder.
*/
public com.xiaofen.record.User.Builder setPassword(java.lang.CharSequence value) {
validate(fields()[2], value);
this.password = value;
fieldSetFlags()[2] = true;
return this;
}
/**
* Checks whether the 'password' field has been set.
* @return True if the 'password' field has been set, false otherwise.
*/
public boolean hasPassword() {
return fieldSetFlags()[2];
}
/**
* Clears the value of the 'password' field.
* @return This builder.
*/
public com.xiaofen.record.User.Builder clearPassword() {
password = null;
fieldSetFlags()[2] = false;
return this;
}
@Override
@SuppressWarnings("unchecked")
public User build() {
try {
User record = new User();
record.id = fieldSetFlags()[0] ? this.id : (java.lang.CharSequence) defaultValue(fields()[0]);
record.name = fieldSetFlags()[1] ? this.name : (java.lang.CharSequence) defaultValue(fields()[1]);
record.password = fieldSetFlags()[2] ? this.password : (java.lang.CharSequence) defaultValue(fields()[2]);
return record;
} catch (java.lang.Exception e) {
throw new org.apache.avro.AvroRuntimeException(e);
}
}
}
@SuppressWarnings("unchecked")
private static final org.apache.avro.io.DatumWriter<User>
WRITER$ = (org.apache.avro.io.DatumWriter<User>)MODEL$.createDatumWriter(SCHEMA$);
@Override public void writeExternal(java.io.ObjectOutput out)
throws java.io.IOException {
WRITER$.write(this, SpecificData.getEncoder(out));
}
@SuppressWarnings("unchecked")
private static final org.apache.avro.io.DatumReader<User>
READER$ = (org.apache.avro.io.DatumReader<User>)MODEL$.createDatumReader(SCHEMA$);
@Override public void readExternal(java.io.ObjectInput in)
throws java.io.IOException {
READER$.read(this, SpecificData.getDecoder(in));
}
}
接着使用IDL语言定义IPC接口协议及远程方法，IDL接口文件"helloprotocol.avdl"定义了名为“HelloProtocol ”的协议，协议接口位于com.xiaofen.protocol包下，接口名为HelloProtocol ，并在其内部定义了位于包com.xiaofen.protocol下的Message对象，并引用了外部协议文件中定义的User作为hello远程方法的参数，hello方法是接口中申明的唯一远程方法。
@namespace("com.xiaofen.protocol")
protocol HelloProtocol {
import schema "user.avsc";
record Message {
string id;
string msg;
}
string hello(com.xiaofen.record.User user);
}
生成后的代码如下:
/**
* Autogenerated by Avro
*
* DO NOT EDIT DIRECTLY
*/
package com.xiaofen.protocol;
@SuppressWarnings("all")
@org.apache.avro.specific.AvroGenerated
public interface HelloProtocol {
public static final org.apache.avro.Protocol PROTOCOL = org.apache.avro.Protocol.parse("{\"protocol\":\"HelloProtocol\",\"namespace\":\"com.xiaofen.protocol\",\"types\":[{\"type\":\"record\",\"name\":\"User\",\"namespace\":\"com.xiaofen.record\",\"fields\":[{\"name\":\"id\",\"type\":\"string\"},{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"password\",\"type\":\"string\"}]},{\"type\":\"record\",\"name\":\"Message\",\"fields\":[{\"name\":\"id\",\"type\":\"string\"},{\"name\":\"msg\",\"type\":\"string\"}]}],\"messages\":{\"hello\":{\"request\":[{\"name\":\"user\",\"type\":\"com.xiaofen.record.User\"}],\"response\":\"string\"}}}");
/**
*/
java.lang.CharSequence hello(com.xiaofen.record.User user) throws org.apache.avro.AvroRemoteException;
@SuppressWarnings("all")
public interface Callback extends HelloProtocol {
public static final org.apache.avro.Protocol PROTOCOL = com.xiaofen.protocol.HelloProtocol.PROTOCOL;
/**
* @throws java.io.IOException The async call could not be completed.
*/
void hello(com.xiaofen.record.User user, org.apache.avro.ipc.Callback<java.lang.CharSequence> callback) throws java.io.IOException;
}
}
以上步骤定义了Server端和Client端实现通信协议及相关的序列化对象。
Avro RPC服务端实现
RPC服务需要实现协议接口,HelloService 实现了HelloProtocol 协议并实现服务端代码。
package com.xiaofen.service;
import org.apache.avro.AvroRemoteException;
import com.xiaofen.protocol.HelloProtocol;
import com.xiaofen.record.User;
public class HelloService implements HelloProtocol {
@Override
public CharSequence hello(User user) throws AvroRemoteException {
System.out.println(user.getId() + "," + user.getName() + "," + user.getPassword());
return "hai i`m from avro service";
}
}
上述代码实现了Server端的业务代码，服务的发布需要使用RPC Server 进行发布,Avro提供了多种协议的Server端实现
NettyServer实现了基于Netty的 Avro RPC服务端对象，以下代码在本机IP 65111端口上发布RPC服务。
package com.xiaofen;
import java.io.IOException;
import java.net.InetSocketAddress;
import org.apache.avro.ipc.NettyServer;
import org.apache.avro.ipc.Responder;
import org.apache.avro.ipc.Server;
import org.apache.avro.ipc.specific.SpecificResponder;
import com.xiaofen.protocol.HelloProtocol;
import com.xiaofen.service.HelloService;
public class App {
@SuppressWarnings("unused")
private static Server server;
public static void main(String[] args) throws IOException {
//服务端协议
Responder responder=new SpecificResponder(HelloProtocol.class, new HelloService());
server=new NettyServer(responder, new InetSocketAddress(65111));
}
}
Avro Client端首先需要获取Server端在Client端的代理对象,客户端通过代理对象与远程对象进行方法调用
import java.io.IOException;
import java.net.InetSocketAddress;
import org.apache.avro.ipc.NettyTransceiver;
import org.apache.avro.ipc.specific.SpecificRequestor;
import com.xiaofen.protocol.HelloProtocol;
import com.xiaofen.record.User;
public class App {
public static void main(String[] args) throws IOException {
NettyTransceiver client = new NettyTransceiver(new InetSocketAddress(65111));
HelloProtocol proxy = SpecificRequestor.getClient(HelloProtocol.class, client);
User user = new User();
user.setId("1");
user.setName("xiaofen");
user.setPassword("root");
System.out.println(proxy.hello(user));
}
}