proto协议
EC
public final class ECSchema implements Serializable {
private static final long serialVersionUID = 0x10953aa0;
public static final String NUM_DATA_UNITS_KEY = "numDataUnits";
public static final String NUM_PARITY_UNITS_KEY = "numParityUnits";
public static final String CODEC_NAME_KEY = "codec";
/**
* The erasure codec name associated.
*/
private final String codecName;
/**
* Number of source data units coded
*/
private final int numDataUnits;
/**
* Number of parity units generated in a coding
*/
private final int numParityUnits;
/*
* An erasure code can have its own specific advanced parameters, subject to
* itself to interpret these key-value settings.
*/
private final Map<String, String> extraOptions;
...
}
public final class ErasureCodingPolicy implements Serializable {
private static final long serialVersionUID = 0x0079fe4e;
private final String name;
private final ECSchema schema;
private final int cellSize;
private final byte id;
...
}
public class ErasureCodingPolicyInfo implements Serializable {
private static final long serialVersionUID = 0x31;
private final ErasureCodingPolicy policy;
private ErasureCodingPolicyState state;
...
}
所以关系
ErasureCodingPolicyInfo -> ErasureCodingPolicy -> ECSchema
那么在定义Proto的时候怎么体现的呢？
ErasureCodingPolicyProto
enum ErasureCodingPolicyState {
DISABLED = 1;
ENABLED = 2;
REMOVED = 3;
}
message ErasureCodingPolicyProto {
optional string name = 1;
optional ECSchemaProto schema = 2;
optional uint32 cellSize = 3;
required uint32 id = 4; // Actually a byte - only 8 bits used
optional ErasureCodingPolicyState state = 5 [default = ENABLED];
}
ECSchemaProto
message ECSchemaOptionEntryProto {
required string key = 1;
required string value = 2;
}
/**
* ECSchema for erasurecoding
*/
message ECSchemaProto {
required string codecName = 1;
required uint32 dataUnits = 2;
required uint32 parityUnits = 3;
repeated ECSchemaOptionEntryProto options = 4;
}
erasurecoding.proto
message GetErasureCodingPoliciesResponseProto {
repeated ErasureCodingPolicyProto ecPolicies = 1;
}
说明：
上述java代码中定义了3种示例即：
ErasureCodingPolicyInfo -> ErasureCodingPolicy -> ECSchema
但是在proto中只定义了
ErasureCodingPolicyProto -> ECSchemaProto，其中ErasureCodingPolicyProto并不是和ErasureCodingPolicy字段一样。所以proto和java类并完全要求字段一致。
在PB类中，进行了GetErasureCodingPoliciesResponseProto和ErasureCodingPolicyInfo[]的转换。
先看server端：
//ClientNamenodeProtocolServerSideTranslatorPB
@Override
public GetErasureCodingPoliciesResponseProto getErasureCodingPolicies(RpcController controller,
GetErasureCodingPoliciesRequestProto request) throws ServiceException {
try {
//调用代码得到的是info数组
ErasureCodingPolicyInfo[] ecpInfos = server.getErasureCodingPolicies();
GetErasureCodingPoliciesResponseProto.Builder resBuilder = GetErasureCodingPoliciesResponseProto
.newBuilder();
for (ErasureCodingPolicyInfo info : ecpInfos) {
resBuilder.addEcPolicies(
//对于每一个info都转化成proto
PBHelperClient.convertErasureCodingPolicy(info));
}
return resBuilder.build();
} catch (IOException e) {
throw new ServiceException(e);
}
}
server调用代码得到的是：
ErasureCodingPolicyInfo[] ecpInfos = server.getErasureCodingPolicies();
ecpInfos数组需要转成proto类型的才能传输。
GetErasureCodingPoliciesResponseProto就是一个repeated类型，所以循环加就行。
转化：（转化是在PBHelperClient中执行的）
扫描二维码关注公众号，回复：
12405337 查看本文章
public static ErasureCodingPolicyProto convertErasureCodingPolicy(
ErasureCodingPolicyInfo info) {
final ErasureCodingPolicyProto.Builder builder =
createECPolicyProtoBuilder(info.getPolicy());
builder.setState(convertECState(info.getState()));
return builder.build();
}
private static ErasureCodingPolicyProto.Builder createECPolicyProtoBuilder(
ErasureCodingPolicy policy) {
final ErasureCodingPolicyProto.Builder builder =
ErasureCodingPolicyProto.newBuilder().setId(policy.getId());
// If it's not a built-in policy, need to set the optional fields.
if (SystemErasureCodingPolicies.getByID(policy.getId()) == null) {
builder.setName(policy.getName())
.setSchema(convertECSchema(policy.getSchema()))
.setCellSize(policy.getCellSize());
}
return builder;
}
上述将java类转成proto类型，其原理是取出java类的字段（基础类型的字段），一个个的set到proto中；对于非基础类型的字段，需要一直转化到底，如：
.setSchema(convertECSchema(policy.getSchema()))
再次进入convertECSchema方法中
public static HdfsProtos.ECSchemaProto convertECSchema(ECSchema schema) {
HdfsProtos.ECSchemaProto.Builder builder =
HdfsProtos.ECSchemaProto.newBuilder()
.setCodecName(schema.getCodecName())
.setDataUnits(schema.getNumDataUnits())
.setParityUnits(schema.getNumParityUnits());
Set<Map.Entry<String, String>> entrySet =
schema.getExtraOptions().entrySet();
for (Map.Entry<String, String> entry : entrySet) {
builder.addOptions(HdfsProtos.ECSchemaOptionEntryProto.newBuilder()
.setKey(entry.getKey()).setValue(entry.getValue()).build());
}
return builder.build();
}
值得注意的是，这里有map类型转成proto的方法。
public static HdfsProtos.ECSchemaProto convertECSchema(ECSchema schema) {
HdfsProtos.ECSchemaProto.Builder builder =
HdfsProtos.ECSchemaProto.newBuilder()
.setCodecName(schema.getCodecName())
.setDataUnits(schema.getNumDataUnits())
.setParityUnits(schema.getNumParityUnits());
Set<Map.Entry<String, String>> entrySet =
schema.getExtraOptions().entrySet();
for (Map.Entry<String, String> entry : entrySet) {
builder.addOptions(HdfsProtos.ECSchemaOptionEntryProto.newBuilder()
.setKey(entry.getKey()).setValue(entry.getValue()).build());
}
return builder.build();
}
再看客户端
客户端就是反过程，将proto转成java类
参考ClientNamenodeProtocolTranslatorPB.java#getErasureCodingPolicies，不再赘述。