avro序列化实例

hadoop中，就是用的avro作为序列化的，现在按照官方文档，我们按照如下步骤来做这个实例。

构建maven工程，加入apache-avro依赖库，以及插件依赖。
编写avro文件
生成java实体类
编写测试程序

pom.xml配置文件

<dependencies>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>4.9</version>
      <scope>test</scope>
    </dependency>
    <dependency>
      <groupId>org.apache.avro</groupId>
      <artifactId>avro</artifactId>
      <version>1.8.2</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-log4j12</artifactId>
      <version>1.7.21</version>
    </dependency>
</dependencies>

<build>
    <plugins>
        <plugin>   
            <groupId>org.apache.avro</groupId>
            <artifactId>avro-maven-plugin</artifactId>
            <version>1.8.2</version>
            <executions>
            <execution>
            <phase>generate-sources</phase>
            <goals>
                <goal>schema</goal>
            </goals>
            <configuration>	
                <sourceDirectory>${project.basedir}/src/main/avro/</sourceDirectory>   
                <outputDirectory>${project.basedir}/src/main/java/</outputDirectory>
            </configuration>
            </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
    </plugins>
</build>

src/main/avro目录下编写user.avsc,注意后缀是avsc

{
  "namespace":"com.xxx.avro.entity",
  "type":"record",
  "name":"User",
  "fields":[
        {"name":"id","type":"int"},
        {"name":"name","type":"string"},
        {"name":"age","type":["int","null"]},
        {"name":"mobile","type":["string","null"]}
  ]
}

这是一个用json格式表示的avro对象

因为我们的pom.xml配置文件中配置了avro生成java的插件，所以这里直接运行Run As ->Maven install，构建成功之后，会在com.xxx.avro.entity目录下生成User.java的文件。

User.java文件部分内容截图：

接下来，编写测试验证：

运行结果：

eclipse下，使用avro-maven-plugin插件，工程中的pom.xml会报错： Plugin execution not covered by lifecycle configuration...这个错误提示上有解决办法，要么在pom.xml配置文件中增加配置，忽略phase，要么在lifecycle-mapping-metadata.xml配置文件中增加配置忽略phase，然后重新加载lifecycle-mapping-metadata.xml文件。这里两种解决办法都贴出来。

1、在pom.xml中增加配置：

<pluginManagement>
   <plugins>
      <plugin>
         <groupId>org.eclipse.m2e</groupId>
         <artifactId>lifecycle-mapping</artifactId>
         <version>1.0.0</version>
         <configuration>
            <lifecycleMappingMetadata>
            <pluginExecutions>
               <pluginExecution>
                  <pluginExecutionFilter>
                  <groupId>org.apache.avro</groupId>
                  <artifactId>avro-maven-plugin</artifactId>
                  <versionRange>[1.8.2,)</versionRange>
                  <goals>
                  	<goal>schema</goal>
                  </goals>
                  </pluginExecutionFilter>
                  <action>
                  	<ignore></ignore>
                  </action>
               </pluginExecution>
            </pluginExecutions>
            </lifecycleMappingMetadata>
         </configuration>
      </plugin>
   </plugins>
</pluginManagement>

2、在lifecycle-mapping-metadata.xml中增加配置：

<lifecycleMappingMetadata>
  <pluginExecutions>
    <pluginExecution>
      <pluginExecutionFilter>
        <groupId>org.apache.avro</groupId>
        <artifactId>avro-maven-plugin</artifactId>
        <versionRange>1.8.2</versionRange>
        <goals>
          <goal>schema</goal>
        </goals>
      </pluginExecutionFilter>
      <action>
        <ignore />
      </action>
    </pluginExecution>
  </pluginExecutions>
</lifecycleMappingMetadata>

猜你喜欢