一：简介

hbase-client是HBase提供的一套比较底层的API，在实际使用时需要对其进行封装，提供更好用的api给用户。

操作hbase的客户端有以下几种方式：

hbase-client 比较底层，需要自己进一步封装api，而且版本号和安装的hbase也要匹配，否则会报错
spring-data-hadoop 2019年4月5停止维护
Apache Phoenix 使用SQL的方式来操作HBase

二：API

package org.apache.hadoop.hbase.client;

public class ConnectionFactory {
	/*
	* 创建连接
	* @param conf hbase属性配置
	*/
	public static Connection createConnection(Configuration conf) throws IOException;
}

package org.apache.hadoop.hbase

public class HBaseConfiguration extends Configuration {
	public static Configuration create();
	
	/**
	* 设置hbase的属性配置
	* @param name property name.
    * @param value property value.
	*/
	public void set(String name, String value);
}

public interface Connection extends Abortable, Closeable {
	Admin getAdmin() throws IOException;
	Table getTable(TableName tableName) throws IOException;
	void close() throws IOException;
}

public final class TableName implements Comparable<TableName> {
	public static TableName valueOf(String name);
}

public final class TableName {
	public static TableName valueOf(String name);
	default Result get(Get get) throws IOException;
	default void close() throws IOException
}

public interface Table extends Closeable {
	default void delete(Delete delete) throws IOException;
	default void put(Put put) throws IOException;
	default ResultScanner getScanner(Scan scan) throws IOException;
	default Result get(Get get) throws IOException;
}

public interface ResultScanner extends Closeable, Iterable<Result> {
	void close();
}

public class TableDescriptorBuilder {
	public static TableDescriptorBuilder newBuilder(final TableName name);

	public TableDescriptor build();
	
	public TableDescriptorBuilder setColumnFamilies(Collection<ColumnFamilyDescriptor> families);
}

public class ColumnFamilyDescriptorBuilder {
	public static ColumnFamilyDescriptorBuilder newBuilder(final byte[] name);
	public ColumnFamilyDescriptor build();
}

public interface ColumnFamilyDescriptor {

}

public class Get extends Query implements Row {
	public Get(byte [] row);
	
	public Get addColumn(byte [] family, byte [] qualifier);
}

public class Put extends Mutation implements HeapSize {
	public Put(byte [] row);

	public Put addColumn(byte [] family, byte [] qualifier, byte [] value);
}

public class Delete extends Mutation {
	public Delete addFamily(final byte [] family);
	public Delete addColumn(final byte [] family, final byte [] qualifier);
}

public class Scan extends Query {
	public Scan setFilter(Filter filter);
}

public class RowFilter extends CompareFilter {
	public RowFilter(final CompareOperator op, final ByteArrayComparable rowComparator);
}

public class Result implements CellScannable, CellScanner {
	public List<Cell> listCells();
	
	byte[] getValueArray();
	int getValueOffset();
	int getValueLength();
}

public interface Admin extends Abortable, Closeable {
	boolean tableExists(TableName tableName) throws IOException;
	void createTable(TableDescriptor desc) throws IOException;
	
	void disableTable(TableName tableName) throws IOException;
	void deleteTable(TableName tableName) throws IOException;
}

public class Result implements CellScannable, CellScanner {
	public Cell[] rawCells();
	public NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>> getMap();
}

public interface Cell {
	byte[] getRowArray();
	byte[] getFamilyArray();
	byte[] getQualifierArray();
	byte[] getValueArray();
	int getValueOffset();
	int getValueLength();
}

public class Bytes implements Comparable<Bytes> {
	// This method will convert utf8 encoded bytes into a string
	public static String toString(final byte[] b, int off, int len);
}

三：简单示例

hbase-client 使用步骤可分为五步：

创建连接Connection
从连接中获取表Table
操作表
关闭表
关闭连接

1. pom.xml

<dependency>
    <groupId>org.apache.hbase</groupId>
    <artifactId>hbase-client</artifactId>
    <version>2.1.3</version>
</dependency>

2. application.yml

hbase:
  config:
    hbase.zookeeper.quorum: 127.0.0.1
    hbase.zookeeper.port: 2181
    hbase.zookeeper.znode: /hbase
    hbase.client.keyvalue.maxsize: 1572864000

3. HbaseProperties

import org.springframework.boot.context.properties.ConfigurationProperties;
import java.util.Map;

@ConfigurationProperties(prefix = "hbase")
public class HbaseProperties {
    private Map<String, String> config;
    public Map<String, String> getConfig() {
        return config;
    }
    public void setConfig(Map<String, String> config) {
        this.config = config;
    }
}

4. HbaseConfig

@Configuration
@EnableConfigurationProperties(HbaseProperties.class)
public class HbaseConfig {
    private final HbaseProperties properties;

    public HbaseConfig(HbaseProperties properties) {
        this.properties = properties;
    }

    public org.apache.hadoop.conf.Configuration configuration() {
        org.apache.hadoop.conf.Configuration configuration = HBaseConfiguration.create();
        Map<String, String> config = properties.getConfig();
        Set<String> keySet = config.keySet();
        for (String key : keySet) {
            configuration.set(key, config.get(key));
        }
        return configuration;
    }
}

5. HBaseClient

这里只是简单的demo实现，如要实际使用还需要对其完善封装。

@DependsOn("hbaseConfig")
@Component
public class HBaseClient {

    @Autowired
    private HbaseConfig config;

    private static Connection connection = null;
    private static Admin admin = null;
    private Logger logger = LoggerFactory.getLogger(this.getClass());

    @PostConstruct
    private void init() {
        if (connection != null) {
            return;
        }

        try {
            connection = ConnectionFactory.createConnection(config.configuration());
            admin = connection.getAdmin();
        } catch (IOException e) {
            logger.error("HBase create connection failed: {}", e);
        }
    }

    public void createTable(String tableName, String[] columnFamilies) throws IOException {
        TableName name = TableName.valueOf(tableName);

        boolean isExists = this.tableExists(tableName);
        if (isExists) {
            throw new TableExistsException(tableName + "is exists!");
        }

        TableDescriptorBuilder descriptorBuilder = TableDescriptorBuilder.newBuilder(name);
        List<ColumnFamilyDescriptor> columnFamilyList = new ArrayList<>();
        for (String columnFamily : columnFamilies) {
            ColumnFamilyDescriptor columnFamilyDescriptor = ColumnFamilyDescriptorBuilder.newBuilder(columnFamily.getBytes()).build();
            columnFamilyList.add(columnFamilyDescriptor);
        }
        descriptorBuilder.setColumnFamilies(columnFamilyList);
        TableDescriptor tableDescriptor = descriptorBuilder.build();
        admin.createTable(tableDescriptor);
    }

    public void insertOrUpdate(String tableName, String rowKey, String columnFamily, String column, String value) throws IOException {
        this.insertOrUpdate(tableName, rowKey, columnFamily, new String[]{column}, new String[]{value});
    }

    public void insertOrUpdate(String tableName, String rowKey, String columnFamily, String[] columns, String[] values) throws IOException {
        Table table = connection.getTable(TableName.valueOf(tableName));
        Put put = new Put(Bytes.toBytes(rowKey));
        for (int i = 0; i < columns.length; i++) {
            put.addColumn(Bytes.toBytes(columnFamily), Bytes.toBytes(columns[i]), Bytes.toBytes(values[i]));
            table.put(put);
        }
    }

    public void deleteRow(String tableName, String rowKey) throws IOException {
        Table table = connection.getTable(TableName.valueOf(tableName));
        Delete delete = new Delete(rowKey.getBytes());
        table.delete(delete);
    }

    public void deleteColumnFamily(String tableName, String rowKey, String columnFamily) throws IOException {
        Table table = connection.getTable(TableName.valueOf(tableName));
        Delete delete = new Delete(rowKey.getBytes());
        delete.addFamily(Bytes.toBytes(columnFamily));
        table.delete(delete);
    }

    public void deleteColumn(String tableName, String rowKey, String columnFamily, String column) throws IOException {
        Table table = connection.getTable(TableName.valueOf(tableName));
        Delete delete = new Delete(rowKey.getBytes());
        delete.addColumn(Bytes.toBytes(columnFamily), Bytes.toBytes(column));
        table.delete(delete);
    }

    public void deleteTable(String tableName) throws IOException {
        boolean isExists = this.tableExists(tableName);
        if (!isExists) {
            return;
        }

        TableName name = TableName.valueOf(tableName);
        admin.disableTable(name);
        admin.deleteTable(name);
    }

    public String getValue(String tableName, String rowkey, String family, String column) {
        Table table = null;

        String value = "";
        if (StringUtils.isBlank(tableName) || StringUtils.isBlank(family)
                || StringUtils.isBlank(rowkey) || StringUtils.isBlank(column)) {
            return null;
        }

        try {
            table = connection.getTable(TableName.valueOf(tableName));
            Get g = new Get(rowkey.getBytes());
            g.addColumn(family.getBytes(), column.getBytes());
            Result result = table.get(g);
            List<Cell> ceList = result.listCells();
            if (ceList != null && ceList.size() > 0) {
                for (Cell cell : ceList) {
                    value = Bytes.toString(cell.getValueArray(), cell.getValueOffset(), cell.getValueLength());
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }finally {
            try {
                table.close();
                connection.close();
            }catch (IOException e) {
                e.printStackTrace();
            }
        }
        return value;
    }

    public String selectOneRow(String tableName, String rowKey) throws IOException {
        Table table = connection.getTable(TableName.valueOf(tableName));
        Get get = new Get(rowKey.getBytes());
        Result result = table.get(get);
        NavigableMap<byte[], NavigableMap<byte[], NavigableMap<Long, byte[]>>> map = result.getMap();
        for (Cell cell : result.rawCells()) {
            String row = Bytes.toString(cell.getRowArray());
            String columnFamily = Bytes.toString(cell.getFamilyArray());
            String column = Bytes.toString(cell.getQualifierArray());
            String value = Bytes.toString(cell.getValueArray());

            // 可以通过反射封装成对象(列名和Java属性保持一致)
            System.out.println(row);
            System.out.println(columnFamily);
            System.out.println(column);
            System.out.println(value);
        }

        return null;
    }

    public String scanTable(String tableName, String rowKeyFilter) throws IOException {
        Table table = connection.getTable(TableName.valueOf(tableName));
        Scan scan = new Scan();
        if (!StringUtils.isEmpty(rowKeyFilter)) {
            RowFilter rowFilter = new RowFilter(CompareOperator.EQUAL, new SubstringComparator(rowKeyFilter));
            scan.setFilter(rowFilter);
        }
        ResultScanner scanner = table.getScanner(scan);

        try {
            for (Result result : scanner) {
                System.out.println(Bytes.toString(result.getRow()));
                for (Cell cell : result.rawCells()) {
                    System.out.println(cell);
                }
            }
        } finally {
            if (scanner != null) {
                scanner.close();
            }
        }

        return null;
    }

    
    /**
     * 判断表是否已经存在，这里使用间接的方式来实现
     *
     * admin.tableExists() 会报NoSuchColumnFamilyException， 有人说是hbase-client版本问题
     * @param tableName
     * @return
     * @throws IOException
     */
    public boolean tableExists(String tableName) throws IOException {
        TableName[] tableNames = admin.listTableNames();
        if (tableNames != null && tableNames.length > 0) {
            for (int i = 0; i < tableNames.length; i++) {
                if (tableName.equals(tableNames[i].getNameAsString())) {
                    return true;
                }
            }
        }
        return false;
    }
}

6. Test


import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import java.io.IOException;

@RunWith(SpringRunner.class)
@SpringBootTest
public class HbaseDemoApplicationTests {

    @Autowired
    private HBaseClient hBaseClient;

    /**
     * 测试删除、创建表
     */
    @Test
    public void testGetValue() {
        String value = hBaseClient.getValue("tbl_user", "mengday", "info", "age");
        System.out.println(value);
    }

    @Test
    public void testCreateTable() throws IOException {
        String tableName = "tbl_abc";
        hBaseClient.deleteTable(tableName);
        hBaseClient.createTable(tableName, new String[] {"cf1", "cf2"});
    }

    @Test
    public void dropTable() throws IOException {
        hBaseClient.deleteTable("tbl_abc");
    }


    @Test
    public void testInsertOrUpdate() throws IOException {
        hBaseClient.insertOrUpdate("tbl_abc", "rowKey1", "cf1", new String[]{"c1", "c2"}, new String[]{"v1", "v22"});
    }

    @Test
    public void testSelectOneRow() throws IOException {
        hBaseClient.selectOneRow("tbl_abc", "rowKey1");
    }

    @Test
    public void testScanTable() throws IOException {
        hBaseClient.scanTable("tbl_abc", "rowKey1");
    }
}

vbirdbest 博客专家

发布了308 篇原创文章 · 获赞 936 · 访问量 133万+

他的留言板关注

HBase Java API: hbase-client

一：简介

二：API

三：简单示例

1. pom.xml

2. application.yml

3. HbaseProperties

4. HbaseConfig

5. HBaseClient

6. Test

猜你喜欢