HBase + Key-Value Store Indexer + Solr

准备

• hbase 启动并开启复制配置
• Solr 启动
• Key-Value Store Indexer 启动
• Lily HBase Indexer 的 Java 堆栈大小（字节）调整

1.solr 创建collection

MIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX

2.solr 添加字段

3.solr 设置自动提交时间

发送POST请求

curl http://n1:8983/solr/MIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX/config 
-H 'Content-type:application/json' -d 
'{"set-property":{"updateHandler.autoSoftCommit.maxTime":"3000"}}'

检索配置 http://n1:8983/solr/MIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX/config

4.hbase 建表

[root@n1 ~]# hbase shell
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
Version 2.0.0-cdh6.0.1, rUnknown, Wed Sep 19 09:14:00 PDT 2018
Took 0.0032 seconds               
hbase(main):003:0> create 'MIDDLE_PLATFORM:MP_HBASE_SOLR_INDEX',{
    
    NAME => 'info', REPLICATION_SCOPE => 1}
Created table MIDDLE_PLATFORM:MP_HBASE_SOLR_INDEX
Took 1.8058 seconds                                                                                                                                                                                               
=> Hbase::Table - MIDDLE_PLATFORM:MP_HBASE_SOLR_INDEX

5.Key-Value Store Indexer 设置 Morphlines 文件

SOLR_LOCATOR : {
  # Name of solr collection
  collection : MIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX
  
  # ZooKeeper ensemble
  zkHost : "$ZK_HOST" 
}
morphlines : [
{
id : MP_HBASE_SOLR_INDEX
importCommands : ["org.kitesdk.**", "com.ngdata.**"]

commands : [                    
  {
    extractHBaseCells {
      mappings : [
        {
          inputColumn : "info:firstname"
          outputField : "info_firstname" 
          type : string 
          source : value
        }
      ]
    }
  }
  { logDebug { format : "output record: {}", args : ["@{}"] } }
]
}
]

6.准备 morphline-hbase-mapper.xml

<?xml version="1.0" encoding="UTF-8"?>
<indexer table="MIDDLE_PLATFORM:MP_HBASE_SOLR_INDEX" mapper="com.ngdata.hbaseindexer.morphline.MorphlineResultToSolrMapper">
	<param name="morphlineFile" value="morphlines.conf"></param>
	<param name="morphlineId" value="MP_HBASE_SOLR_INDEX"></param>
</indexer>

上传到

$HOME/MIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX/morphline-hbase-mapper.xml

7.添加索引

hbase-indexer add-indexer \
--name MIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX \
--indexer-conf $HOME/MIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX/morphline-hbase-mapper.xml \
--connection-param solr.zk=n1,n2,n3:2181/solr \
--connection-param solr.collection=MIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX \
--zookeeper n1,n2,n3:2181

删除索引
hbase-indexer delete-indexer --name ‘MIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX’ --zookeeper n1,n2,n3:2181

查看所有索引清单
hbase-indexer list-indexers -dump --zookeeper n1,n2,n3:2181

8.HBase 添加数据测试

put 'MIDDLE_PLATFORM:MP_HBASE_SOLR_INDEX','1','info:firstname','GENG'

9.查询Solr

http://n1:8983/solr/MIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX/select?q=*:*