Phoenix中常用shell操作
1、客户端登录
[root@dn3 ~]# cd /opt/module/phoenix && /usr/bin/python2 bin/sqlline.py dn3,dn4,dn5:2181
Setting property: [incremental, false]
Setting property: [isolation, TRANSACTION_READ_COMMITTED]
issuing: !connect jdbc:phoenix:dn3,dn4,dn5:2181 none none org.apache.phoenix.jdbc.PhoenixDriver
Connecting to jdbc:phoenix:dn3,dn4,dn5:2181
22/07/25 18:40:48 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Connected to: Phoenix (version 5.0)
Driver: PhoenixEmbeddedDriver (version 5.0)
Autocommit status: true
Transaction isolation: TRANSACTION_READ_COMMITTED
Building list of tables and columns for tab-completion (set fastconnect to true to skip)...
225/225 (100%) Done
Done
sqlline version 1.2.0
0: jdbc:phoenix:dn3,dn4,dn5:2181>
2、客户端退出
可以使用如下2种命令:
- !quit
- !exit
0: jdbc:phoenix:dn3,dn4,dn5:2181> !quit
Closing: org.apache.phoenix.jdbc.PhoenixConnection
[root@dn3 ~]#
0: jdbc:phoenix:dn3,dn4,dn5:2181> !exit
Closing: org.apache.phoenix.jdbc.PhoenixConnection
[root@dn3 ~]#
3、查询全部表的list
可以使用如下2种命令:
- !tables
- !table
0: jdbc:phoenix:dn3,dn4,dn5:2181> !tables
+------------+-----------------+-------------------------+-+
| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | |
+------------+-----------------+-------------------------+-+
| | SYSTEM | CATALOG | |
| | SYSTEM | FUNCTION | |
| | SYSTEM | LOG | |
| | SYSTEM | SEQUENCE | |
| | SYSTEM | STATS | |
| | | hbase_test | |
| | GMALL_REALTIME | DIM_ACTIVITY_INFO | |
| | GMALL_REALTIME | DIM_ACTIVITY_RULE | |
| | GMALL_REALTIME | DIM_ACTIVITY_SKU | |
| | GMALL_REALTIME | DIM_BASE_CATEGORY1 | |
| | GMALL_REALTIME | DIM_BASE_CATEGORY2 | |
| | GMALL_REALTIME | DIM_BASE_CATEGORY3 | |
| | GMALL_REALTIME | DIM_BASE_PROVINCE | |
| | GMALL_REALTIME | DIM_BASE_REGION | |
| | GMALL_REALTIME | DIM_BASE_TRADEMARK | |
| | GMALL_REALTIME | DIM_COUPON_INFO | |
| | GMALL_REALTIME | DIM_COUPON_RANGE | |
| | GMALL_REALTIME | DIM_FINANCIAL_SKU_COST | |
| | GMALL_REALTIME | DIM_SKU_INFO | |
| | GMALL_REALTIME | DIM_SPU_INFO | |
| | GMALL_REALTIME | DIM_USER_INFO | |
+------------+-----------------+-------------------------+-+
说明: Phoenix中的表信息都在SYSTEM.CATALOG表中, 亦可通过如下sql查看系统的表信息:
select * from SYSTEM.CATALOG;
说明: 上述命令都是以"
!" 开头, 下面的命令均需要以";" 结尾.describe也需要以!开头.
4、创建表
（1）表名和列族名以及表名如果需要小写都得加双引号。
create table "person" ("id" integer not null primary key, "cf"."name" varchar, "cf"."age" integer);
（2）这样创建出来的列族名称和列名都是大写的。
create table "person" (id integer not null primary key, cf.name varchar, cf.age integer);
注意：表名和列族名区分大小写，加了双引号就必须用小写表名去查否则查不到。列名不区分大小写，显示是大写，用小写依然查得到。
示例:
0: jdbc:phoenix:dn3,dn4,dn5:2181> create table if not exists "person" ("id" integer not null primary key, "cf"."name" varchar, "cf"."age" integer);
No rows affected (2.389 seconds)
0: jdbc:phoenix:dn3,dn4,dn5:2181> !tables
+------------+-----------------+-------------------------+-+
| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | |
+------------+-----------------+-------------------------+-+
| | SYSTEM | CATALOG | |
| | SYSTEM | FUNCTION | |
| | SYSTEM | LOG | |
| | SYSTEM | SEQUENCE | |
| | SYSTEM | STATS | |
| | | hbase_test | |
| | | person | |
+------------+-----------------+-------------------------+-+
0: jdbc:phoenix:dn3,dn4,dn5:2181> !describe "person";
+------------+--------------+-------------+--------------+-+
| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | COLUMN_NAME | |
+------------+--------------+-------------+--------------+-+
| | | person | id | |
| | | person | name | |
| | | person | age | |
+------------+--------------+-------------+--------------+-+
5、删除表
0: jdbc:phoenix:dn3,dn4,dn5:2181> drop table "person";
No rows affected (1.25 seconds)
6、修改表结构
注意如下添加的字段:
- address 因为没有加引号, 默认为大写;
- birthday 因为加了引号, 所以为小写;
0: jdbc:phoenix:dn3,dn4,dn5:2181> alter table "person" add address varchar;
No rows affected (6.629 seconds)
0: jdbc:phoenix:dn3,dn4,dn5:2181> !describe "person";
+------------+--------------+-------------+--------------+------------+------------+--------------+--------+
| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | COLUMN_NAME | DATA_TYPE | TYPE_NAME | COLUMN_SIZE | BUFFER |
+------------+--------------+-------------+--------------+------------+------------+--------------+--------+
| | | person | id | 4 | INTEGER | null | null |
| | | person | name | 12 | VARCHAR | null | null |
| | | person | age | 4 | INTEGER | null | null |
| | | person | ADDRESS | 12 | VARCHAR | null | null |
+------------+--------------+-------------+--------------+------------+------------+--------------+--------
0: jdbc:phoenix:dn3,dn4,dn5:2181> alter table "person" add "birthday" timestamp;
No rows affected (6.786 seconds)
0: jdbc:phoenix:dn3,dn4,dn5:2181> !describe "person";
+------------+--------------+-------------+--------------+------------+------------+--------------+--------+
| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | COLUMN_NAME | DATA_TYPE | TYPE_NAME | COLUMN_SIZE | BUFFER |
+------------+--------------+-------------+--------------+------------+------------+--------------+--------+
| | | person | id | 4 | INTEGER | null | null |
| | | person | name | 12 | VARCHAR | null | null |
| | | person | age | 4 | INTEGER | null | null |
| | | person | ADDRESS | 12 | VARCHAR | null | null |
| | | person | birthday | 93 | TIMESTAMP | null | null |
+------------+--------------+-------------+--------------+------------+------------+--------------+--------
7、查看表结构(详见6)
[root@dn3 ~]# !describe "person";
8、为表的某一列创建索引
注意,建索引前,需要先在RegionServer上的 hbase-site.xml 文件中添加如下配置代码:
否则会报错:
Error: ERROR 1029 (42Y88): Mutable secondary indexes must have the hbase.regionserver.wal.codec property set to org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec in the hbase-sites.xml of every region server. tableName=index_device_data_test08 (state=42Y88,code=1029)
两个双引号，第一个是索引的名称，第二个是原表的名称
这边的索引字段为：deviceID
Include括号里面是包含要返回的列是哪些
0: jdbc:phoenix:dn3,dn4,dn5:2181> create index "person_index" on "person"("cf"."name") include ("cf"."name","cf"."age");
9、删除索引
0: jdbc:phoenix:dn3,dn4,dn5:2181> drop index "person_index" on "person";
10、插入一条记录
注意：数据值需要用引号时只能用单引号，双引号会报错
0: jdbc:phoenix:dn3,dn4,dn5:2181> upsert into "person" values(1,'david', 20, 'ShangHai', '2002-01-20 00:00:00');
1 row affected (0.071 seconds)
0: jdbc:phoenix:dn3,dn4,dn5:2181> select * from "person";
+-----+-----------+------+-----------+--------------------------+
| id | name | age | ADDRESS | birthday |
+-----+-----------+------+-----------+--------------------------+
| 1 | david | 20 | ShangHai | 2002-01-20 00:00:00.000 |
+-----+-----------+------+-----------+--------------------------+
1 row selected (0.037 seconds)
11、删除表中数据
注意：数据值需要用引号时只能用单引号，双引号会报错
0: jdbc:phoenix:dn3,dn4,dn5:2181> delete from "person" where "cf"."name"='david';
1 row affected (0.031 seconds)
0: jdbc:phoenix:dn3,dn4,dn5:2181> select * from "person";
+-----+-------+------+----------+-----------+
| id | name | age | ADDRESS | birthday |
+-----+-------+------+----------+-----------+
+-----+-------+------+----------+-----------+
No rows selected (0.036 seconds)
12、修改表中数据
注意：修改时必须带上id，否则会报错
注意：数据值需要用引号时只能用单引号，双引号会报错
0: jdbc:phoenix:dn3,dn4,dn5:2181> upsert into "person"("id",ADDRESS) values(1,'BeiJing');
1 row affected (0.013 seconds)
0: jdbc:phoenix:dn3,dn4,dn5:2181> select * from "person";
+-----+--------+------+----------+--------------------------+
| id | name | age | ADDRESS | birthday |
+-----+--------+------+----------+--------------------------+
| 1 | david | 20 | BeiJing | 2002-01-20 00:00:00.000 |
+-----+--------+------+----------+--------------------------+
1 row selected (0.026 seconds)
13、查询表中数据
注意：数据值需要用引号时只能用单引号，双引号会报错
①.全表查询
0: jdbc:phoenix:dn3,dn4,dn5:2181> select * from "person";
+-----+---------+------+------------+--------------------------+
| id | name | age | ADDRESS | birthday |
+-----+---------+------+------------+--------------------------+
| 1 | david | 20 | BeiJing | 2002-01-20 00:00:00.000 |
| 2 | amanda | 18 | ShangHai | 2004-10-13 00:00:00.000 |
| 3 | andy | 2 | GuangZhou | 2020-07-29 00:00:00.000 |
+-----+---------+------+------------+--------------------------+
②.条件查询
0: jdbc:phoenix:dn3,dn4,dn5:2181> select * from "person" where "name"='andy';
+-----+-------+------+------------+--------------------------+
| id | name | age | ADDRESS | birthday |
+-----+-------+------+------------+--------------------------+
| 3 | andy | 2 | GuangZhou | 2020-07-29 00:00:00.000 |
+-----+-------+------+------------+--------------------------+
1 row selected (0.025 seconds)
③.分组查询(group by)
0: jdbc:phoenix:dn3,dn4,dn5:2181> select ADDRESS, count(ADDRESS) from "person" where "age" > 16 group by ADDRESS;
+-----------+-----------------+
| ADDRESS | COUNT(ADDRESS) |
+-----------+-----------------+
| BeiJing | 1 |
| ShangHai | 1 |
+-----------+-----------------+
2 rows selected (0.023 seconds)
④. case when 变换查询
0: jdbc:phoenix:dn3,dn4,dn5:2181> select "name", (case "name" when 'david' then 'haha' when 'amanda' then 'xixi' else "name" end) as nicky_name from "person";
+---------+-------------+
| name | NICKY_NAME |
+---------+-------------+
| david | haha |
| amanda | xixi |
| andy | andy |
+---------+-------------+
3 rows selected (0.015 seconds)
14、关联Hbase中已存在的表
语法:
create view if not exists "device_data"(
"pk" varchar not null primary key,
"data"."deviceID" varchar,
"data"."deviceTime" varchar,
"data"."modelID" varchar,
"data"."processState" varchar,
"data"."subDevice" varchar
);
hbase数据准备:
#创建Hbase 表
create 'device_data', 'data'
#向Hbase表里插入测试数据
put 'device_data','rowKey_001', 'data:deviceID', 'D1001'
put 'device_data','rowKey_001', 'data:deviceTime', '2022-07-13 11:11:11'
put 'device_data','rowKey_001', 'data:modelID', 'M1001'
put 'device_data','rowKey_001', 'data:processState', 'Fail'
put 'device_data','rowKey_001', 'data:subDevice', 'D1001_S0001'
put 'device_data','rowKey_002', 'data:deviceID', 'D1002'
put 'device_data','rowKey_002', 'data:deviceTime', '2022-07-14 12:12:12'
put 'device_data','rowKey_002', 'data:modelID', 'M1002'
put 'device_data','rowKey_002', 'data:processState', 'Succ'
put 'device_data','rowKey_002', 'data:subDevice', 'D1002_S0002'
[root@dn3 ~]# hbase shell
2022-07-25 19:45:15,606 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.0.5, r76458dd074df17520ad451ded198cd832138e929, Mon Mar 18 00:41:49 UTC 2019
Took 0.0030 seconds
hbase(main):001:0>
hbase(main):020:0> scan 'device_data'
ROW COLUMN+CELL
rowKey_001 column=data:deviceID, timestamp=1658750104769, value=D1001
rowKey_001 column=data:deviceTime, timestamp=1658750124167, value=2022-07-13 11:11:11
rowKey_001 column=data:modelID, timestamp=1658750124195, value=M1001
rowKey_001 column=data:processState, timestamp=1658750124224, value=Fail
rowKey_001 column=data:subDevice, timestamp=1658750124255, value=D1001_S0001
rowKey_002 column=data:deviceID, timestamp=1658750124281, value=D1002
rowKey_002 column=data:deviceTime, timestamp=1658750124303, value=2022-07-14 12:12:12
rowKey_002 column=data:modelID, timestamp=1658750124330, value=M1002
rowKey_002 column=data:processState, timestamp=1658750124355, value=Succ
rowKey_002 column=data:subDevice, timestamp=1658750124929, value=D1002_S0002
2 row(s)
Took 0.0410 seconds
Phonenix 中创建Hbase 映射表:
0: jdbc:phoenix:dn3,dn4,dn5:2181> create view if not exists "device_data"(
. . . . . . . . . . . . . . . . > "pk" varchar not null primary key,
. . . . . . . . . . . . . . . . > "data"."deviceID" varchar,
. . . . . . . . . . . . . . . . > "data"."deviceTime" varchar,
. . . . . . . . . . . . . . . . > "data"."modelID" varchar,
. . . . . . . . . . . . . . . . > "data"."processState" varchar,
. . . . . . . . . . . . . . . . > "data"."subDevice" varchar
. . . . . . . . . . . . . . . . > );
注意：
（1）如果不加列族会报错如下：
Error: ERROR 505 (42000): Table is read only. (state=42000,code=505)
（2）如果不加双引号则会匹配不到hbase表中的字段，结果就是虽然关联上数据库但是没有值！！！
（3）关联的时候，Phoenix建表最好都是varchar类型，不容易出错
（4）最好创建view视图，不要创建table表格。因为Phoenix端删除table会连带删除hbase表格，如果是view则不会。
查询Phonenix 表(注意添加双引号):
0: jdbc:phoenix:dn3,dn4,dn5:2181> select * from "device_data";
+-------------+-----------+----------------------+----------+---------------+--------------+
| pk | deviceID | deviceTime | modelID | processState | subDevice |
+-------------+-----------+----------------------+----------+---------------+--------------+
| rowKey_001 | D1001 | 2022-07-13 11:11:11 | M1001 | Fail | D1001_S0001 |
| rowKey_002 | D1002 | 2022-07-14 12:12:12 | M1002 | Succ | D1002_S0002 |
+-------------+-----------+----------------------+----------+---------------+--------------+
2 rows selected (0.071 seconds)
15、help帮助手册
0: jdbc:phoenix:dn3,dn4,dn5:2181> !help
!all Execute the specified SQL against all the current
connections
!autocommit Set autocommit mode on or off
!batch Start or execute a batch of statements
!brief Set verbose mode off
!call Execute a callable statement
!close Close the current connection to the database
!closeall Close all current open connections
!columns List all the columns for the specified table
!commit Commit the current transaction (if autocommit is off)
!connect Open a new connection to the database.
!dbinfo Give metadata information about the database
!describe Describe a table
!dropall Drop all tables in the current database
!exportedkeys List all the exported keys for the specified table
!go Select the current connection
!help Print a summary of command usage
!history Display the command history
!importedkeys List all the imported keys for the specified table
!indexes List all the indexes for the specified table
!isolation Set the transaction isolation for this connection
!list List the current connections
!manual Display the SQLLine manual
!metadata Obtain metadata information
!nativesql Show the native SQL for the specified statement
!outputformat Set the output format for displaying results
(table,vertical,csv,tsv,xmlattrs,xmlelements)
!primarykeys List all the primary keys for the specified table
!procedures List all the procedures
!properties Connect to the database specified in the properties file(s)
!quit Exits the program
!reconnect Reconnect to the database
!record Record all output to the specified file
!rehash Fetch table and column names for command completion
!rollback Roll back the current transaction (if autocommit is off)
!run Run a script from the specified file
!save Save the current variabes and aliases
!scan Scan for installed JDBC drivers
!script Start saving a script to a file
!set Set a sqlline variable
Variable Value Description
=============== ========== ================================
autoCommit true/false Enable/disable automatic
transaction commit
autoSave true/false Automatically save preferences
color true/false Control whether color is used
for display
fastConnect true/false Skip building table/column list
for tab-completion
force true/false Continue running script even
after errors
headerInterval integer The interval between which
headers are displayed
historyFile path File in which to save command
history. Default is
$HOME/.sqlline/history (UNIX,
Linux, Mac OS),
$HOME/sqlline/history (Windows)
incremental true/false Do not receive all rows from
server before printing the first
row. Uses fewer resources,
especially for long-running
queries, but column widths may
be incorrect.
isolation LEVEL Set transaction isolation level
maxColumnWidth integer The maximum width to use when
displaying columns
maxHeight integer The maximum height of the
terminal
maxWidth integer The maximum width of the
terminal
numberFormat pattern Format numbers using
DecimalFormat pattern
outputFormat table/vertical/csv/tsv Format mode for
result display
propertiesFile path File from which SqlLine reads
properties on startup; default is
$HOME/.sqlline/sqlline.properties
(UNIX, Linux, Mac OS),
$HOME/sqlline/sqlline.properties
(Windows)
rowLimit integer Maximum number of rows returned
from a query; zero means no
limit
showElapsedTime true/false Display execution time when
verbose
showHeader true/false Show column names in query
results
showNestedErrs true/false Display nested errors
showWarnings true/false Display connection warnings
silent true/false Be more silent
timeout integer Query timeout in seconds; less
than zero means no timeout
trimScripts true/false Remove trailing spaces from
lines read from script files
verbose true/false Show verbose error messages and
debug info
!sql Execute a SQL command
!tables List all the tables in the database
!typeinfo Display the type map for the current connection
!verbose Set verbose mode on
Comments, bug reports, and patches go to ???
0: jdbc:phoenix:dn3,dn4,dn5:2181>
16、查询metadata信息
0: jdbc:phoenix:dn3,dn4,dn5:2181> !dbinfo
allProceduresAreCallable false
allTablesAreSelectable true
dataDefinitionCausesTransactionCommit false
dataDefinitionIgnoredInTransactions false
doesMaxRowSizeIncludeBlobs false
getCatalogSeparator .
getCatalogTerm Tenant
getDatabaseProductName Phoenix
getDatabaseProductVersion 5.0
getDefaultTransactionIsolation 2
getDriverMajorVersion 5
getDriverMinorVersion 0
getDriverName PhoenixEmbeddedDriver
getDriverVersion 5.0
getExtraNameCharacters
getIdentifierQuoteString "
getMaxBinaryLiteralLength 0
getMaxCatalogNameLength 0
getMaxCharLiteralLength 4000
getMaxColumnNameLength 200
getMaxColumnsInGroupBy 1
getMaxColumnsInIndex 0
getMaxColumnsInOrderBy 0
getMaxColumnsInSelect 0
getMaxColumnsInTable 0
getMaxConnections 0
getMaxCursorNameLength 0
getMaxIndexLength 0
getMaxProcedureNameLength 0
getMaxRowSize 0
getMaxSchemaNameLength 0
getMaxStatementLength 0
getMaxStatements 0
getMaxTableNameLength 0
getMaxTablesInSelect 1
getMaxUserNameLength 0
getNumericFunctions
getProcedureTerm procedure
getSchemaTerm schema
getSearchStringEscape \
getSQLKeywords
getStringFunctions
getSystemFunctions
getTimeDateFunctions
getURL jdbc:phoenix:dn3,dn4,dn5:2181
getUserName
isCatalogAtStart false
isReadOnly false
nullPlusNonNullIsNull true
nullsAreSortedAtEnd false
nullsAreSortedAtStart true
nullsAreSortedHigh false
nullsAreSortedLow true
storesLowerCaseIdentifiers false
storesLowerCaseQuotedIdentifiers false
storesMixedCaseIdentifiers false
storesMixedCaseQuotedIdentifiers true
storesUpperCaseIdentifiers true
storesUpperCaseQuotedIdentifiers true
supportsAlterTableWithAddColumn true
supportsAlterTableWithDropColumn true
supportsANSI92EntryLevelSQL false
supportsANSI92FullSQL false
supportsANSI92IntermediateSQL false
supportsBatchUpdates true
supportsCatalogsInDataManipulation false
supportsCatalogsInIndexDefinitions false
supportsCatalogsInPrivilegeDefinitions false
supportsCatalogsInProcedureCalls false
supportsCatalogsInTableDefinitions false
supportsColumnAliasing true
supportsConvert true
supportsCoreSQLGrammar false
supportsCorrelatedSubqueries false
supportsDataDefinitionAndDataManipulationTransactionstrue
supportsDataManipulationTransactionsOnly false
supportsDifferentTableCorrelationNames false
supportsExpressionsInOrderBy true
supportsExtendedSQLGrammar false
supportsFullOuterJoins false
supportsGroupBy true
supportsGroupByBeyondSelect false
supportsGroupByUnrelated false
supportsIntegrityEnhancementFacility false
supportsLikeEscapeClause true
supportsLimitedOuterJoins false
supportsMinimumSQLGrammar false
supportsMixedCaseIdentifiers false
supportsMixedCaseQuotedIdentifiers true
supportsMultipleResultSets true
supportsMultipleTransactions true
supportsNonNullableColumns true
supportsOpenCursorsAcrossCommit false
supportsOpenCursorsAcrossRollback false
supportsOpenStatementsAcrossCommit false
supportsOpenStatementsAcrossRollback false
supportsOrderByUnrelated false
supportsOuterJoins true
supportsPositionedDelete false
supportsPositionedUpdate false
supportsSchemasInDataManipulation true
supportsSchemasInIndexDefinitions false
supportsSchemasInPrivilegeDefinitions false
supportsSchemasInProcedureCalls false
supportsSchemasInTableDefinitions false
supportsSelectForUpdate false
supportsStoredProcedures false
supportsSubqueriesInComparisons false
supportsSubqueriesInExists false
supportsSubqueriesInIns false
supportsSubqueriesInQuantifieds false
supportsTableCorrelationNames false
supportsTransactions true
supportsUnion false
supportsUnionAll false
usesLocalFilePerTable false
usesLocalFiles false
0: jdbc:phoenix:dn3,dn4,dn5:2181>