hive --metastore three modes

The official website to introduce these types of models are as follows:

Metastore database by location points:

1, local / Metastore embedded database (Derby)

2, Metastore remote database (other relational databases, like mysql, oracle, etc.)

Press Metastore server divided into the following:

1, the local embedded service Metastore

2, remote service Metastore

 

The following are three ways for the database location points.

A, derby mode (local / Metastore embedded database), the default mode is the hive

Metastore embedded database mainly for unit testing. Only one process Metastore connection to the database, so it is not really a practical solution, but is suitable for unit testing. Do not use the production environment in this way.

Configuration:

(1) to extract the hive / data1 / hadoop / directory

(2) modify the configuration file

#copy hive-default.xml.template  hive-site.xml

Configuration is as follows:

<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:derby:;databaseName=metastore_db;create=true</value>
  <description>JDBC connect string for a JDBC metastore</description>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>org.apache.derby.jdbc.EmbeddedDriver</value>
  <description>Driver class name for a JDBC metastore</description>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>APP</value>
  <description>username to use against metastore database</description>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>mine</value>
  <description>password to use against metastore database</description>
</property>
 
<property>
  <name>hive.metastore.warehouse.dir</name>
  <value>file:///Users/micmiu/tmp/hive/warehouse</value>
  <description>unit test data goes in here on your local filesystem</description>
</property>

Perform initialization commands:schematool -dbType derby -initSchema

View information after initialization: schematool -dbType derby -info

This model with less

Second, local mode (Metastore local database, mysql storage.)

<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://localhost/hive?createDatabaseIfNotExist=true</value>
  <description>JDBC connect string for a JDBC metastore</description>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>Driver class name for a JDBC metastore</description>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
  <description>username to use against metastore database</description>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>micmiu</value>
  <description>password to use against metastore database</description>
</property>
 
<property>
  <name>hive.metastore.warehouse.dir</name>
  <!-- base hdfs path -->
  <value>/user/hive/warehouse</value>
  <description>location of default database for the warehouse</description>

Mysql driver package need to copy the directory <HIVE_HOME> / lib in

The first time you need to perform the initialization commands:schematool -dbType mysql -initSchema

Third, the remote mode (that is, mysql database and hive installed on different servers)

This storage need to run a mysql server on a remote server, and services need to start the meta Hive server.

Mysql server installation is slave1

Master server installation hive sevrver

<configuration>
 
<property>
  <name>hive.metastore.warehouse.dir</name>
  <value>/user/hive/warehouse</value>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://slave1:3306/hive_remote?createDatabaseIfNotExist=true</value>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>password</value>
</ Property> 
 
<Property> 
  <name> hive.metastore.local </ name> # mode into a false, that is, using the remote mode, the default is true, which is to use native mode 
  <value> false </ value> 
</ Property> 
 
<Property> 
  <name> hive.metastore.uris </ name> 
  <value> Thrift: // Master: 9083 </ value> #MASTER to install hive or start metastore services server address 
</ Property> 
 
< / configuration>

 

Above the hive server and client are arranged in a machine. The following split:

server configuration:

<configuration>
 
<property>
  <name>hive.metastore.warehouse.dir</name>
  <value>/user/hive/warehouse</value>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://slave1:3306/hive_remote?createDatabaseIfNotExist=true</value>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>hive</value>
</property>
 
<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>password</value>
</property>

</configuration>

Client Configuration:

<configuration>
 
<property>
  <name>hive.metastore.warehouse.dir</name>
  <value>/user/hive/warehouse</value>
</property>

<Property> 
  <name> hive.metastore.local </ name>          # mode into a false, that is, using the remote mode, the default is true, which is to use native mode
   <value> false </ value> 
</ Property> 
 
<Property> 
  <name> hive.metastore.uris </ name> 
  <value> Thrift: // Master: 9083 </ value> #MASTER to install hive or start the server address metastore services 
</ Property> 
 
</ the Configuration>

Mysql driver package need to copy the directory <HIVE_HOME> / lib in

The first time you need to perform the initialization commands:schematool -dbType mysql -initSchema

hive metastore server startup command:
hive --service metastore -p <port_num>
if we do not start by default port: hive --service metastorethe default listening port is: 9083, note that the port port configuration the client's needs and start listening. After the server starts normally, the client can perform the operation hive.

If you are using jdbc way to connect, you need to start hiveserver2 service server, the service or services are using thrift metastore agreement, listening to port 10000.

Start hiveserver2

#hive --service hiveserver2 & Alternatively or hiveservre2 & nohup hiveserver2 1> xxx.log 2> xxx_err.log &