[Practical project experience] Building openGauss Datakit 5.1.1 based on openEuler22.03

[Shenzhen] Yuanchuanghui: 5.26pm, the party hall is waiting for you.”

Preface There are considerable differences between the latest version of Datakit 5.1 and Datakit 5.0. The warehouse structure has changed and the previous documents are no longer applicable. This article will share the latest version of the Datakit practice manual and personal experience for the reference of openGauss enthusiasts.

1 Project Introduction This project is a web-based openGauss visual integrated tool: DataKit. The purpose is to facilitate customers to use and manage the openGauss visualization tool. It can lower the threshold for customers to install and use the openGauss database, achieve security center management, plug-in management, and The upper-layer development and operation and maintenance tools are plug-in-based, and each plug-in is independent of each other, making it easy for users to introduce it on demand. Each plug-in is expanded around DataKit's resource center, providing one-click deployment, uninstallation, component installation, multi-version upgrades and daily operation and maintenance, monitoring, migration, development, modeling and other complex operations.

2 Environment preparation 2.1 Notes The current platform operation depends on openJDK11

The database used by the platform currently only supports the openGauss database, and the database needs to be created in advance.

The deployment server IP needs to be configured in the whitelist of the openGauss database used by the platform.

2.2 Supported server systems

openEuler 20.3LTS（x86_x64，ARM）
centos7.x（x86_x64)

Note: It has been verified that it can run perfectly on openEuler 22.03 LTS.

Check operating system version

[root@localhost ~]# cat /etc/os-release 
NAME="openEuler"
VERSION="22.03 LTS"
ID="openEuler"
VERSION_ID="22.03"
PRETTY_NAME="openEuler 22.03 LTS"
ANSI_COLOR="0;31"

2.3 Environmental Summary

This article deploys Datakit 5.1.1 based on the openEuler 22.03 LTS operating system.

2.4 Document summary

openGauss official documentation

https://docs-opengauss.osinfra.cn/zh/docs/5.1.0/docs/ToolandCommandReference/DataKit.htmlUsage documentation and development documentation

https://gitee.com/opengauss/openGauss-workbench/tree/master/openGauss-datakit/doc 2.5 Software package acquisition Here I have organized the various offline packages used into datakit-about.zip, and sent the datakit in the background You can get it.

You can also install online or download the corresponding offline package for installation.

2.5.1 There are two ways to obtain the Datakit software package:

Download through the openGauss official website

https://opengauss.org/zh/download/

Download through the opengauss official gitee repository

Warehouse Address

https://gitee.com/opengauss/openGauss-workbench

The current latest version installation package address

https://opengauss.obs.cn-south-1.myhuaweicloud.com/latest/tools/Datakit/Datakit-5.1.1.tar.gz The installation package of the openGauss official gitee warehouse is generally a higher version than the installation package of the openGauss official website. , containing the latest features and fundamental fixes.

2.5.2 Obtaining the openGauss software package Starting from the openEuler 22.03 LTS version, the operating system image comes with the openGauss RPM package, which can be installed through rpm or yum. It supports both arm64 and x86_64 architectures.

The mirror address of openEuler22.03 LTS is:

https://mirrors.huaweicloud.com/openeuler/openEuler-22.03-LTS You can download the corresponding RPM package at the above address.

# opengauss安装包
opengauss-2.1.0-5.oe2203.x86_64.rpm# 依赖包
java-1.8.0-openjdk-1.8.0.352.b08-3.oe2203.x86_64.rpm
java-1.8.0-openjdk-devel-1.8.0.352.b08-3.oe2203.x86_64.rpm
java-1.8.0-openjdk-headless-1.8.0.352.b08-3.oe2203.x86_64.rpm

2.5.3 Obtaining openJDK11 package

# openjdk11安装包
java-11-openjdk-11.0.20.8-0.oe2203.x86_64.rpm
java-11-openjdk-devel-11.0.20.8-0.oe2203.x86_64.rpm
java-11-openjdk-headless-11.0.20.8-0.oe2203.x86_64.rpm# 依赖包
giflib-5.2.1-4.oe2203.x86_64.rpm

2.6 Upload datakit-about.zip to the server

# 创建安装包存放目录
[root@localhost ~]# mkdir -p /soft/
# 上传 datakit-about.zip 到 /soft 并解压
[root@localhost ~]# cd /soft
[root@localhost /soft]# unzip datakit-about.zip
# 解压后的目录结构如下
├── datakit
│   └── Datakit-5.1.1.tar.gz
├── opengaussrpm
│   ├── java-1.8.0-openjdk-1.8.0.352.b08-3.oe2203.x86_64.rpm
│   ├── java-1.8.0-openjdk-devel-1.8.0.352.b08-3.oe2203.x86_64.rpm
│   ├── java-1.8.0-openjdk-headless-1.8.0.352.b08-3.oe2203.x86_64.rpm
│   └── opengauss-2.1.0-5.oe2203.x86_64.rpm
└── openjdk11
    ├── giflib-5.2.1-4.oe2203.x86_64.rpm
    ├── java-11-openjdk-11.0.20.8-0.oe2203.x86_64.rpm
    ├── java-11-openjdk-devel-11.0.20.8-0.oe2203.x86_64.rpm
    └── java-11-openjdk-headless-11.0.20.8-0.oe2203.x86_64.rpm
 
3 directories, 9 files

2.7 Set host name and IP address resolution

2.7.1 Set hostname

[root@localhost ~]# hostnamectl set-hostname trex

2.7.2 IP address resolution

[root@localhost ~]# echo '192.168.7.100 trex' >> /etc/hosts

2.7.3 Refresh host name

[root@localhost ~]# su [root@trex ~]#

You can see that the host name has changed from the default localhost to trex.

2.8 Turn off the firewall

 

# 关闭防火墙
[root@trex ~]# systemctl stop firewalld.service
# 禁止开机自启
[root@trex ~]# systemctl disable firewalld.service

2.9 Turn off selinux 

2.9.1 Immediate shutdown

[root@trex ~]# setenforce 0

Note: Invalid after restarting the operating system.

2.9.2 Modify the configuration file by editing /etc/sysconfig/selinux and change SELINUX=enforcing to SELINUX=disabled

sed -ri 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/config

Note: The operating system needs to be restarted to take effect permanently.

2.10 Install the database used by the Datakit platform. As mentioned earlier in the article, the platform currently only supports openGauss.

Installation via Internet

yum install opengauss -y

Offline installation

# cd /soft/datakit-about/opengaussrpm/
[root@trex opengaussrpm]# rpm -ivh *.rpm
Verifying...                          ################################# [100%]
Preparing...                          ################################# [100%]
Updating / installing...
   1:java-1.8.0-openjdk-headless-1:1.8################################# [ 25%]
   2:java-1.8.0-openjdk-1:1.8.0.352.b0################################# [ 50%]
   3:java-1.8.0-openjdk-devel-1:1.8.0.################################# [ 75%]
   4:opengauss-2.1.0-5.oe2203         ################################# [100%]
Created symlink /etc/systemd/system/multi-user.target.wants/opengauss.service → /usr/lib/systemd/system/opengauss.service.
Init openGauss database success.
Start openGauss database success.

2.11 opengauss database management and configuration

2.11.1 View startup status

# 切换到opengauss用户
[root@trex ~]# su - opengauss
# 查看opengauss启动状态
[opengauss@trex ~]$ gs_ctl status
# 打印信息如下
[2023-12-06 11:50:31.604][69114][][gs_ctl]: gs_ctl status,datadir is /var/lib/opengauss/data 
gs_ctl: server is running (PID: 61978)
/usr/local/opengauss/bin/gaussdb "-D" "/var/lib/opengauss/data"

Displaying server is running indicates that the status is normal.

Note: Common commands for openGauss service management are as follows:

Start gs_ctl start

close gs_ctl stop

restartgs_ctl restart

statusgs_ctl status

2.11.2 Set a whitelist for database access

This is set to allow clients with any IP address to access the openGauss database through TCP/IP

# 进入openGauss数据目录
[opengauss@trex ~]$ cd $PGDATA
# 在修改前先备份，避免异常情况
[opengauss@trex data]$ cp pg_hba.conf pg_hba.conf.bak
# 追加配置信息
[opengauss@trex data]$ echo 'host    all             all             0.0.0.0/0               md5' >> pg_hba.conf

Detailed explanation of parameters:

host: Indicates that this rule applies to TCP/IP connections.

all: Indicates that this rule applies to all databases and all users.

0.0.0.0/0: is an IP address range in CIDR format, indicating that clients with any IP address are allowed to connect.

md5: Indicates using MD5 encryption method to verify the password.

2.11.3 Modify database parameters

# 设置监听所有ip
[opengauss@trex ~]$ sed -i "s/^listen_addresses = .*/listen_addresses = '*'/" postgresql.conf
# 修改加密算法为0
[opengauss@trex ~]$ sed -i "s/^#password_encryption_type = .*/password_encryption_type = 0/" postgresql.conf
# 修改最大连接数为1000
[opengauss@trex ~]$ sed -i "s/^max_connections = .*/max_connections = 1000/" postgresql.conf

Check whether the modification is correct

[opengauss@trex ~]$ grep -E 'listen_addresses|password_encryption_type|max_connections' postgresql.conf

Detailed explanation of parameters:
listen_addresses = '*' # Listen to all ip
password_encryption_type = 0 # Encryption algorithm
max_connections = 1000 # Maximum number of connections

2.11.4 Restart the openGauss service and check the startup status. After modifying the parameters, you need to restart the openGauss service to take effect.

[opengauss@trex ~]$ gs_ctl restart > /dev/null 2>&1

查看启动状态
[opengauss@trex ~]$ gs_ctl status

2.11.5 Create user

登录数据库（操作系统命令）
[opengauss@trex ~]$ gsql -d postgres -p 7654 -r

初次登录需要修改密码（sql命令）
openGauss=# ALTER ROLE opengauss PASSWORD 'Trex@123';

创建用户（sql命令）
openGauss=# CREATE USER trex IDENTIFIED BY 'Trex@123';

授权（sql命令）
openGauss=# ALTER USER trex SYSADMIN;

创建数据库（sql命令）
openGauss=# create database trexdb;

退出sql（sql命令）
openGauss=# \q 

2.12 Install openJDK11 and execute as root user

Installation via Internet

[root@trex ~]# yum install -y java-11-openjdk

Offline installation

[root@trex ~]# cd /soft/datakit-about/openjdk11/ [root@trex openjdk]# rpm -ivh *.rpm

Careful readers will find that when using the rpm or yum command to install opengauss, the dependency package openJDK8 was installed, and the Datakit platform depends on openJDK11. There are currently two versions of JDK in the system. There will be no conflict, right?

Let’s check it through the command

[root @trex openjdk]# rpm -qa | grep openjdk The result is as follows

Sure enough, there are two versions of JDK, will they conflict?

The answer is: the two versions of openJDK can coexist. It will be mentioned later and will not be repeated here.

3 Install and deploy Datakit and execute it as root user

3.1 Datakit has two deployment methods: jar package deployment

Deploy via docker

Next, install it as a jar package for demonstration.

3.2 Create datakit working directory

[root@trex ~]# mkdir -p /ops/{logs,config,ssl,files}

3.3 Move the Datakit installation package to the /ops directory

[root@trex ~]# cd /soft/datakit-about/datakit/ [root@trex datakit]# mv Datakit-5.1.1.tar.gz /ops/

3.4 Unzip the installation package

[root@trex datakit]# cd /ops [root@trex datakit]# tar -xf Datakit-5.1.1.tar.gz You can see the following information by executing the ls command

[root@trex ops]# ls application-temp.yml Datakit-5.1.1.tar.gz files openGauss-datakit-5.1.1.jar visualtool-plugin config doc logs ssl

3.5 Modify configuration file

3.5.1 Move the configuration file to the /ops/config directory [root@trex ops]# mv /ops/application-temp.yml /ops/config/

3.5.2 Edit configuration file

[root@trex ops]# vi /ops/config/application-temp.yml

Modify application-temp.ymlthe data link information in the file ip、port、database、dbuser、dbpassword.
Modify it according to your actual environment. The modified file content here is as follows

system:
  # File storage path
  defaultStoragePath: /ops/files
  # Whitelist control switch
  whitelist:
    enabled: false
server:
  port: 9494
  ssl:
    key-store: /ops/ssl/keystore.p12
    key-store-password: 123456
    key-store-type: PKCS12
    enabled: true
  servlet:
    context-path: /
logging:
  file:
    path: /ops/logs/
spring:
  datasource:
    type: com.alibaba.druid.pool.DruidDataSource
    driver-class-name: org.opengauss.Driver
    url: jdbc:opengauss://192.168.7.100:7654/trexdb?currentSchema=public&batchMode=off
    username: trex
    password: Trex@123
    druid:
      test-while-idle: true
      test-on-borrow: true
      validation-query: "select 1"
      validation-query-timeout: 10000
      connection-error-retry-attempts: 0
      break-after-acquire-failure: true
      max-wait: 6000
      keep-alive: true
      max-active: 30
      min-evictable-idle-time-millis: 600000
management:
  server:
    port: 9494

 

3.6 Create key file Here we use the keytool tool to generate the key. It should be noted that the keytool -genkey command is used to create SSL files (including key pairs and certificates), and it is related to the JDK (Java Development Kit). keytool is a tool provided by the JDK for managing keystores and certificates for secure communications such as encryption and authentication. We mentioned earlier that the platform relies on the JDK11 environment, so JDK11 is used here to create the key file.

3.6.1 Switch to the openJDK11 environment and check the JDK version

# 查看当前jdk版本[root@trex config]# java -version# 切换版本到jdk11
[root@trex config]# alternatives --config java# 再次查看当前jdk版本
[root@trex config]# java -version# 执行结果如下

3.6.2 Create key file

keytool -genkey -noprompt \
    -dname "CN=opengauss, OU=opengauss, O=opengauss, L=Beijing, S=Beijing, C=CN"\
    -alias opengauss\
    -storetype PKCS12 \
    -keyalg RSA \
    -keysize 2048 \
    -keystore /ops/ssl/keystore.p12 \
    -validity 3650 \
    -storepass 123456

Note: The storepass is consistent with the key-store-password in the configuration file application-temp.yml. The keystore path is consistent with the key-store path in the configuration file application-temp.yml.

3.7 Execute startup command

# 进入 /ops 目录
cd /ops
# 启动程序
nohup java -Xms2048m -Xmx4096m -jar openGauss-datakit-5.1.1.jar --spring.profiles.active=temp >datakit.out 2>&1 &

Note: You can view the running log through tail -20f datakit.out

4 Use of Datakit

4.1 To log in to the management interface, open the following address in the browser

https://192.168.7.100:9494

Because the browser has security protection, click [Advanced] here - select Continue to enter the login interface.

The default login account password of the platform is: admin/admin123. When logging in for the first time, you will be prompted to reset the default password.

Enter the main interface

At this point, openGauss Datakit has been successfully deployed through the jar package.

The pirated resources of "Qing Yu Nian 2" were uploaded to npm, causing npmmirror to have to suspend the unpkg service. Zhou Hongyi: There is not much time left for Google. I suggest that all products be open source. Please tell me, time.sleep(6) here plays a role. What does it do? Linus is the most active in "eating dog food"! The new iPad Pro uses 12GB of memory chips, but claims to have 8GB of memory. People’s Daily Online reviews office software’s matryoshka-style charging: Only by actively solving the “set” can we have a future. Flutter 3.22 and Dart 3.4 release a new development paradigm for Vue3, without the need for `ref/reactive `, no need for `ref.value` MySQL 8.4 LTS Chinese manual released: Help you master the new realm of database management Tongyi Qianwen GPT-4 level main model price reduced by 97%, 1 yuan and 2 million tokens

{{o.name}}

{{m.name}}