oracle 19c rac installation manual

Official documentation: https://docs.oracle.com/en/database/oracle/oracle-database/19/index.html

1. Cluster planning

Before deploying a cluster, you need to plan both network and storage aspects. The following is a brief description of what needs to be paid attention to in terms of network and storage.

Oracle RAC database private network communication must use a separate switch and cannot be directly connected through network cables.
It is recommended to use 10GE (10 Gigabit) network for the communication network between nodes.
The network card binding operation must be performed on the public and private network card networks, using mod=4, that is: (802.3ad) IEEE 802.3adDynamic link aggregation (IEEE 802.3ad dynamic link aggregation). The corresponding port of the switch needs to be configured in 802.3ad mode.
Multipath software is managed using multipath+udev.

1.1 Server planning

cluster role	CPU name	YOU	Public IP	Virtual IP	Private IP	Scan IP	Instance name	Grid/Oracle version
rac01	rac01	Oracle Linux Server 7.6	192.168.10.111	192.168.10.115	192.168.20.111	192.168.10.119	orcl1	19.3.0 to 19.11.0
rac02	rac02	Oracle Linux Server 7.6	192.168.10.113	192.168.10.117	192.168.20.113	192.168.10.119	orcl1	19.3.0 to 19.11.0

1.2 Storage planning

Because Oracle 19c Grid comes with a management library, you can choose to install it or not install it during installation. If you want to install it, the OCR disk group capacity requirement is relatively high. Configure at least 30Gx3 as an OCR disk group, otherwise the installation may fail.

Disk functions	number	Size of each (GB)	redundant mode
OCR+VOTING+(19C Grid management library)	3	30	NORMAL
DATA	2	20	EXTERNAL

2. Basic environment requirements and configuration

In all the following configurations, (rac01&rac02) means execution on two nodes, and (rac01) means execution only on one node.

2.1 Operating system requirements

According to Oracle official documentation, the operating system requirements for installing oracle19c are as follows:

Linux x86-64 operating system requirements:

- Oracle Linux 8.1 with the Unbreakable Enterprise Kernel 6: 5.4.17-2011.0.7.el8uek.x86_64 or later
  Oracle Linux 8 with the Red Hat Compatible kernel: 4.18.0-80.el8.x86_64 or later
- Oracle Linux 7.4 with the Unbreakable Enterprise Kernel 4: 4.1.12-124.19.2.el7uek.x86_64 or later
  Oracle Linux 7.4 with the Unbreakable Enterprise Kernel 5: 4.14.35-1818.1.6.el7uek.x86_64 or later
  Oracle Linux 7.7 with the Unbreakable Enterprise Kernel 6: 5.4.17-2011.4.4.el7uek.x86_64 or later
  Oracle Linux 7.5 with the Red Hat Compatible Kernel: 3.10.0-862.11.6.el7.x86_64 or later
- Red Hat Enterprise Linux 8: 4.18.0-80.el8.x86_64 or later
- Red Hat Enterprise Linux 7.5: 3.10.0-862.11.6.el7.x86_64 or later
- SUSE Linux Enterprise Server 15: 4.12.14-23-default or later
- SUSE Linux Enterprise Server 12 SP3: 4.4.162-94.72-default or later

IBM: Linux on System z operating system requirements:

- Red Hat Enterprise Linux 8.3: 4.18.0-240.el8.s390x or later
- Red Hat Enterprise Linux 7.4: 3.10.0-693.el7.s390x or later
- SUSE Linux Enterprise Server 15: 5.3.18-57-default s390x or later
- SUSE Linux Enterprise Server 12: 4.4.73-5-default s390x or later

2.2 Network card

According to the Oracle rac principle, two network cards are required to install rac, and the network cards need to be used accordingly, and the IP address is a static IP.

2.2.1 Disable automatic naming of network cards (rac01&rac02)

1. First modify the network card configuration file /etc/sysconfig/network-scripts/ifcfg-ens33 file and change the DEVICE and NAME items in ifcfg-ens33 to eth0

2. Then change the configuration file name to ifcfg-eth0

3. Disable the predictable naming rule. For this, you can pass the "net.ifnames=0 biosdevname=0" kernel parameter at boot time. This is achieved by editing /etc/default/grub and adding "net.ifnames=0 biosdevname=0" after the variable GRUB_CMDLINE_LINUX. The parameters are separated by spaces.

4. Run the command grub2-mkconfig -o /boot/grub2/grub.cfg to regenerate the GRUB configuration and update the kernel parameters, and restart.

[root@rac01 network-scripts]# cat /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap net.ifnames=0 biosdevname=0 rhgb quiet"
GRUB_DISABLE_RECOVERY="true"

#重新生成GRUB配置并更新内核参数
[root@rac01 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.14.35-1818.3.3.el7uek.x86_64
Found initrd image: /boot/initramfs-4.14.35-1818.3.3.el7uek.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-506ed14f59eb41c89599dc75decbb045
Found initrd image: /boot/initramfs-0-rescue-506ed14f59eb41c89599dc75decbb045.img
done
#重启服务器
[root@rac1 ~]# reboot

The above example is the operation performed on the rac01 server. Please perform the same operation on the rac02 server.

2.2.2 Change the network card to static ip (rac01&rac02)

Because the Oracle database has strict requirements on IP addresses, only static IPs can be used. The following is how to set a static IP.

#修改网卡的配置文件/etc/sysconfig/network-scripts/ifcfg-eth0文件
BOOTPROTO=static #启用静态IP地址
IPADDR=192.168.10.111 #设置IP地址
NETMASK=255.255.255.0 #设置子网掩码
GATEWAY=192.168.10.2 #设置网关
ONBOOT="yes" #设置网卡自启动
#重启网络
[root@rac01 network-scripts]# service network restart

The above example is the operation performed on the rac01 server. Please perform the same operation on the rac02 server.

2.2.3 Private network card configuration MTU (rac01&rac02)

Note: The private network card needs to be configured with MTU

Synchronously update the network card configuration file and add a line of MTU=9000 to ensure that mtu=9000 remains unchanged after restarting the network card/host:

vi /etc/sysconfig/network-scripts/ifcfg-eth1

PERSON=“9000”

2.3 Set the host name (rac01&rac02)

rac01

hostnamectl set-hostname rac01

rca02

hostnamectl set-hostname rac02

2.4 Disable Transparent HugePages(rac01&rac02)

Oracle officially requires disabling Transparent HugePages, but some books recommend enabling it with memory above 8G. It is recommended to follow the official requirements first. Edit /etc/default/grub and add "transparent_hugepage=never" after the variable GRUB_CMDLINE_LINUX to achieve this.

[root@rac01 network-scripts]# cat /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap net.ifnames=0 biosdevname=0 transparent_hugepage=never rhgb quiet"
GRUB_DISABLE_RECOVERY="true"

#重新生成GRUB配置并更新内核参数
[root@rac01 ~]# grub2-mkconfig -o /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-4.14.35-1818.3.3.el7uek.x86_64
Found initrd image: /boot/initramfs-4.14.35-1818.3.3.el7uek.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-506ed14f59eb41c89599dc75decbb045
Found initrd image: /boot/initramfs-0-rescue-506ed14f59eb41c89599dc75decbb045.img
done
#重启服务器
[root@rac1 ~]# reboot

After restarting, check the status as follows:

[root@rac01 ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]
[root@rac01 ~]# grep AnonHugePages /proc/meminfo
AnonHugePages:         0 kB

The above example is the operation performed on the rac01 server. Please perform the same operation on the rac02 server.

2.5 Turn off selinux and firewall (rac01&rac02)

selinux

sed -i 's/^ *SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config

firewall

systemctl disable firewalld
systemctl stop firewalld

2.6 Configure yum local source (rac01&rac02)

If the server can access the Internet, you do not need to configure the yum local source and use the yum configuration that comes with the operating system. If you cannot access the Internet or there is no available yum, you need to configure it. The specific configuration method is as follows:

#上传操作系统镜像文件至服务器某个目录，以/root/目录为例
mount -o loop -t iso9660 OracleLinux-Release7-Update6-x86_64-dvd.iso /media
#配置yum源
vi /etc/yum.repos.d/local.repo
[local]
name=local
baseurl=file:///media
gpgcheck=0
enabled=1
#清理缓存
yum clean all

2.7 Configure hosts (rac01&rac02)

Configure hosts files on 2 nodes, taking node 1 as an example:

[root@rac1 ~]# vim /etc/hosts
# 在文件的最后面加上
# public
192.168.10.111 rac01
192.168.10.113 rac02
#virtual
192.168.10.115 rac01-vip
192.168.10.117 rac02-vip
#private
192.168.20.111 rac01-priv
192.168.20.113 rac02-priv
#scan
192.168.10.119 rac-cluster-scan
配置完成后，重启网卡
[root@rac1 ~]# systemctl restart network

The above example is the operation performed on the rac01 server. Please perform the same operation on the rac02 server.

2.8 NTP clock synchronization (rac01&rac02)

If conditions permit, configure ntpd automatic clock synchronization. If there is no available clock synchronization server, you can try to use one of the two servers as a time server to provide clock synchronization services to the other server. In short, the purpose is to keep the clocks of the two servers synchronized.

yum install -y ntp ntpdate
systemctl enable ntpd --now

2.9 avahi-daemon configuration (rac01&rac02)

avahi-daemon will affect the heartbeat network between clusters and is recommended to be turned off (if it is not installed, install it and turn it off to avoid problems caused by careless installation later).

yum install -y avahi*
systemctl stop avahi-daemon.socket
systemctl stop avahi-daemon.service
pgrep -f avahi-daemon | awk '{print "kill -9 "$2}'
##配置NOZEROCONF=yes
cat <<EOF>>/etc/sysconfig/network
NOZEROCONF=yes
EOF

2.10 Install basic dependency packages (rac01&rac02)

yum groupinstall -y "Server with GUI"

yum install -y bc binutils compat-libcap1