ORACLE + RAC 원리 및 실습 시리즈 : 1 개의 RAC 설치

수요와 환경

ORACLE 19c RAC 환경을 설치하고 배포합니다. 55.34.4.153, 55.34.4.154는, 두 개의 VM웨어 호스트는 각각 두 개의 네트워크 카드로 구성 하고, 적용되는 동일한 네트워크 세그먼트 5 인 IP (공용 IP) , 및 2 개의 VM웨어 호스트 세 공유 디스크 (OCRDG, DATADG로 구성된 FRADG);

예비 준비

그리드 사용자 생성 (듀얼 노드)

useradd 그리드

groupadd oinstall

groupadd dba

groupadd 연산자

groupadd asmadmin

groupadd asmoper

groupadd asmdba

groupadd racdba

usermod -g oinstall -G dba, oper, asmadmin, asmoper, asmdba, racdba 그리드

uid = 1779 (grid) gid = 1778 (oinstall) groups = 1778 (oinstall), 1779 (dba), 1780 (oper), 504 (asmadmin), 505 (asmoper), 506 (asmdba), 1784 (racdba)

두 사용자 간의 2 노드 구성 그리드 / 루트 상호 신뢰 (자신 노드에 대한 상호 신뢰 및 다른 노드에 대한 상호 신뢰 포함)

설치 디렉터리 생성 (이중 노드)

mkdir gridhome
mkdir gridbase
mkdir gridInventory
chown grid:oinstall grid*

공유 디스크 영구 바인딩

#!/bin/bash

for i in c d e f
do
echo $i
echo "KERNEL==\"sd?\",SUBSYSTEM==\"block\", PROGRAM==\"/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/\$name\",RESULT==\"`/usr/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$i`\", SYMLINK+=\"asm-disk$i\",OWNER=\"grid\", GROUP=\"asmadmin\",MODE=\"0660\""  >>/etc/udev/rules.d/99-oracle-asmdevices.rules

done

/sbin/udevadm trigger --type=devices --action=change
ls -l /dev/asm*

exit 0

공유 디스크 형식

dd if = / dev / zero of = / dev / sdc bs = 512K count = 1

dd if = / dev / zero of = / dev / sdd bs = 512K count = 1

dd if = / dev / zero of = / dev / sde bs = 512K count = 1

네트워크 구성

######################################## RAC 시작

#pubilc ip

55.34.4.153 ora19node1

55.34.4.154 ora19node2

# 비공개 IP

192.168.124.5 ora19node1-priv

192.168.124.6 ora19node2-priv

#vitrual ip

55.34.4.157 ora19node1-vip

55.34.4.158 ora19node2-vip

# 스캔 ip

55.34.4.159 ora19nodescan

############################# RAC END ################# ##############

/ etc / hosts 구성 (이중 노드)

공용 IP, 공용 네트워크 VIP, 스캔 IP 및 사설 네트워크 IP 포함;

ifconfig ens224 192.168.124.5 위로

ifconfig ens224 192.168.124.6 위로

IP 주소 삭제 192.168.124.5 dev ens224

2 노드 사설 네트워크 카드는 별도로 온라인에 연결됩니다.

공식 설치 (GI)

해 grid home
권한 수정

chown  -R grid:oinstall   /data/gridhome/

그리드 사용자로 확인

./runcluvfy.sh stage -post hwos -n ora19node1, ora19node2 -verbose	하드웨어 및 OS 확인
./runcluvfy.sh stage -pre crsinst -n ora19node1, ora19node2 -fixup -verbose	crs 설치 확인

참고 : 실패 할 경우 자동 복구 스크립트가 나타나며 루트 사용자로 두 노드에서 실행할 수 있습니다 (runclufy.sh 실행 결과 참조). 내 vmware 호스트에는 테이블과 같이 설치되지 않은 여러 종속 패키지가 있습니다.

/tmp/CVU_19.0.0.0.0_grid/runfixup.sh

libaio-devel-0.3.109	yum libaio-devel 설치
nfs-utils-1.2.3-15
compat-libstdc ++-33-3.2.3

shm 크기 수정 (/ etc / fstab 수정)

#rac 용

tmpfs /dev/shm tmpfs defaults,size=8G 0 0

mount -o remount /dev/shm

注意：上面几个步骤都是本人通过runcluvfy.sh 脚本检查出来的，读者在实践过程中，用此脚本多执行几次，逐一解决问题即可。

创建安装响应文件

vim grid_install.rsp

此相应文件在grid_home 有模板，读者请仔细研读模板（如GI安装类型、各种用户配置等），再填写各个参数。

[grid@sgpbnk01kf ~]<20200902 08:37:39>$ cat grid19_install_new.rsp

oracle.install.responseFileVersion=/oracle/install/rspfmt_crsinstall_response_schema_v19.0.0

####A

INVENTORY_LOCATION=/data/gridInventory

oracle.install.option=CRS_CONFIG

ORACLE_BASE=/data/grid19base

####B

oracle.install.asm.OSDBA=asmdba

oracle.install.asm.OSOPER=asmoper

oracle.install.asm.OSASM=asmadmin

####C

oracle.install.crs.config.scanType=LOCAL_SCAN

oracle.install.crs.config.SCANClientDataFile=

oracle.install.crs.config.gpnp.scanName=sgpbnkkfscan

oracle.install.crs.config.gpnp.scanPort=1530

####D

oracle.install.crs.config.ClusterConfiguration=STANDALONE

oracle.install.crs.config.configureAsExtendedCluster=false

oracle.install.crs.config.memberClusterManifestFile=

oracle.install.crs.config.clusterName=sgpbnkkfclus

oracle.install.crs.config.gpnp.configureGNS=false

oracle.install.crs.config.autoConfigureClusterNodeVIP=false

oracle.install.crs.config.gpnp.gnsOption=

oracle.install.crs.config.gpnp.gnsClientDataFile=

oracle.install.crs.config.gpnp.gnsSubDomain=

oracle.install.crs.config.gpnp.gnsVIPAddress=

oracle.install.crs.config.sites=

oracle.install.crs.config.clusterNodes=sgpbnk01kf:sgpbnk01kf-vip,sgpbnk02kf:sgpbnk02kf-vip

oracle.install.crs.config.networkInterfaceList=eth0:55.14.64.0:1,ens256:192.168.124.0:5

oracle.install.crs.configureGIMR=

oracle.install.asm.configureGIMRDataDG=false

####E

oracle.install.crs.config.storageOption=CLIENT_ASM_STORAGE

oracle.install.crs.config.sharedFileSystemStorage.votingDiskLocations=/dev/asm-diskf

oracle.install.crs.config.sharedFileSystemStorage.ocrLocations=/dev/asm-diskf

####F

oracle.install.crs.config.useIPMI=false

oracle.install.crs.config.ipmi.bmcUsername=

oracle.install.crs.config.ipmi.bmcPassword=

####G

oracle.install.asm.SYSASMPassword=Password12345A

oracle.install.asm.diskGroup.name=OCRDG

oracle.install.asm.diskGroup.redundancy=EXTERNAL

oracle.install.asm.diskGroup.AUSize=4

oracle.install.asm.diskGroup.FailureGroups=

oracle.install.asm.diskGroup.disksWithFailureGroupNames=

oracle.install.asm.diskGroup.disks=/dev/asm-diskf

oracle.install.asm.diskGroup.quorumFailureGroupNames=

oracle.install.asm.diskGroup.diskDiscoveryString=/dev/asm-disk*

oracle.install.asm.monitorPassword=Password12345A

oracle.install.asm.gimrDG.name=OCRDG

oracle.install.asm.gimrDG.redundancy=

oracle.install.asm.gimrDG.AUSize=4

oracle.install.asm.gimrDG.FailureGroups=

oracle.install.asm.gimrDG.disksWithFailureGroupNames=

oracle.install.asm.gimrDG.disks=/dev/asm-diskf

oracle.install.asm.gimrDG.quorumFailureGroupNames=

oracle.install.asm.configureAFD=

oracle.install.crs.configureRHPS=

##END

./gridSetup.sh -silent -ignorePrereq -responseFile /home/grid/grid_install.rsp

静默安装

[grid@sgpbnk01kf gridhome]<20200902 08:45:30>$ ./gridSetup.sh -silent -ignorePrereq -responseFile /home/grid/grid_install.rsp

Launching Oracle Grid Infrastructure Setup Wizard...

[WARNING] [INS-13013] Target environment does not meet some mandatory requirements.

CAUSE: Some of the mandatory prerequisites are not met. See logs for details. /tmp/GridSetupActions2020-09-02_08-45-59AM/gridSetupActions2020-09-02_08-45-59AM.log

ACTION: Identify the list of failed prerequisite checks from the log: /tmp/GridSetupActions2020-09-02_08-45-59AM/gridSetupActions2020-09-02_08-45-59AM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually.

The response file for this session can be found at:

/u01/gridhome/install/response/grid_2020-09-02_08-45-59AM.rsp

You can find the log of this install session at:

/tmp/GridSetupActions2020-09-02_08-45-59AM/gridSetupActions2020-09-02_08-45-59AM.log

As a root user, execute the following script(s):

1. /u01/gridInventory/orainstRoot.sh

2. /u01/gridhome/root.sh

Execute /u01/gridInventory/orainstRoot.sh on the following nodes:

[sgpbnk01kf, sgpbnk02kf]

Execute /u01/gridhome/root.sh on the following nodes:

[sgpbnk01kf, sgpbnk02kf]

Run the script on the local node first. After successful completion, you can start the script in parallel on all other nodes.

Successfully Setup Software with warning(s).

As install user, execute the following command to complete the configuration.

/u01/gridhome/gridSetup.sh -executeConfigTools -responseFile /home/grid/grid19_install_new.rsp [-silent]

一次性成功安装。

双节点依次执行root脚本，注意要依次执行：现在一个节点执行，并观察日志，确保成功再在第二个节点执行

sh /data/gridInventory/orainstRoot.sh
sh  /data/gridhome/root.sh

[root@sgpbnk01kf ~]<20200902 08:51:57># sh /u01/gridInventory/orainstRoot.sh

Changing permissions of /u01/gridInventory.

Adding read,write permissions for group.

Removing read,write,execute permissions for world.

Changing groupname of /u01/gridInventory to oinstall.

The execution of the script is complete.

[root@sgpbnk01kf ~]<20200902 08:53:01># sh /u01/gridhome/root.sh

Check /u01/gridhome/install/root_sgpbnk01kf_2020-09-02_08-53-03-790273956.log for the output of root script

GI安装后的自动化配置

./gridSetup.sh -silent -executeConfigTools -responseFile /home/grid/grid_install.rsp

GI检查

按照上面的流程，GI安装完毕，可以运行相关命令检查GI的状态。

crsctl status resource -t	查看资源状态，按表格显示-t
srvctl status scan	查看scan的状态
crsctl status resourcegroup -t	查看资源组状态
crsctl status type	查看类型状态（不懂）
crsctl status serverpool	查看服务池状态
crsctl status server	查看服务状态
crsctl check crs	检查集群同步服务
crsctl check ctss	检查集群时间同步(Cluster Time Synchronization Service)
crsctl get cpu equivalency	获取CPU 信息
crsctl get node role status -all	获取节点角色
Olsnodes	查看节点数
crsctl query css votedisk	查询vote盘
srvctl status asm	查看ASM状态
crsctl check cluster	查看集群状态

正式安装（DB）

用户

uid=1778(oracle) gid=1778(oinstall) groups=1778(oinstall),1779(dba),1780(oper),504(asmadmin),506(asmdba),1781(backupdba),1782(dgdba),1783(kmdba),1784(racdba)

useradd oracle

groupadd dba

groupadd oper

groupadd asmadmin

groupadd asmdba

groupadd backupdba

groupadd dgdba

groupadd kmdba

groupadd racdba

usermod -g oinstall -G dba,oper,asmadmin,asmdba,backupdba,dgdba,kmdba,racdba oracle

配置oracle 用户互信

磁盘规划

节点1	节点2
/u01/dbhome	/u01/dbhome
/u01/dbbase	/u01/dbbase
/u01/dbInventory	/u01/dbInventory

DB安装检查

DB安装检查(grid_home目录)

./runcluvfy.sh stage -pre dbinst -n "ora19node1,ora19node2" -verbose

编写响应文件

[oracle@sgpbnk01kf ~]<20200902 09:44:13>$ cat db19c_install.rsp

oracle.install.responseFileVersion=/oracle/install/rspfmt_dbinstall_response_schema_v19.0.0

oracle.install.option=INSTALL_DB_SWONLY

UNIX_GROUP_NAME=oinstall

INVENTORY_LOCATION=/u01/dbInventory

ORACLE_HOME=/u01/dbhome

ORACLE_BASE=/u01/dbbase

oracle.install.db.InstallEdition=EE

oracle.install.db.OSDBA_GROUP=dba

oracle.install.db.OSOPER_GROUP=oper

oracle.install.db.OSBACKUPDBA_GROUP=backupdba

oracle.install.db.OSDGDBA_GROUP=dgdba

oracle.install.db.OSKMDBA_GROUP=kmdba

oracle.install.db.OSRACDBA_GROUP=racdba

oracle.install.db.rootconfig.executeRootScript=true

oracle.install.db.rootconfig.configMethod=ROOT

oracle.install.db.rootconfig.sudoPath=

oracle.install.db.rootconfig.sudoUserName=

oracle.install.db.CLUSTER_NODES=sgpbnk01kf,sgpbnk02kf

oracle.install.db.config.starterdb.type=

oracle.install.db.config.starterdb.globalDBName=

oracle.install.db.config.starterdb.SID=

oracle.install.db.ConfigureAsContainerDB=

oracle.install.db.config.PDBName=

oracle.install.db.config.starterdb.characterSet=

oracle.install.db.config.starterdb.memoryOption=

oracle.install.db.config.starterdb.memoryLimit=

oracle.install.db.config.starterdb.installExampleSchemas=

oracle.install.db.config.starterdb.password.ALL=

oracle.install.db.config.starterdb.password.SYS=

oracle.install.db.config.starterdb.password.SYSTEM=

oracle.install.db.config.starterdb.password.DBSNMP=

oracle.install.db.config.starterdb.password.PDBADMIN=

oracle.install.db.config.starterdb.managementOption=

oracle.install.db.config.starterdb.omsHost=

oracle.install.db.config.starterdb.omsPort=

oracle.install.db.config.starterdb.emAdminUser=

oracle.install.db.config.starterdb.emAdminPassword=

oracle.install.db.config.starterdb.enableRecovery=

oracle.install.db.config.starterdb.storageType=

oracle.install.db.config.starterdb.fileSystemStorage.dataLocation=

oracle.install.db.config.starterdb.fileSystemStorage.recoveryLocation=

oracle.install.db.config.asm.diskGroup=

oracle.install.db.config.asm.ASMSNMPPassword=

[oracle@ora19node1 dbhome]<20210125 15:52:14>$ ./runInstaller -silent -ignorePrereqFailure -waitForCompletion -responseFile /home/oracle/dbinstall.rsp

Launching Oracle Database Setup Wizard...

Enter password for 'root' user:

[WARNING] [INS-13013] Target environment does not meet some mandatory requirements.

CAUSE: Some of the mandatory prerequisites are not met. See logs for details. /data/gridInventory/logs/InstallActions2021-01-25_03-52-17PM/installActions2021-01-25_03-52-17PM.log

ACTION: Identify the list of failed prerequisite checks from the log: /data/gridInventory/logs/InstallActions2021-01-25_03-52-17PM/installActions2021-01-25_03-52-17PM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually.

The response file for this session can be found at:

/data/dbhome/install/response/db_2021-01-25_03-52-17PM.rsp

You can find the log of this install session at:

/data/gridInventory/logs/InstallActions2021-01-25_03-52-17PM/installActions2021-01-25_03-52-17PM.log

RAC建库

oracle用户检查

dbca -silent -executePrereqs -databaseConfigType RAC -nodelist ora19node1,ora19node2

检查并创建磁盘组

select GROUP_NUMBER,name,path from v$asm_disk;
select NAME,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;
ALTER DISKGROUP OCRDG SET ATTRIBUTE 'compatible.asm' = '19.0';
ALTER DISKGROUP OCRDG SET ATTRIBUTE 'compatible.rdbms' = '19.0';


CREATE DISKGROUP FRADG EXTERNAL redundancy DISK '/dev/asm-diskd' ATTRIBUTE 'compatible.asm' = '19.0', 'compatible.rdbms'='19.0','compatible.advm' = '19.0';
CREATE DISKGROUP DATADG EXTERNAL redundancy DISK '/dev/asm-diske' ATTRIBUTE 'compatible.asm' = '19.0', 'compatible.rdbms'='19.0','compatible.advm' = '19.0';
alter diskgroup FRADG mount;
alter diskgroup DATADG mount;

创建RAC响应文件

[oracle@sgpbnk01kf ~]<20200902 15:15:30>$ cat dbca_rac.rsp

responseFileVersion=/oracle/assistants/rspfmt_dbca_response_schema_v19.0.0

gdbName=ORA19RAC

sid=ORA19RAC

databaseConfigType=RAC

RACOneNodeServiceName=

policyManaged=

createServerPool=

serverPoolName=

cardinality=

force=

pqPoolName=

pqCardinality=

createAsContainerDatabase=TRUE

numberOfPDBs=0

pdbName=

useLocalUndoForPDBs=TRUE

pdbAdminPassword=zzhzj321zg*A

nodelist=sgpbnk01kf,sgpbnk02kf

templateName=General_Purpose.dbc

sysPassword=zzhzj321zg*A

systemPassword=zzhzj321zg*A

oracleHomeUserPassword=zzhzj321zg*A

emConfiguration=

emExpressPort=5500

runCVUChecks=

dbsnmpPassword=

omsHost=

omsPort=

emUser=

emPassword=

dvConfiguration=

dvUserName=

dvUserPassword=

dvAccountManagerName=

dvAccountManagerPassword=

olsConfiguration=

datafileJarLocation=

datafileDestination=+DATADG

recoveryAreaDestination=+FRADG

storageType=ASM

diskGroupName=DATADG

asmsnmpPassword=zzhzj321zg*A

recoveryGroupName=FRADG

characterSet=

nationalCharacterSet=

registerWithDirService=

dirServiceUserName=

dirServicePassword=

walletPassword=

listeners=

variablesFile=

variables=

initParams=

sampleSchema=FALSE

memoryPercentage=

databaseType=

automaticMemoryManagement=

totalMemory=

静默建库

dbca -silent -ignorePrereqFailure -createDatabase -responseFile /home/oracle/dbca.rsp

[oracle@ora19node1 ~]<20210125 17:16:13>$ dbca -silent -ignorePrereqFailure -createDatabase -responseFile /home/oracle/dbca.rsp

Prepare for db operation

8% complete

Copying database files

33% complete

Creating and starting Oracle instance

34% complete

35% complete

39% complete

44% complete

47% complete

50% complete

Creating cluster database views

52% complete

67% complete

Completing Database Creation

71% complete

73% complete

75% complete

Executing Post Configuration Actions

100% complete

Database creation complete. For details check the logfiles at:

/data/dbbase/cfgtoollogs/dbca/ORA19RAC.

Database Information:

Global Database Name:ORA19RAC

System Identifier(SID) Prefix:ORA19RAC

Look at the log file "/data/dbbase/cfgtoollogs/dbca/ORA19RAC/ORA19RAC.log" for further details.

总结

尽管按照上面的操作：GI---> DB--->DBCA建库一整套流程最终操作成功。但是笔者在此要提醒读者：

1. ORACLE RAC 涉及到知识点非常丰富，实验前建议阅读关于RAC 的架构和原理的文献；

2. 尽管看着笔者操作顺利，但中间其实有一些小小的错误，被笔者忽略了，请读者认真思索每一个步骤；

3. 由于笔者安装过太多次，文献总结不完善，本文中的相关目录的设计可能有些错误，请读者自行规划、修正；总体磁盘规划为：

GI ： gridbase、gridhome、gridInventory

DB ： dbbase、dbhome、dbInventory

4. 附图几张：

GI安装

root脚本执行

查看磁盘组：