How to install Oracle RAC 12.2.0.1 in silent mode on Red Hat Linux 7.3 with VirtualBox – Part 2

Both posts document how to install a 2-node Oracle RAC cluster with Oracle 12.2.0.1 on Red Hat Linux 7.3 (RH7) with VirtualBox (VBOX) 5.0.28 on Centos 7.3.

Part 1 details Red Hat Linux installation and configuration and part 2 details Oracle software installation (Grid Infrastructure (GI), Oracle Database) and database creation.

In this article GIMR stands for Grid Infrastructure Management Repository.

Disclaimer: this is only an example that can be used to setup a lab or test system: it is not designed to be used for production purpose.

Grid Infrastructure silent installation

I have created ORACLE_BASE and ORACLE_HOME target directories on both nodes:

# mkdir -p /u01/base
# chown oracle:dba /u01/base
# mkdir /u01/gi12201
# chown oracle:oinstall /u01/gi12201
# chmod -R 775 /u01

I have created /etc/oraInst.loc to define Oracle Inventory location on both nodes:

# cat /etc/oraInst.loc
inventory_loc=/u01/orainv
inst_group=oinstall
# mkdir  /u01/orainv
# chown oracle:oinstall /u01/orainv

I have unzipped GI media only on first cluster node:

$ pwd
/u01/gi12201
$ unzip -q /stage/linuxx64_12201_grid_home.zip

I have created following script to run OUI in silent mode:

set -x
/u01/gi12201/gridSetup.sh \
-silent \
-ignorePrereqFailure \
-responseFile /u01/gi12201/install/response/gridsetup.rsp \
INVENTORY_LOCATION=/u01/orainv \
oracle.install.option=CRS_CONFIG \
ORACLE_BASE=/u01/base \
oracle.install.asm.OSDBA=dba \
oracle.install.asm.OSOPER=dba \
oracle.install.asm.OSASM=dba \
oracle.install.crs.config.gpnp.scanName=rh7ttc-scan \
oracle.install.crs.config.gpnp.scanPort=1521 \
oracle.install.crs.config.ClusterConfiguration=STANDALONE \
oracle.install.crs.config.gpnp.configureGNS=false \
oracle.install.crs.config.clusterName=rh7ttc \
oracle.install.crs.config.clusterNodes=rh7ttcn1:rh7ttcn1-vip:HUB,rh7ttcn2:rh7ttcn2-vip:HUB \
oracle.install.crs.config.networkInterfaceList=enp0s8:192.168.56.0:1,enp0s9:192.168.43.0:5 \
oracle.install.crs.config.gpnp.configureGNS=false \
oracle.install.asm.storageOption=ASM \
oracle.install.asm.configureGIMRDataDG=false \
oracle.install.asm.SYSASMPassword=oracle \
oracle.install.asm.diskGroup.name=OVM \
oracle.install.asm.diskGroup.redundancy=EXTERNAL \
oracle.install.asm.diskGroup.disks=/dev/asm-disk2 \
oracle.install.asm.diskGroup.diskDiscoveryString=/dev/asm* \
oracle.install.asm.monitorPassword=oracle \
-waitForCompletion                          

The above script is using following variables defined in ./install/response/gridsetup.rsp response file:

parameter name description
INVENTORY_LOCATION Directory where Oracle Inventory will be stored. Must match /etc/oraInst.loc contents.
oracle.install.option Must be set to CRS_CONFIG for Grid Infrastructure for cluster.
oracle.install.crs.config.gpnp.scanName Must be set to DNS alias for the 3 SCAN listener adresses.
oracle.install.crs.config.gpnp.scanPort Must be to set to TCP port to be used by SCAN listener.
oracle.install.crs.config.ClusterConfiguration Must be set to STANDALONE (no Domain Cluster)
oracle.install.crs.config.clusterName No dependency for this name.
oracle.install.crs.config.gpnp.configureGNS Must be set to FALSE to not use Grid Naming Service (GNS).
oracle.install.crs.config.clusterNodes Must be set to public-hostname:vip-hostname:HUB comma separated host name list (for STANDALONE cluster type).
oracle.install.crs.config.networkInterfaceList Must be set to network-interface:network-interface-subnet:network-interface_type comma separated strings
where network-interface_type must be set to 1 for public interface and to 5 for private interface (interconnect).
oracle.install.crs.config.storageOption Set to ASM for OCR and voting disk.
oracle.install.crs.config.useIPMI Must be set to FALSE to not use Intelligent Power Management Interface.
oracle.install.asm.SYSASMPassword Must be set to SYS password for ASM instance.
oracle.install.asm.diskGroup.diskDiscoveryString Used to set ASM instance ASM_DISKSTRING parameter.
oracle.install.asm.diskGroup.name Name of ASM disk group used for OCR, voting disk and GIMR database.
oracle.install.asm.configureGIMRDataDG must be set to false to not use a separate disk for GIMR database
oracle.install.asm.diskGroup.disks Comma separated ASM disk names list to be used for above ASM disk group.
oracle.install.asm.diskGroup.redundancy Disk group redundancy used by above ASM disk group.
oracle.install.asm.monitorPassword Must be set to ASMSNMP password for ASM instance.

Note that I have set oracle.install.asm.configureGIMRDataDG to false because I have not been able to use a separate disk
for GIMR. In my environment I have tried to set:

oracle.install.asm.configureGIMRDataDG=true
oracle.install.asm.gimrDG.name=MGMT
oracle.install.asm.gimrDG.redundancy=EXTERNAL
oracle.install.asm.gimrDG.disks=/dev/asm-disk2

But this has always failed in root.sh steps on first node with:

CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
PROTL-4: Failed to retrieve data from the local registry
CLSRSC-169: Failed to create or upgrade OLR
Died at /u01/gi12201/crs/install/oraolr.pm line 495.
The command '/u01/gi12201/perl/bin/perl -I/u01/gi12201/perl/lib -I/u01/gi12201/crs/install /u01/gi12201/crs/install/rootcrs.pl ' execution failed

Running OUI script for GI has given following output:

Launching Oracle Grid Infrastructure Setup Wizard...

[WARNING] [INS-30011] The SYS password entered does not conform to the Oracle recommended standards.
   CAUSE: Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9].
   ACTION: Provide a password that conforms to the Oracle recommended standards.
[WARNING] [INS-30011] The ASMSNMP password entered does not conform to the Oracle recommended standards.
   CAUSE: Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9].
   ACTION: Provide a password that conforms to the Oracle recommended standards.
[WARNING] [INS-41808] Possible invalid choice for OSASM Group.
   CAUSE: The name of the group you selected for the OSASM group is commonly used to grant other system privileges (For example: asmdba, asmoper, dba, oper).
   ACTION: Oracle recommends that you designate asmadmin as the OSASM group.
[WARNING] [INS-41809] Possible invalid choice for OSDBA Group.
   CAUSE: The group name you selected as the OSDBA for ASM group is commonly used for Oracle Database administrator privileges.
   ACTION: Oracle recommends that you designate asmdba as the OSDBA for ASM group, and that the group should not be the same group as an Oracle Database OSDBA group.
[WARNING] [INS-41810] Possible invalid choice for OSOPER Group.
   CAUSE: The group name you selected as the OSOPER for ASM group is commonly used for Oracle Database administrator privileges.
   ACTION: Oracle recommends that you designate asmoper as the OSOPER for ASM group, and that the group should not be the same group as an Oracle Database OSOPER group.
[WARNING] [INS-41813] OSDBA for ASM, OSOPER for ASM, and OSASM are the same OS group.
   CAUSE: The group you selected for granting the OSDBA for ASM group for database access, and the OSOPER for ASM group for startup and shutdown of Oracle ASM, is the same group as the OSASM group, whose members have SYSASM privileges on Oracle ASM.
   ACTION: Choose different groups as the OSASM, OSDBA for ASM, and OSOPER for ASM groups.
[WARNING] [INS-13013] Target environment does not meet some mandatory requirements.
   CAUSE: Some of the mandatory prerequisites are not met. See logs for details. /u01/orainv/logs/GridSetupActions2017-03-22_07-46-05PM/gridSetupActions2017-03-22_07-46-05PM.log
   ACTION: Identify the list of failed prerequisite checks from the log: /u01/orainv/logs/GridSetupActions2017-03-22_07-46-05PM/gridSetupActions2017-03-22_07-46-05PM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually.
You can find the log of this install session at:
 /u01/orainv/logs/GridSetupActions2017-03-22_07-46-05PM/gridSetupActions2017-03-22_07-46-05PM.log

As a root user, execute the following script(s):
	1. /u01/gi12201/root.sh

Execute /u01/gi12201/root.sh on the following nodes: 
[rh7ttcn1, rh7ttcn2]

Run the script on the local node first. After successful completion, you can start the script in parallel on all other nodes.

Successfully Setup Software.
As install user, execute the following command to complete the configuration.
	/u01/gi12201/gridSetup.sh -executeConfigTools -responseFile /u01/gi12201/install/response/gridsetup.rsp [-silent]

I have ignore following failed prerequisites with -ignorePrereqFailure switch:

INFO:  [Mar 22, 2017 7:48:22 PM] ------------------List of failed Tasks------------------
INFO:  [Mar 22, 2017 7:48:22 PM] *********************************************
INFO:  [Mar 22, 2017 7:48:22 PM] Package: gcc-c++-4.8.2: This is a prerequisite condition to test whether the package "gcc-c++-4.8.2" is available on the system.
INFO:  [Mar 22, 2017 7:48:22 PM] Severity:IGNORABLE
INFO:  [Mar 22, 2017 7:48:22 PM] OverallStatus:VERIFICATION_FAILED
INFO:  [Mar 22, 2017 7:48:22 PM] *********************************************
INFO:  [Mar 22, 2017 7:48:22 PM] Domain Sockets: This task verifies Clusterware domain sockets
INFO:  [Mar 22, 2017 7:48:22 PM] Severity:IGNORABLE
INFO:  [Mar 22, 2017 7:48:22 PM] OverallStatus:VERIFICATION_FAILED
INFO:  [Mar 22, 2017 7:48:22 PM] *********************************************
INFO:  [Mar 22, 2017 7:48:22 PM] /dev/shm mounted as temporary file system: Checks whether /dev/shm is mounted correctly as temporary file system
INFO:  [Mar 22, 2017 7:48:22 PM] Severity:IGNORABLE
INFO:  [Mar 22, 2017 7:48:22 PM] OverallStatus:VERIFICATION_FAILED
INFO:  [Mar 22, 2017 7:48:22 PM] *********************************************
INFO:  [Mar 22, 2017 7:48:22 PM] I/O scheduler: This task checks the I/O scheduler parameter configured
INFO:  [Mar 22, 2017 7:48:22 PM] Severity:IGNORABLE
INFO:  [Mar 22, 2017 7:48:22 PM] OverallStatus:WARNING
INFO:  [Mar 22, 2017 7:48:22 PM] *********************************************
INFO:  [Mar 22, 2017 7:48:22 PM] resolv.conf Integrity: resolv.conf Integrity
INFO:  [Mar 22, 2017 7:48:22 PM] Severity:CRITICAL
INFO:  [Mar 22, 2017 7:48:22 PM] OverallStatus:VERIFICATION_FAILED
INFO:  [Mar 22, 2017 7:48:22 PM] *********************************************
INFO:  [Mar 22, 2017 7:48:22 PM] (Linux) resolv.conf Integrity: This task checks consistency of file /etc/resolv.conf file across nodes
INFO:  [Mar 22, 2017 7:48:22 PM] Severity:CRITICAL
INFO:  [Mar 22, 2017 7:48:22 PM] OverallStatus:VERIFICATION_FAILED
INFO:  [Mar 22, 2017 7:48:22 PM] -----------------End of failed Tasks List----------------

On first cluster node I have run root.sh:

# /u01/gi12201/root.sh
Check /u01/gi12201/install/root_rh7ttcn1.localdomain_2017-03-22_19-56-08-393599429.log for the output of root script

Corresponding log is:

cat  /u01/gi12201/install/root_rh7ttcn1.localdomain_2017-03-22_19-56-08-393599429.log
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/gi12201
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/gi12201/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/base/crsdata/rh7ttcn1/crsconfig/rootcrs_rh7ttcn1_2017-03-22_07-56-17PM.log
2017/03/22 19:56:21 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2017/03/22 19:56:21 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2017/03/22 19:56:51 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2017/03/22 19:56:51 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2017/03/22 19:56:56 CLSRSC-363: User ignored prerequisites during installation
2017/03/22 19:56:56 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2017/03/22 19:56:57 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2017/03/22 19:56:59 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2017/03/22 19:57:05 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2017/03/22 19:57:06 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2017/03/22 19:57:06 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2017/03/22 19:57:36 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2017/03/22 19:57:44 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2017/03/22 19:57:44 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2017/03/22 19:57:50 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2017/03/22 19:58:06 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2017/03/22 19:58:25 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2017/03/22 19:58:31 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn1'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2017/03/22 19:58:52 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2017/03/22 19:58:58 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn1'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.evmd' on 'rh7ttcn1'
CRS-2672: Attempting to start 'ora.mdnsd' on 'rh7ttcn1'
CRS-2676: Start of 'ora.mdnsd' on 'rh7ttcn1' succeeded
CRS-2676: Start of 'ora.evmd' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rh7ttcn1'
CRS-2676: Start of 'ora.gpnpd' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rh7ttcn1'
CRS-2672: Attempting to start 'ora.gipcd' on 'rh7ttcn1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rh7ttcn1' succeeded
CRS-2676: Start of 'ora.gipcd' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rh7ttcn1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rh7ttcn1'
CRS-2676: Start of 'ora.diskmon' on 'rh7ttcn1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rh7ttcn1' succeeded

Disk groups created successfully. Check /u01/base/cfgtoollogs/asmca/asmca-170322PM075939.log for details.


2017/03/22 20:01:12 CLSRSC-482: Running command: '/u01/gi12201/bin/ocrconfig -upgrade oracle oinstall'
CRS-2672: Attempting to start 'ora.crf' on 'rh7ttcn1'
CRS-2672: Attempting to start 'ora.storage' on 'rh7ttcn1'
CRS-2676: Start of 'ora.storage' on 'rh7ttcn1' succeeded
CRS-2676: Start of 'ora.crf' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rh7ttcn1'
CRS-2676: Start of 'ora.crsd' on 'rh7ttcn1' succeeded
CRS-4256: Updating the profile
Successful addition of voting disk d61c25e4f4ec4f0fbfd283e630fba8a4.
Successfully replaced voting disk group with +OVM.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   d61c25e4f4ec4f0fbfd283e630fba8a4 (/dev/asm-disk2) [OVM]
Located 1 voting disk(s).
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn1'
CRS-2673: Attempting to stop 'ora.crsd' on 'rh7ttcn1'
CRS-2677: Stop of 'ora.crsd' on 'rh7ttcn1' succeeded
CRS-2673: Attempting to stop 'ora.storage' on 'rh7ttcn1'
CRS-2673: Attempting to stop 'ora.crf' on 'rh7ttcn1'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rh7ttcn1'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rh7ttcn1'
CRS-2677: Stop of 'ora.crf' on 'rh7ttcn1' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'rh7ttcn1' succeeded
CRS-2677: Stop of 'ora.storage' on 'rh7ttcn1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rh7ttcn1'
CRS-2677: Stop of 'ora.mdnsd' on 'rh7ttcn1' succeeded
CRS-2677: Stop of 'ora.asm' on 'rh7ttcn1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rh7ttcn1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rh7ttcn1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rh7ttcn1'
CRS-2673: Attempting to stop 'ora.evmd' on 'rh7ttcn1'
CRS-2677: Stop of 'ora.evmd' on 'rh7ttcn1' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'rh7ttcn1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rh7ttcn1'
CRS-2677: Stop of 'ora.cssd' on 'rh7ttcn1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rh7ttcn1'
CRS-2677: Stop of 'ora.gipcd' on 'rh7ttcn1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2017/03/22 20:02:26 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'rh7ttcn1'
CRS-2672: Attempting to start 'ora.evmd' on 'rh7ttcn1'
CRS-2676: Start of 'ora.mdnsd' on 'rh7ttcn1' succeeded
CRS-2676: Start of 'ora.evmd' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rh7ttcn1'
CRS-2676: Start of 'ora.gpnpd' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rh7ttcn1'
CRS-2676: Start of 'ora.gipcd' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rh7ttcn1'
CRS-2676: Start of 'ora.cssdmonitor' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rh7ttcn1'
CRS-2672: Attempting to start 'ora.diskmon' on 'rh7ttcn1'
CRS-2676: Start of 'ora.diskmon' on 'rh7ttcn1' succeeded
CRS-2676: Start of 'ora.cssd' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rh7ttcn1'
CRS-2672: Attempting to start 'ora.ctssd' on 'rh7ttcn1'
CRS-2676: Start of 'ora.ctssd' on 'rh7ttcn1' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rh7ttcn1'
CRS-2676: Start of 'ora.asm' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'rh7ttcn1'
CRS-2676: Start of 'ora.storage' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'rh7ttcn1'
CRS-2676: Start of 'ora.crf' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rh7ttcn1'
CRS-2676: Start of 'ora.crsd' on 'rh7ttcn1' succeeded
CRS-6023: Starting Oracle Cluster Ready Services-managed resources
CRS-6017: Processing resource auto-start for servers: rh7ttcn1
CRS-6016: Resource auto-start has completed for server rh7ttcn1
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2017/03/22 20:04:36 CLSRSC-343: Successfully started Oracle Clusterware stack
2017/03/22 20:04:36 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'rh7ttcn1'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rh7ttcn1'
CRS-2676: Start of 'ora.asm' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.OVM.dg' on 'rh7ttcn1'
CRS-2676: Start of 'ora.OVM.dg' on 'rh7ttcn1' succeeded
2017/03/22 20:06:46 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2017/03/22 20:07:19 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

I have run root.sh on second cluster node:

# /u01/gi12201/root.sh
Check /u01/gi12201/install/root_rh7ttcn2.localdomain_2017-03-22_20-09-10-533930444.log for the output of root script

Corresponding log is:

# cat /u01/gi12201/install/root_rh7ttcn2.localdomain_2017-03-22_20-09-10-533930444.log
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/gi12201
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/gi12201/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/base/crsdata/rh7ttcn2/crsconfig/rootcrs_rh7ttcn2_2017-03-22_08-09-22PM.log
2017/03/22 20:09:26 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2017/03/22 20:09:27 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.
2017/03/22 20:09:54 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2017/03/22 20:09:55 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2017/03/22 20:09:57 CLSRSC-363: User ignored prerequisites during installation
2017/03/22 20:09:57 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2017/03/22 20:09:58 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2017/03/22 20:09:58 CLSRSC-594: Executing installation step 5 of 19: 'SaveParamFile'.
2017/03/22 20:10:01 CLSRSC-594: Executing installation step 6 of 19: 'SetupOSD'.
2017/03/22 20:10:05 CLSRSC-594: Executing installation step 7 of 19: 'CheckCRSConfig'.
2017/03/22 20:10:06 CLSRSC-594: Executing installation step 8 of 19: 'SetupLocalGPNP'.
2017/03/22 20:10:08 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2017/03/22 20:10:09 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2017/03/22 20:10:09 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2017/03/22 20:10:11 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2017/03/22 20:10:26 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2017/03/22 20:10:41 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2017/03/22 20:10:42 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
2017/03/22 20:10:59 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2017/03/22 20:11:00 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
CRS-4123: Oracle High Availability Services has been started.
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn2'
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rh7ttcn2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
2017/03/22 20:11:18 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
CRS-4123: Starting Oracle High Availability Services-managed resources
CRS-2672: Attempting to start 'ora.mdnsd' on 'rh7ttcn2'
CRS-2672: Attempting to start 'ora.evmd' on 'rh7ttcn2'
CRS-2676: Start of 'ora.evmd' on 'rh7ttcn2' succeeded
CRS-2676: Start of 'ora.mdnsd' on 'rh7ttcn2' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'rh7ttcn2'
CRS-2676: Start of 'ora.gpnpd' on 'rh7ttcn2' succeeded
CRS-2672: Attempting to start 'ora.gipcd' on 'rh7ttcn2'
CRS-2676: Start of 'ora.gipcd' on 'rh7ttcn2' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'rh7ttcn2'
CRS-2676: Start of 'ora.cssdmonitor' on 'rh7ttcn2' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'rh7ttcn2'
CRS-2672: Attempting to start 'ora.diskmon' on 'rh7ttcn2'
CRS-2676: Start of 'ora.diskmon' on 'rh7ttcn2' succeeded
CRS-2676: Start of 'ora.cssd' on 'rh7ttcn2' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'rh7ttcn2'
CRS-2672: Attempting to start 'ora.ctssd' on 'rh7ttcn2'
CRS-2676: Start of 'ora.ctssd' on 'rh7ttcn2' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'rh7ttcn2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rh7ttcn2'
CRS-2676: Start of 'ora.asm' on 'rh7ttcn2' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'rh7ttcn2'
CRS-2676: Start of 'ora.storage' on 'rh7ttcn2' succeeded
CRS-2672: Attempting to start 'ora.crf' on 'rh7ttcn2'
CRS-2676: Start of 'ora.crf' on 'rh7ttcn2' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'rh7ttcn2'
CRS-2676: Start of 'ora.crsd' on 'rh7ttcn2' succeeded
CRS-6017: Processing resource auto-start for servers: rh7ttcn2
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'rh7ttcn1'
CRS-2672: Attempting to start 'ora.net1.network' on 'rh7ttcn2'
CRS-2672: Attempting to start 'ora.ASMNET1LSNR_ASM.lsnr' on 'rh7ttcn2'
CRS-2677: Stop of 'ora.LISTENER_SCAN1.lsnr' on 'rh7ttcn1' succeeded
CRS-2676: Start of 'ora.net1.network' on 'rh7ttcn2' succeeded
CRS-2673: Attempting to stop 'ora.scan1.vip' on 'rh7ttcn1'
CRS-2672: Attempting to start 'ora.ons' on 'rh7ttcn2'
CRS-2677: Stop of 'ora.scan1.vip' on 'rh7ttcn1' succeeded
CRS-2672: Attempting to start 'ora.scan1.vip' on 'rh7ttcn2'
CRS-2676: Start of 'ora.scan1.vip' on 'rh7ttcn2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'rh7ttcn2'
CRS-2676: Start of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rh7ttcn2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'rh7ttcn2'
CRS-2676: Start of 'ora.ons' on 'rh7ttcn2' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'rh7ttcn2' succeeded
CRS-2676: Start of 'ora.asm' on 'rh7ttcn2' succeeded
CRS-6016: Resource auto-start has completed for server rh7ttcn2
CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources
CRS-4123: Oracle High Availability Services has been started.
2017/03/22 20:13:04 CLSRSC-343: Successfully started Oracle Clusterware stack
2017/03/22 20:13:04 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2017/03/22 20:13:19 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2017/03/22 20:13:28 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

I have run the last GI configuration script with oracle user account:

/u01/gi12201/gridSetup.sh -executeConfigTools \
-responseFile /u01/gi12201/install/response/gridsetup.rsp \
-silent \
INVENTORY_LOCATION=/u01/orainv \
oracle.install.option=CRS_CONFIG \
ORACLE_BASE=/u01/base \
oracle.install.asm.OSDBA=dba \
oracle.install.asm.OSOPER=dba \
oracle.install.asm.OSASM=dba \
oracle.install.crs.config.gpnp.scanName=rh7ttc-scan \
oracle.install.crs.config.gpnp.scanPort=1521 \
oracle.install.crs.config.ClusterConfiguration=STANDALONE \
oracle.install.crs.config.gpnp.configureGNS=false \
oracle.install.crs.config.clusterName=rh7ttc \
oracle.install.crs.config.clusterNodes=rh7ttcn1:rh7ttcn1-vip:HUB,rh7ttcn2:rh7ttcn2-vip:HUB \
oracle.install.crs.config.networkInterfaceList=enp0s8:192.168.56.0:1,enp0s9:192.168.43.0:5 \
oracle.install.crs.config.gpnp.configureGNS=false \
oracle.install.asm.storageOption=ASM \
oracle.install.asm.configureGIMRDataDG=false \
oracle.install.asm.SYSASMPassword=oracle \
oracle.install.asm.diskGroup.name=OVM \
oracle.install.asm.diskGroup.redundancy=EXTERNAL \
oracle.install.asm.diskGroup.disks=/dev/asm-disk2 \
oracle.install.asm.diskGroup.diskDiscoveryString=/dev/asm* \
oracle.install.asm.monitorPassword=oracle \
-waitForCompletion

Output is:

Launching Oracle Grid Infrastructure Setup Wizard...

You can find the logs of this session at:
/u01/orainv/logs/GridSetupActions2017-03-22_08-24-45PM

Successfully Configured Software.

I have checked current cluster resources:

$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.OVM.dg
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.chad
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.net1.network
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.ons
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       rh7ttcn1                 169.254.253.229 192.
                                                             168.43.141,STABLE
ora.asm
      1        ONLINE  ONLINE       rh7ttcn1                 Started,STABLE
      2        ONLINE  ONLINE       rh7ttcn2                 Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       rh7ttcn1                 Open,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.rh7ttcn1.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.rh7ttcn2.vip
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
--------------------------------------------------------------------------------

This step has taken is quite long (in my environment it has taken about 30 minutes).

I have also checked OCR and voting disk with:

$ asmcmd lsdg
State    Type    Rebal  Sector  Logical_Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512             512   4096  1048576     40959     7081                0            7081              0             Y  OVM/
$ ocrcheck
Status of Oracle Cluster Registry is as follows :
	 Version                  :          4
	 Total space (kbytes)     :     409568
	 Used space (kbytes)      :       2096
	 Available space (kbytes) :     407472
	 ID                       : 1666908690
	 Device/File Name         :       +OVM
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

	 Cluster registry integrity check succeeded

	 Logical corruption check bypassed due to non-privileged user

$ crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   d61c25e4f4ec4f0fbfd283e630fba8a4 (/dev/asm-disk2) [OVM]
Located 1 voting disk(s).

I have checked GIMR configuration database with:

$ srvctl config mgmtdb
Database unique name: _mgmtdb
Database name: 
Oracle home: 
Oracle user: oracle
Spfile: +OVM/_MGMTDB/PARAMETERFILE/spfile.270.939328471
Password file: 
Domain: 
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Type: Management
PDB name: GIMR_DSCREP_10
PDB service: GIMR_DSCREP_10
Cluster name: rh7ttc
Database instance: -MGMTDB

Oracle Database silent installation

I have unzipped the media file:

$ unzip linuxx64_12201_database.zip 

I have created Oracle Home directory on both nodes:

# mkdir /u01/db12201
# chown oracle:dba /u01/db12201

On first node I have run following script:

cd database
export DISTRIB=`pwd`
./runInstaller -silent \
-responseFile $DISTRIB/response/db_install.rsp      \
oracle.install.option=INSTALL_DB_SWONLY             \
oracle.install.db.CLUSTER_NODES=rh7ttcn1,rh7ttcn2   \
UNIX_GROUP_NAME=oinstall                            \
SELECTED_LANGUAGES=en                               \
ORACLE_HOME=/u01/db12201                            \
ORACLE_BASE=/u01/base                               \
oracle.install.db.InstallEdition=EE                 \
oracle.install.db.isCustomInstall=false             \
oracle.install.db.OSDBA_GROUP=dba                   \
oracle.install.db.OSBACKUPDBA_GROUP=dba             \
oracle.install.db.OSDGDBA_GROUP=dba                 \
oracle.install.db.OSKMDBA_GROUP=dba                 \
oracle.install.db.OSRACDBA_GROUP=dba                \
SECURITY_UPDATES_VIA_MYORACLESUPPORT=false          \
DECLINE_SECURITY_UPDATES=true                       \
-waitForCompletion

I have ignored following failed prerequisites:

INFO: ------------------List of failed Tasks------------------
INFO: *********************************************
INFO: Package: gcc-c++-4.8.2: This is a prerequisite condition to test whether the package "gcc-c++-4.8.2" is available on the system.
INFO: Severity:IGNORABLE
INFO: OverallStatus:VERIFICATION_FAILED
INFO: *********************************************
INFO: /dev/shm mounted as temporary file system: Checks whether /dev/shm is mounted correctly as temporary file system
INFO: Severity:IGNORABLE
INFO: OverallStatus:VERIFICATION_FAILED
INFO: *********************************************
INFO: resolv.conf Integrity: resolv.conf Integrity
INFO: Severity:CRITICAL
INFO: OverallStatus:VERIFICATION_FAILED
INFO: *********************************************
INFO: (Linux) resolv.conf Integrity: This task checks consistency of file /etc/resolv.conf file across nodes
INFO: Severity:CRITICAL
INFO: OverallStatus:VERIFICATION_FAILED
INFO: -----------------End of failed Tasks List----------------

Running this script has produced following output:

Starting Oracle Universal Installer...

Checking Temp space: must be greater than 500 MB.   Actual 12001 MB    Passed
Checking swap space: must be greater than 150 MB.   Actual 8176 MB    Passed
Preparing to launch Oracle Universal Installer from /tmp/OraInstall2017-03-23_06-35-19PM. Please wait ...[WARNING] [INS-32018] The selected Oracle home is outside of Oracle base.
   ACTION: Oracle recommends installing Oracle software within the Oracle base directory. Adjust the Oracle home or Oracle base accordingly.
[WARNING] [INS-13013] Target environment does not meet some mandatory requirements.
   CAUSE: Some of the mandatory prerequisites are not met. See logs for details. /u01/orainv/logs/installActions2017-03-23_06-35-19PM.log
   ACTION: Identify the list of failed prerequisite checks from the log: /u01/orainv/logs/installActions2017-03-23_06-35-19PM.log. Then either from the log file or from installation manual find the appropriate configuration to meet the prerequisites and fix it manually.
You can find the log of this install session at:
 /u01/orainv/logs/installActions2017-03-23_06-35-19PM.log
The installation of Oracle Database 12c was successful.
Please check '/u01/orainv/logs/silentInstall2017-03-23_06-35-19PM.log' for more details.
The Cluster Node Addition of /u01/db12201 was successful.
Please check '/u01/orainv/logs/silentInstall2017-03-23_06-35-19PM.log' for more details.

As a root user, execute the following script(s):
	1. /u01/db12201/root.sh

Execute /u01/db12201/root.sh on the following nodes: 
[rh7ttcn1, rh7ttcn2]


Successfully Setup Software.

I have run root.sh on first node …

# /u01/db12201/root.sh
Check /u01/db12201/install/root_rh7ttcn1.localdomain_2017-03-23_18-51-55-954652456.log for the output of root script
# cat /u01/db12201/install/root_rh7ttcn2.localdomain_2017-03-23_18-52-27-511950573.log
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/db12201
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.

… and on second node:

# /u01/db12201/root.sh
Check /u01/db12201/install/root_rh7ttcn2.localdomain_2017-03-23_18-52-27-511950573.log for the output of root script
[root@rh7ttcn2 ~]# cat /u01/db12201/install/root_rh7ttcn2.localdomain_2017-03-23_18-52-27-511950573.log
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/db12201
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.

I have run opatch on first node:

$ export ORACLE_HOME=/u01/db12201
$ $ORACLE_HOME/OPatch/opatch lsinv
Oracle Interim Patch Installer version 12.2.0.1.6
Copyright (c) 2017, Oracle Corporation.  All rights reserved.


Oracle Home       : /u01/db12201
Central Inventory : /u01/orainv
   from           : /u01/db12201/oraInst.loc
OPatch version    : 12.2.0.1.6
OUI version       : 12.2.0.1.4
Log file location : /u01/db12201/cfgtoollogs/opatch/opatch2017-03-23_18-53-43PM_1.log

Lsinventory Output file location : /u01/db12201/cfgtoollogs/opatch/lsinv/lsinventory2017-03-23_18-53-43PM.txt

--------------------------------------------------------------------------------
Local Machine Information::
Hostname: rh7ttcn1
ARU platform id: 226
ARU platform description:: Linux x86-64

Installed Top-level Products (1): 

Oracle Database 12c                                                  12.2.0.1.0
There are 1 products installed in this Oracle Home.


There are no Interim patches installed in this Oracle Home.


--------------------------------------------------------------------------------

OPatch succeeded.

Fast Recovery Area (FRA) disk group creation

I have switched to oracle account to create FRA disk group that will be used for database creation.

First I have set current instance to ASM instance on first cluster node:

$ . oraenv
ORACLE_SID = [+ASM1] ? +ASM1
The Oracle base remains unchanged with value /u01/base
$ sqlplus / as sysasm

SQL*Plus: Release 12.2.0.1.0 Production on Thu Mar 23 18:54:45 2017

Copyright (c) 1982, 2016, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> create diskgroup FRA external redundancy disk '/dev/asm-disk4';

Diskgroup created.

SQL> 

I have checked that disk group resource ora.FRA.dg has been added to OCR:

$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.FRA.dg
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               OFFLINE OFFLINE      rh7ttcn2                 STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.OVM.dg
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.chad
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.net1.network
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.ons
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       rh7ttcn1                 169.254.253.229 192.
                                                             168.43.141,STABLE
ora.asm
      1        ONLINE  ONLINE       rh7ttcn1                 Started,STABLE
      2        ONLINE  ONLINE       rh7ttcn2                 Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       rh7ttcn1                 Open,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.rh7ttcn1.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.rh7ttcn2.vip
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
--------------------------------------------------------------------------------

I have started disk group on cluster node 2:

$ srvctl start diskgroup -diskgroup FRA
$ srvctl status diskgroup -diskgroup FRA
Disk Group FRA is running on rh7ttcn1,rh7ttcn2

Database files disk group creation

I have run similar steps for last ASM disk group.

$ sqlplus / as sysasm

SQL*Plus: Release 12.2.0.1.0 Production on Thu Mar 23 18:56:50 2017

Copyright (c) 1982, 2016, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> create diskgroup DATA external redundancy disk '/dev/asm-disk3';

Diskgroup created.

SQL> Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

$ srvctl start diskgroup -diskgroup DATA
$ srvctl status diskgroup -diskgroup DATA
Disk Group DATA is running on rh7ttcn1,rh7ttcn2
$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.DATA.dg
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.FRA.dg
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.OVM.dg
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.chad
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.net1.network
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.ons
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       rh7ttcn1                 169.254.253.229 192.
                                                             168.43.141,STABLE
ora.asm
      1        ONLINE  ONLINE       rh7ttcn1                 Started,STABLE
      2        ONLINE  ONLINE       rh7ttcn2                 Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cvu
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       rh7ttcn1                 Open,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.rh7ttcn1.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.rh7ttcn2.vip
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
--------------------------------------------------------------------------------
$ 

I have changed disk groups compatibility to make sure that I can create a 12.2.0.1 database and avoid following error:

[FATAL] [DBT-05802] Creating password file on diskgroup (DATA) would fail since it requires compatible.asm of version (12.1.0.0.0) or higher. 
Current compatible.asm version is '11.2.0.2.0'.
ACTION: Choose a diskgroup with correct value for compatible.asm attribute.

I have run a SYSASM connection to local ASM instance:

$ sqlplus / as sysasm

SQL*Plus: Release 12.2.0.1.0 Production on Thu Mar 23 19:04:43 2017

Copyright (c) 1982, 2016, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> alter diskgroup DATA set attribute 'compatible.asm'='12.1.0.0';

Diskgroup altered.

RAC database creation

I have created a RAC database with following script:

/u01/db12201/bin/dbca \
 -silent \
 -nodelist rh7ttcn1,rh7ttcn2 \
 -createDatabase \
 -templateName General_Purpose.dbc   \
 -gdbName CDBRAC \
 -createAsContainerDatabase true \
 -numberOfPdbs 1 \
 -pdbName pdb \
 -pdbadminUsername pdba \
 -pdbadminPassword oracle \
 -SysPassword oracle \
 -SystemPassword oracle \
 -emConfiguration NONE \
 -storageType ASM \
 -asmSysPassword oracle \
 -diskGroupName DATA \
 -characterSet AL32UTF8 \
 -totalMemory 1024 \
 -recoveryGroupName FRA

This script has printed following output:

[WARNING] [DBT-06208] The 'SYS' password entered does not conform to the Oracle recommended standards.
   CAUSE: 
a. Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9].
b.The password entered is a keyword that Oracle does not recommend to be used as password
   ACTION: Specify a strong password. If required refer Oracle documentation for guidelines.
[WARNING] [DBT-06208] The 'SYSTEM' password entered does not conform to the Oracle recommended standards.
   CAUSE: 
a. Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9].
b.The password entered is a keyword that Oracle does not recommend to be used as password
   ACTION: Specify a strong password. If required refer Oracle documentation for guidelines.
[WARNING] [DBT-06208] The 'PDBADMIN' password entered does not conform to the Oracle recommended standards.
   CAUSE: 
a. Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9].
b.The password entered is a keyword that Oracle does not recommend to be used as password
   ACTION: Specify a strong password. If required refer Oracle documentation for guidelines.
[WARNING] [DBT-09102] Target environment does not meet some optional requirements.
   CAUSE: Some of the optional prerequisites are not met. See logs for details. /u01/base/cfgtoollogs/dbca/trace.log_2017-03-23_07-05-46-PM
   ACTION: Find the appropriate configuration from the log file or from the installation guide to meet the prerequisites and fix this manually.
Copying database files
1% complete
11% complete
21% complete
Creating and starting Oracle instance
23% complete
25% complete
26% complete
27% complete
30% complete
33% complete
34% complete
35% complete
Creating cluster database views
37% complete
50% complete
Completing Database Creation
51% complete
53% complete
55% complete
57% complete
Creating Pluggable Databases
61% complete
78% complete
Executing Post Configuration Actions
100% complete
Look at the log file "/u01/base/cfgtoollogs/dbca/CDBRAC/CDBRAC.log" for further details.

I have ignored following warnings:

INFO: ------------------List of failed Tasks------------------


INFO: Mar 23, 2017 7:06:59 PM oracle.install.commons.base.prereq.PrereqCheckerJob logTaskOverallResult
INFO: *********************************************


INFO: Mar 23, 2017 7:06:59 PM oracle.install.commons.base.prereq.PrereqCheckerJob logTaskOverallResult
INFO: Package: gcc-c++-4.8.2: This is a prerequisite condition to test whether the package "gcc-c++-4.8.2" is available on the system.


INFO: Mar 23, 2017 7:06:59 PM oracle.install.commons.base.prereq.PrereqCheckerJob logTaskOverallResult
INFO: Severity:IGNORABLE


INFO: Mar 23, 2017 7:06:59 PM oracle.install.commons.base.prereq.PrereqCheckerJob logTaskOverallResult
INFO: OverallStatus:VERIFICATION_FAILED


INFO: Mar 23, 2017 7:06:59 PM oracle.install.commons.base.prereq.PrereqCheckerJob logTaskOverallResult
INFO: *********************************************


INFO: Mar 23, 2017 7:06:59 PM oracle.install.commons.base.prereq.PrereqCheckerJob logTaskOverallResult
INFO: /dev/shm mounted as temporary file system: Checks whether /dev/shm is mounted correctly as temporary file system


INFO: Mar 23, 2017 7:06:59 PM oracle.install.commons.base.prereq.PrereqCheckerJob logTaskOverallResult
INFO: Severity:IGNORABLE


INFO: Mar 23, 2017 7:06:59 PM oracle.install.commons.base.prereq.PrereqCheckerJob logTaskOverallResult
INFO: OverallStatus:VERIFICATION_FAILED


INFO: Mar 23, 2017 7:06:59 PM oracle.assistants.dbca.validator.PrereqValidator validate
INFO: -----------------End of failed Tasks List----------------

I have fixed /etc/oratab on node 1 to add instance name …:

$ tail -n 3 /etc/oratab
-MGMTDB:/u01/gi12201:N
CDBRAC1:/u01/db12201:N	
+ASM1:/u01/gi12201:N		# line added by Agent

… and on node 2:

$ tail  -n 3 /etc/oratab
#
+ASM2:/u01/gi12201:N		# line added by Agent
CDBRAC2:/u01/db12201:N	

I have connected to database instance and run some checks:

$ . oraenv
ORACLE_SID = [+ASM1] ? CDBRAC1
The Oracle base remains unchanged with value /u01/base
$ sqlplus / as sysdba

SQL*Plus: Release 12.2.0.1.0 Production on Thu Mar 23 19:25:03 2017

Copyright (c) 1982, 2016, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> select name, cdb from v$database;

NAME	  CDB
--------- ---
CDBRAC	  YES

SQL> show pdbs

    CON_ID CON_NAME			  OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
	 2 PDB$SEED			  READ ONLY  NO
	 3 PDB				  READ WRITE NO

SQL> select comp_name, version, status from dba_registry;

COMP_NAME			    VERSION		 STATUS
----------------------------------- -------------------- --------------------
Oracle Database Catalog Views	    12.2.0.1.0		 VALID
Oracle Database Packages and Types  12.2.0.1.0		 VALID
JServer JAVA Virtual Machine	    12.2.0.1.0		 VALID
Oracle XDK			    12.2.0.1.0		 VALID
Oracle Database Java Packages	    12.2.0.1.0		 VALID
OLAP Analytic Workspace 	    12.2.0.1.0		 VALID
Oracle Real Application Clusters    12.2.0.1.0		 VALID
Oracle XML Database		    12.2.0.1.0		 VALID
Oracle Workspace Manager	    12.2.0.1.0		 VALID
Oracle Text			    12.2.0.1.0		 VALID
Oracle Multimedia		    12.2.0.1.0		 VALID

COMP_NAME			    VERSION		 STATUS
----------------------------------- -------------------- --------------------
Spatial 			    12.2.0.1.0		 VALID
Oracle OLAP API 		    12.2.0.1.0		 VALID
Oracle Label Security		    12.2.0.1.0		 VALID
Oracle Database Vault		    12.2.0.1.0		 VALID

15 rows selected.

SQL> 

I have exited SQL*Plus and checked database configuration OCR:

$ srvctl config database -d CDBRAC
Database unique name: CDBRAC
Database name: CDBRAC
Oracle home: /u01/db12201
Oracle user: oracle
Spfile: +DATA/CDBRAC/PARAMETERFILE/spfile.272.939410123
Password file: +DATA/CDBRAC/PASSWORD/pwdcdbrac.256.939409649
Domain: 
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools: 
Disk Groups: FRA,DATA
Mount point paths: 
Services: 
Type: RAC
Start concurrency: 
Stop concurrency: 
OSDBA group: dba
OSOPER group: 
Database instances: CDBRAC1,CDBRAC2
Configured nodes: rh7ttcn1,rh7ttcn2
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services: 
Database is administrator managed

Note that password file is by default stored in database disk group.

I have also checked that crsctl displays now all resources including the new database:


$ crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details       
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.DATA.dg
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.FRA.dg
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.OVM.dg
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.chad
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.net1.network
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.ons
               ONLINE  ONLINE       rh7ttcn1                 STABLE
               ONLINE  ONLINE       rh7ttcn2                 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.MGMTLSNR
      1        ONLINE  ONLINE       rh7ttcn1                 169.254.253.229 192.
                                                             168.43.141,STABLE
ora.asm
      1        ONLINE  ONLINE       rh7ttcn1                 Started,STABLE
      2        ONLINE  ONLINE       rh7ttcn2                 Started,STABLE
      3        OFFLINE OFFLINE                               STABLE
ora.cdbrac.db
      1        ONLINE  ONLINE       rh7ttcn1                 Open,HOME=/u01/db122
                                                             01,STABLE
      2        ONLINE  ONLINE       rh7ttcn2                 Open,HOME=/u01/db122
                                                             01,STABLE
ora.cvu
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.mgmtdb
      1        ONLINE  ONLINE       rh7ttcn1                 Open,STABLE
ora.qosmserver
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.rh7ttcn1.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.rh7ttcn2.vip
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       rh7ttcn2                 STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       rh7ttcn1                 STABLE
--------------------------------------------------------------------------------

And I have rebooted both cluster node to check to each resource is started automatically: it has taken after about 7 minutes to have each resource (except ora.asm number 3) in ONLINE status.

Because it was not possible at installation time to separate OCR/Voting Disk files from GIMR database,
I have created a new ASM disk group named OCRVD for OCR and voting disk using /dev/asm-disk1. I have checked that /dev/asm-disk1 is currently unsued from ASM instance:

SQL> select name, path, mount_status, header_status, mode_status from v$asm_disk;

NAME			       PATH		    MOUNT_S HEADER_STATU MODE_ST
------------------------------ -------------------- ------- ------------ -------
			       /dev/asm-disk1	    CLOSED  CANDIDATE	 ONLINE
OVM_0000		       /dev/asm-disk2	    CACHED  MEMBER	 ONLINE
DATA_0000		       /dev/asm-disk3	    CACHED  MEMBER	 ONLINE
FRA_0000		       /dev/asm-disk4	    CACHED  MEMBER	 ONLINE

I have created a new diskgroup:

SQL> create diskgroup OCRVD external redundancy disk '/dev/asm-disk1';
Diskgroup created.
SQL> Disconnected from Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
$ asmcmd lsdg
State    Type    Rebal  Sector  Logical_Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512             512   4096  1048576     40959    36953                0           36953              0             N  DATA/
MOUNTED  EXTERN  N         512             512   4096  1048576     20479    19545                0           19545              0             N  FRA/
MOUNTED  EXTERN  N         512             512   4096  1048576      2047     1995                0            1995              0             N  OCRVD/
MOUNTED  EXTERN  N         512             512   4096  1048576     40959     7081                0            7081              0             Y  OVM/
$ srvctl start diskgroup -diskgroup OCRVD
$ srvctl status diskgroup -diskgroup OCRVD
Disk Group OCRVD is running on rh7ttcn1,rh7ttcn2

I have moved OCR to this new diskgroup using root account:

# export ORACLE_HOME=/u01/gi12201
# PATH=$ORACLE_HOME/bin:$PATH
# ocrcheck
Status of Oracle Cluster Registry is as follows :
	 Version                  :          4
	 Total space (kbytes)     :     409568
	 Used space (kbytes)      :       2192
	 Available space (kbytes) :     407376
	 ID                       : 1666908690
	 Device/File Name         :       +OVM
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

	 Cluster registry integrity check succeeded
# ocrconfig -add +OCRVD
# ocrcheck
Status of Oracle Cluster Registry is as follows :
	 Version                  :          4
	 Total space (kbytes)     :     409568
	 Used space (kbytes)      :       2192
	 Available space (kbytes) :     407376
	 ID                       : 1666908690
	 Device/File Name         :       +OVM
                                    Device/File integrity check succeeded
	 Device/File Name         :     +OCRVD
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

	 Cluster registry integrity check succeeded

	 Logical corruption check succeeded

# ocrconfig -delete +OVM
# ocrcheck
Status of Oracle Cluster Registry is as follows :
	 Version                  :          4
	 Total space (kbytes)     :     409568
	 Used space (kbytes)      :       2192
	 Available space (kbytes) :     407376
	 ID                       : 1666908690
	 Device/File Name         :     +OCRVD
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

	 Cluster registry integrity check succeeded

	 Logical corruption check succeeded

#

And I have moved voting disk to this new diskgroup with:

# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   d61c25e4f4ec4f0fbfd283e630fba8a4 (/dev/asm-disk2) [OVM]
Located 1 voting disk(s).
# crsctl replace votedisk +OCRVD
Successful addition of voting disk 8f30f93417ca4f66bf9c8df28b933b65.
Successful deletion of voting disk d61c25e4f4ec4f0fbfd283e630fba8a4.
Successfully replaced voting disk group with +OCRVD.
CRS-4266: Voting file(s) successfully replaced
# crsctl query css votedisk
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   8f30f93417ca4f66bf9c8df28b933b65 (/dev/asm-disk1) [OCRVD]
Located 1 voting disk(s).

The 2-node RAC 12.2.0.1 cluster is now ready.

Conclusion

Comparing with a 12.1.0.2 installation the biggest change is the GI image based installation.

There are also new parameters:

  • the cluster configuration type (here: STANDALONE)
  • the HUB/LEAF flag for cluster nodes (here: HUB)
  • network interface type can now be “ASM and private”
  • the dedicated disk group parameters for GIMR (although I have not been able to use them).
  • I note new cluster resources:

  • ora.ASMNET1LSNR_ASM.ora which looks like a new listener for ASM
  • ora.chad which is currently unknown for me (maybe Cluster High Availability Daemon ?)
  • ora.qosmserver which is also currently unknown for me (maybe Quality Of Service Management Server ?).
  • ASM resources (ora.asm) were previously local resources and are now cluster resources (and there are at least 3 resources even for a 2-node cluster).

    The ora.oc4j resource has been removed.

    I understand that
    starting with Oracle Grid Infrastructure 12c Release 2 (12.2), Oracle Grid Infrastructure cluster configurations are Oracle Flex Clusters deployments
    .

    Advertisements
    Post a comment or leave a trackback: Trackback URL.

    Leave a Reply

    Fill in your details below or click an icon to log in:

    WordPress.com Logo

    You are commenting using your WordPress.com account. Log Out / Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out / Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out / Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out / Change )

    Connecting to %s

    %d bloggers like this: