Overview
This document provides step-by-step instructions for upgrading Oracle Database and Oracle Grid Infrastructure from release 12.2.0.1 to 19c
Updates and additional patches may be required for your existing installation before upgrading to Oracle Database 19c and Oracle Grid Infrastructure 19c.
Please click on the INDEX and browse for more interesting posts.
There are five main sections to the upgrade:
Phase 1 – Prepare the Existing Environment
Execute prerequisites outside planned maintenance window to ensure target database and system meets minimum requirements. By executing these steps a week or weeks prior to the maintenance window, you can reduce downtime window and avoid potential upgrade errors.
The software releases and patches installed in the current environment must be at certain minimum levels before upgrading to 19c can begin.
Phase 2 – Upgrade Grid Infrastructure to 19c
This section describes how to upgrade your Grid Infrastructure from 12.2.0.1 to Oracle Grid Infrastructure version 19c.
Phase 3 – Install Database 19c Software
This section describes how install your Database 19c software into a new ORACLE_HOME directory. The installation is performed with no impact to running applications.
Phase 4 – Upgrade Database to 19c
This section describes how to upgrade your Oracle Database from versions 12.2.0.1 to Oracle Database version 19c. The database upgrade will require complete downtime of the target primary database and potentially any standby database(s) associated with the primary database.
Phase 5 – Post-upgrade Steps
This section includes potential post upgrade steps depending on your configuration and removing unused ORACLE_HOME’s.
Conventions
- Source database home will be /u01/app/oracle/product/12c/db_1
- Source grid home will be /u01/app/12c/grid
- Target database home will be /u01/app/oracle/product/19c/db_1
- Target grid home will be /u01/app/19c/grid
Assumptions
- The grid software owner is oracle.
- The database software owner is oracle.
- The Oracle inventory group is oinstall.
- The database to be upgraded is named INFRADB.
Phase 1 – Prepare the Existing Environment
Here are the steps performed in this section:
- Step 1.1 – Planning
- Step 1.2 – Review 19c Upgrade Prerequisites
- Step 1.3 – Preparations for upgrades on Oracle Database
- Step 1.4 – Validate Environment
Step 1.1 – Planning
Here are the steps performed in this section:
- Step 1.1.1 – Testing on non-production first
- Step 1.1.2 – Backup and Fallback
- Step 1.1.3 – Obtain Required Database and Operating System Account Access Account Access
Step 1.1.1 – Testing on non-production first
Upgrades or patches should always be applied first on test environments. Testing on non-production environments allows operators to become familiar with the patching steps and learn how the patching will impact system and applications.
Step 1.1.2 – Backup and Fallback
Recommend using out-of-place upgrade using a separate OH and Grid Home. Furthermore ensure you have a good, reliable backup prior to any major upgrade. Refer to RMAN documentation. Additional to a good backup, a Guaranteed Restore Point (GRP) can simplify any possible fallback operation if required.
1. Take Tar backup of grid home and database home
2. Take backup of asm metadata , ocr and olr
Backup all Disk Groups
ASMCMD> md_backup /u01/software/backup/alldg
Disk group metadata to be backed up: DATA
Disk group metadata to be backed up: FRA
Current alias directory path: ASM
Current alias directory path: _mgmtdb/C105952235736715E05301BEA8AC124B
Current alias directory path: infradb/ONLINELOG
Current alias directory path: _mgmtdb/4700AA69A9563E5FE05387E5E50AC8DA
Current alias directory path: _mgmtdb/CONTROLFILE
Current alias directory path: _mgmtdb/C105952235736715E05301BEA8AC124B/TEMPFILE
Current alias directory path: _mgmtdb/DATAFILE
Current alias directory path: _mgmtdb/TEMPFILE
Current alias directory path: crsprod/ASMPARAMETERFILE
Current alias directory path: infradb/PASSWORD
Current alias directory path: _mgmtdb
Current alias directory path: infradb/PARAMETERFILE
Current alias directory path: _mgmtdb/PARAMETERFILE
Current alias directory path: crsprod/OCRBACKUP
Current alias directory path: _mgmtdb/C10576AF34987CDEE05301BEA8AC4917
Current alias directory path: _mgmtdb/4700AA69A9563E5FE05387E5E50AC8DA/DATAFILE
Current alias directory path: _mgmtdb/C105952235736715E05301BEA8AC124B/DATAFILE
Current alias directory path: infradb/TEMPFILE
Current alias directory path: crsprod/OCRFILE
Current alias directory path: _mgmtdb/ONLINELOG
Current alias directory path: infradb/DATAFILE
Current alias directory path: infradb/CONTROLFILE
Current alias directory path: crsprod
Current alias directory path: infradb
Current alias directory path: ASM/PASSWORD
Current alias directory path: _mgmtdb/C10576AF34987CDEE05301BEA8AC4917/TEMPFILE
Current alias directory path: INFRADB/ONLINELOG
Current alias directory path: INFRADB
Current alias directory path: INFRADB/CONTROLFILE
ASMCMD>
Backup particular Disk Group
ASMCMD> md_backup /u01/software/backup/alldata -G data
Disk group metadata to be backed up: DATA
Current alias directory path: infradb/PASSWORD
Current alias directory path: _mgmtdb
Current alias directory path: infradb/PARAMETERFILE
Current alias directory path: _mgmtdb/CONTROLFILE
Current alias directory path: _mgmtdb/C105952235736715E05301BEA8AC124B/TEMPFILE
Current alias directory path: _mgmtdb/TEMPFILE
Current alias directory path: _mgmtdb/DATAFILE
Current alias directory path: crsprod/ASMPARAMETERFILE
Current alias directory path: infradb/ONLINELOG
Current alias directory path: _mgmtdb/C105952235736715E05301BEA8AC124B
Current alias directory path: _mgmtdb/4700AA69A9563E5FE05387E5E50AC8DA
Current alias directory path: ASM
Current alias directory path: infradb
Current alias directory path: crsprod
Current alias directory path: ASM/PASSWORD
Current alias directory path: _mgmtdb/C10576AF34987CDEE05301BEA8AC4917/TEMPFILE
Current alias directory path: _mgmtdb/ONLINELOG
Current alias directory path: infradb/DATAFILE
Current alias directory path: infradb/CONTROLFILE
Current alias directory path: _mgmtdb/4700AA69A9563E5FE05387E5E50AC8DA/DATAFILE
Current alias directory path: _mgmtdb/C105952235736715E05301BEA8AC124B/DATAFILE
Current alias directory path: infradb/TEMPFILE
Current alias directory path: crsprod/OCRFILE
Current alias directory path: _mgmtdb/PARAMETERFILE
Current alias directory path: _mgmtdb/C10576AF34987CDEE05301BEA8AC4917
Current alias directory path: crsprod/OCRBACKUP
Check for OCR Backups
[oracle@jack software]$ ocrconfig -showbackup
jack 2021/04/28 22:09:58 +DATA:/crsprod/OCRBACKUP/backup00.ocr.298.1071094183 0
jack 2021/04/28 18:09:39 +DATA:/crsprod/OCRBACKUP/backup01.ocr.282.1071079767 0
jack 2021/04/28 18:09:39 +DATA:/crsprod/OCRBACKUP/day.ocr.283.1071079779 0
jack 2021/04/28 18:09:39 +DATA:/crsprod/OCRBACKUP/week.ocr.284.1071079781 0
PROT-25: Manual backups for the Oracle Cluster Registry are not available
Take a manual Backup of OCR
[root@jack bin]# ./ocrconfig -manualbackup
jill 2021/04/29 18:10:57 +DATA:/crsprod/OCRBACKUP/backup_20210429_181057.ocr.299.1071166259 0
Take Manual backup of OLR from both Nodes
[root@jack bin]# ./ocrconfig -local -manualbackup
jack 2021/04/29 18:17:29 /u01/app/12c/grid/cdata/jack/backup_20210429_181729.olr 0
jack 2021/04/28 13:59:21 /u01/app/12c/grid/cdata/jack/backup_20210428_135921.olr 0
Step 1.2 – Review 19c Upgrade Prerequisites
Here are the steps to be performed in this section.
Step 1.2.1 – Grid Infrastructure and Database Software
Oracle Database and Grid Infrastructure 12c Release 2 (12.2.0.1.0), minimum version 12.2.0.1.180717.
Step 1.2.2 – Required Grid Infrastructure and Database Patches
Oracle patch:28553832 required for 12.2.0.1
Also, the latest OPatch version needs to be present
Step 1.3 – Preparations for upgrades on Oracle Database
Step 1.3.1 – Download and staging Required Files
Download the following files and stage on the first database server only:
• “Oracle Database 19.3.0.0.0” for Linux x86-64 – part number V982063-01.zip
o Download from Oracle Software Delivery Cloud (https://edelivery.oracle.com)
• “Oracle Database Grid Infrastructure 19.3.0.0.0” for Linux x86-64 – part number V982068-01.zip
o Download from Oracle Software Delivery Cloud (https://edelivery.oracle.com)
Step 1.3.2 – Update OPatch in the source Grid Home and Database Homes on all Database Servers in both 12c and 19c
If the latest OPatch release is not in place and (bundle) patches need to be applied to an existing 12.2.0.1 Grid Infrastructure and Database homes before upgrading, then first update OPatch to the latest version.
Dowload the latest Opatch for both Grid and Oracle home from https://updates.oracle.com/download/6880880.html
[root@jack grid]# pwd
/u01/app/12c/grid
[root@jack grid]# cd OPatch/
[root@jack OPatch]# ./opatch version
OPatch Version: 12.2.0.1.6
OPatch succeeded.
[root@jack grid]# mv OPatch OPatch_old
[root@jack grid]# cp /media/sf_DB_Binaries/p6880880_122010_Linux-x86-64.zip ./
[root@jack grid]# unzip p6880880_122010_Linux-x86-64.zip
[root@jack grid]# chown -R oracle:oinstall OPatch
[root@jack grid]# cd OPatch
[root@jack OPatch]# ./opatch version
OPatch Version: 12.2.0.1.25
OPatch succeeded.
Repeat the same for Database and Grid homes for both Nodes
Step 1.3.3 – Update Patch in the source Grid Home and Database Homes on all Database Servers in 12c home
Download required patches.
28553832
Apply Patch 28553832
[root@jack ~]# export PATH=$PATH:<GI_HOME>/OPatch [root@jack ~]# opatchauto apply /u01/software/28553832 OPatchauto session is initiated at Fri May 21 17:25:37 2021 System initialization log file is /u01/app/12c/grid/cfgtoollogs/opatchautodb/systemconfig2021-05-21_05-25-42PM.log. Session log file is /u01/app/12c/grid/cfgtoollogs/opatchauto/opatchauto2021-05-21_05-27-17PM.log The id for this session is W1UR Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/12c/db_1 Executing OPatch prereq operations to verify patch applicability on home /u01/app/12c/grid Patch applicability verified successfully on home /u01/app/12c/grid Patch applicability verified successfully on home /u01/app/oracle/product/12c/db_1 Executing patch validation checks on home /u01/app/12c/grid Patch validation checks successfully completed on home /u01/app/12c/grid Executing patch validation checks on home /u01/app/oracle/product/12c/db_1 Patch validation checks successfully completed on home /u01/app/oracle/product/12c/db_1 Verifying SQL patch applicability on home /u01/app/oracle/product/12c/db_1 No step execution required......... Preparing to bring down database service on home /u01/app/oracle/product/12c/db_1 Successfully prepared home /u01/app/oracle/product/12c/db_1 to bring down database service Performing prepatch operations on CRS - bringing down CRS service on home /u01/app/12c/grid Prepatch operation log file location: /u01/app/12c/gridbase/crsdata/jack/crsconfig/crspatch_jack_2021-05-21_05-28-02PM.log CRS service brought down successfully on home /u01/app/12c/grid Performing prepatch operation on home /u01/app/oracle/product/12c/db_1 Perpatch operation completed successfully on home /u01/app/oracle/product/12c/db_1 Start applying binary patch on home /u01/app/oracle/product/12c/db_1 Binary patch applied successfully on home /u01/app/oracle/product/12c/db_1 Performing postpatch operation on home /u01/app/oracle/product/12c/db_1 Postpatch operation completed successfully on home /u01/app/oracle/product/12c/db_1 Start applying binary patch on home /u01/app/12c/grid Binary patch applied successfully on home /u01/app/12c/grid Performing postpatch operations on CRS - starting CRS service on home /u01/app/12c/grid Postpatch operation log file location: /u01/app/12c/gridbase/crsdata/jack/crsconfig/crspatch_jack_2021-05-21_05-31-00PM.log CRS service started successfully on home /u01/app/12c/grid Preparing home /u01/app/oracle/product/12c/db_1 after database service restarted No step execution required......... Trying to apply SQL patch on home /u01/app/oracle/product/12c/db_1 No step execution required......... OPatchAuto successful. --------------------------------Summary-------------------------------- Patching is completed successfully. Please find the summary as follows: Host:jack RAC Home:/u01/app/oracle/product/12c/db_1 Version:12.2.0.1.0 Summary: ==Following patches were SUCCESSFULLY applied: Patch: /u01/software/28553832/28553832 Log: /u01/app/oracle/product/12c/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2021-05-21_17-29-36PM_1.log Host:jack CRS Home:/u01/app/12c/grid Version:12.2.0.1.0 Summary: ==Following patches were SUCCESSFULLY applied: Patch: /u01/software/28553832/28553832 Log: /u01/app/12c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-05-21_17-30-03PM_1.log OPatchauto session completed at Fri May 21 17:37:26 2021 Time taken to complete the session 11 minutes, 49 seconds
Repeat the same in Node 2
Check the patch applied :
[oracle@jack OPatch]$ ./opatch lsinv|grep -i applied
Patch 28553832 : applied on Tue May 25 19:31:45 IST 2021
[oracle@jack OPatch]$ ./opatch lspatches|sort -nr
28553832;OCW Interim patch for 28553832
OPatch succeeded.
Step 1.3.4 – Check Software Version
[oracle@jack software]$ crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [12.2.0.1.0]
[oracle@jack software]$ crsctl query crs softwareversion
Oracle Clusterware version on node [jack] is [12.2.0.1.0]
Step 1.3.5 – Prepare installation software
Create the new 19c Grid Infrastructure (GI_HOME) – mkdir –p /u01/app/19c/grid
Unzip Grid Installation software
Unzip latest OPatch into the new Grid Installation Software
Step 1.3.6 – Validate Readiness for Oracle Clusterware Upgrade using CVU
cd /u01/app/19c/grid
syntax – >
./runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome -dest_crshome -dest_version 19.0.0.0.0 -fixup -verbose
[oracle@jack grid]$ ./runcluvfy.sh stage -pre crsinst -upgrade -rolling -src_crshome /u01/app/12c/grid -dest_crshome /u01/app/19c/grid -dest_version 19.0.0.0.0 -fixup -verbose
Verifying Physical Memory ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack 5.6877GB (5964032.0KB) 8GB (8388608.0KB) failed
jill 5.6877GB (5964032.0KB) 8GB (8388608.0KB) failed
Verifying Physical Memory ...FAILED (PRVF-7530)
Verifying Available Physical Memory ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack 2.4121GB (2529288.0KB) 50MB (51200.0KB) passed
jill 3.3303GB (3492108.0KB) 50MB (51200.0KB) passed
Verifying Available Physical Memory ...PASSED
Verifying Swap Size ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack 16GB (1.6777212E7KB) 5.6877GB (5964032.0KB) passed
jill 16GB (1.6777212E7KB) 5.6877GB (5964032.0KB) passed
Verifying Swap Size ...PASSED
Verifying Free Space: jack:/usr,jack:/var,jack:/etc,jack:/sbin ...
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/usr jack / 4.7295GB 25MB passed
/var jack / 4.7295GB 5MB passed
/etc jack / 4.7295GB 25MB passed
/sbin jack / 4.7295GB 10MB passed
Verifying Free Space: jack:/usr,jack:/var,jack:/etc,jack:/sbin ...PASSED
Verifying Free Space: jack:/tmp ...
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/tmp jack /tmp 5.1631GB 1GB passed
Verifying Free Space: jack:/tmp ...PASSED
Verifying Free Space: jill:/usr,jill:/var,jill:/etc,jill:/sbin ...
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/usr jill / 4.7646GB 25MB passed
/var jill / 4.7646GB 5MB passed
/etc jill / 4.7646GB 25MB passed
/sbin jill / 4.7646GB 10MB passed
Verifying Free Space: jill:/usr,jill:/var,jill:/etc,jill:/sbin ...PASSED
Verifying Free Space: jill:/tmp ...
Path Node Name Mount point Available Required Status
---------------- ------------ ------------ ------------ ------------ ------------
/tmp jill /tmp 5.1553GB 1GB passed
Verifying Free Space: jill:/tmp ...PASSED
Verifying User Existence: oracle ...
Node Name Status Comment
------------ ------------------------ ------------------------
jack passed exists(1000)
jill passed exists(1000)
Verifying Users With Same UID: 1000 ...PASSED
Verifying User Existence: oracle ...PASSED
Verifying Group Existence: oinstall ...
Node Name Status Comment
------------ ------------------------ ------------------------
jack passed exists
jill passed exists
Verifying Group Existence: oinstall ...PASSED
Verifying Group Membership: oinstall(Primary) ...
Node Name User Exists Group Exists User in Group Primary Status
---------------- ------------ ------------ ------------ ------------ ------------
jack yes yes yes yes passed
jill yes yes yes yes passed
Verifying Group Membership: oinstall(Primary) ...PASSED
Verifying Run Level ...
Node Name run level Required Status
------------ ------------------------ ------------------------ ----------
jack 5 3,5 passed
jill 5 3,5 passed
Verifying Run Level ...PASSED
Verifying Hard Limit: maximum open file descriptors ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
jack hard 65536 65536 passed
jill hard 65536 65536 passed
Verifying Hard Limit: maximum open file descriptors ...PASSED
Verifying Soft Limit: maximum open file descriptors ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
jack soft 1024 1024 passed
jill soft 1024 1024 passed
Verifying Soft Limit: maximum open file descriptors ...PASSED
Verifying Hard Limit: maximum user processes ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
jack hard 16384 16384 passed
jill hard 16384 16384 passed
Verifying Hard Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum user processes ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
jack soft 16384 2047 passed
jill soft 16384 2047 passed
Verifying Soft Limit: maximum user processes ...PASSED
Verifying Soft Limit: maximum stack size ...
Node Name Type Available Required Status
---------------- ------------ ------------ ------------ ----------------
jack soft 10240 10240 passed
jill soft 10240 10240 passed
Verifying Soft Limit: maximum stack size ...PASSED
Verifying Oracle patch:28553832 ...
Node Name Applied Required Comment
------------ ------------------------ ------------------------ ----------
jack 28553832 28553832 passed
jill 28553832 28553832 passed
Verifying Oracle patch:28553832 ...PASSED
Verifying Oracle patch:27006180 ...
Node Name Applied Required Comment
------------ ------------------------ ------------------------ ----------
jack 27006180 27006180 passed
jill 27006180 27006180 passed
Verifying Oracle patch:27006180 ...PASSED
Verifying This test checks that the source home "/u01/app/12c/grid" is suitable for upgrading to version "19.0.0.0.0". ...PASSED
Verifying Architecture ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack x86_64 x86_64 passed
jill x86_64 x86_64 passed
Verifying Architecture ...PASSED
Verifying OS Kernel Version ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack 4.1.12-61.1.18.el7uek.x86_64 4.1.12 passed
jill 4.1.12-61.1.18.el7uek.x86_64 4.1.12 passed
Verifying OS Kernel Version ...PASSED
Verifying OS Kernel Parameter: semmsl ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 250 250 250 passed
jill 250 250 250 passed
Verifying OS Kernel Parameter: semmsl ...PASSED
Verifying OS Kernel Parameter: semmns ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 32000 32000 32000 passed
jill 32000 32000 32000 passed
Verifying OS Kernel Parameter: semmns ...PASSED
Verifying OS Kernel Parameter: semopm ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 100 100 100 passed
jill 100 100 100 passed
Verifying OS Kernel Parameter: semopm ...PASSED
Verifying OS Kernel Parameter: semmni ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 128 128 128 passed
jill 128 128 128 passed
Verifying OS Kernel Parameter: semmni ...PASSED
Verifying OS Kernel Parameter: shmmax ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 4398046511104 4398046511104 3053584384 passed
jill 4398046511104 4398046511104 3053584384 passed
Verifying OS Kernel Parameter: shmmax ...PASSED
Verifying OS Kernel Parameter: shmmni ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 4096 4096 4096 passed
jill 4096 4096 4096 passed
Verifying OS Kernel Parameter: shmmni ...PASSED
Verifying OS Kernel Parameter: shmall ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 1073741824 1073741824 1073741824 passed
jill 1073741824 1073741824 1073741824 passed
Verifying OS Kernel Parameter: shmall ...PASSED
Verifying OS Kernel Parameter: file-max ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 6815744 6815744 6815744 passed
jill 6815744 6815744 6815744 passed
Verifying OS Kernel Parameter: file-max ...PASSED
Verifying OS Kernel Parameter: ip_local_port_range ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack between 9000 & 65500 between 9000 & 65500 between 9000 & 65535 passed
jill between 9000 & 65500 between 9000 & 65500 between 9000 & 65535 passed
Verifying OS Kernel Parameter: ip_local_port_range ...PASSED
Verifying OS Kernel Parameter: rmem_default ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 262144 262144 262144 passed
jill 262144 262144 262144 passed
Verifying OS Kernel Parameter: rmem_default ...PASSED
Verifying OS Kernel Parameter: rmem_max ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 4194304 4194304 4194304 passed
jill 4194304 4194304 4194304 passed
Verifying OS Kernel Parameter: rmem_max ...PASSED
Verifying OS Kernel Parameter: wmem_default ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 262144 262144 262144 passed
jill 262144 262144 262144 passed
Verifying OS Kernel Parameter: wmem_default ...PASSED
Verifying OS Kernel Parameter: wmem_max ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 1048576 1048576 1048576 passed
jill 1048576 1048576 1048576 passed
Verifying OS Kernel Parameter: wmem_max ...PASSED
Verifying OS Kernel Parameter: aio-max-nr ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 1048576 1048576 1048576 passed
jill 1048576 1048576 1048576 passed
Verifying OS Kernel Parameter: aio-max-nr ...PASSED
Verifying OS Kernel Parameter: panic_on_oops ...
Node Name Current Configured Required Status Comment
---------------- ------------ ------------ ------------ ------------ ------------
jack 1 1 1 passed
jill 1 1 1 passed
Verifying OS Kernel Parameter: panic_on_oops ...PASSED
Verifying Package: kmod-20-21 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack kmod(x86_64)-20-21.el7 kmod(x86_64)-20-21 passed
jill kmod(x86_64)-20-21.el7 kmod(x86_64)-20-21 passed
Verifying Package: kmod-20-21 (x86_64) ...PASSED
Verifying Package: kmod-libs-20-21 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack kmod-libs(x86_64)-20-21.el7 kmod-libs(x86_64)-20-21 passed
jill kmod-libs(x86_64)-20-21.el7 kmod-libs(x86_64)-20-21 passed
Verifying Package: kmod-libs-20-21 (x86_64) ...PASSED
Verifying Package: binutils-2.23.52.0.1 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack binutils-2.25.1-22.base.el7 binutils-2.23.52.0.1 passed
jill binutils-2.25.1-22.base.el7 binutils-2.23.52.0.1 passed
Verifying Package: binutils-2.23.52.0.1 ...PASSED
Verifying Package: compat-libcap1-1.10 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack compat-libcap1-1.10-7.el7 compat-libcap1-1.10 passed
jill compat-libcap1-1.10-7.el7 compat-libcap1-1.10 passed
Verifying Package: compat-libcap1-1.10 ...PASSED
Verifying Package: libgcc-4.8.2 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack libgcc(x86_64)-4.8.5-11.el7 libgcc(x86_64)-4.8.2 passed
jill libgcc(x86_64)-4.8.5-11.el7 libgcc(x86_64)-4.8.2 passed
Verifying Package: libgcc-4.8.2 (x86_64) ...PASSED
Verifying Package: libstdc++-4.8.2 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack libstdc++(x86_64)-4.8.5-11.el7 libstdc++(x86_64)-4.8.2 passed
jill libstdc++(x86_64)-4.8.5-11.el7 libstdc++(x86_64)-4.8.2 passed
Verifying Package: libstdc++-4.8.2 (x86_64) ...PASSED
Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack libstdc++-devel(x86_64)-4.8.5-11.el7 libstdc++-devel(x86_64)-4.8.2 passed
jill libstdc++-devel(x86_64)-4.8.5-11.el7 libstdc++-devel(x86_64)-4.8.2 passed
Verifying Package: libstdc++-devel-4.8.2 (x86_64) ...PASSED
Verifying Package: sysstat-10.1.5 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack sysstat-10.1.5-11.el7 sysstat-10.1.5 passed
jill sysstat-10.1.5-11.el7 sysstat-10.1.5 passed
Verifying Package: sysstat-10.1.5 ...PASSED
Verifying Package: ksh ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack ksh ksh passed
jill ksh ksh passed
Verifying Package: ksh ...PASSED
Verifying Package: make-3.82 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack make-3.82-23.el7 make-3.82 passed
jill make-3.82-23.el7 make-3.82 passed
Verifying Package: make-3.82 ...PASSED
Verifying Package: glibc-2.17 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack glibc(x86_64)-2.17-157.el7 glibc(x86_64)-2.17 passed
jill glibc(x86_64)-2.17-157.el7 glibc(x86_64)-2.17 passed
Verifying Package: glibc-2.17 (x86_64) ...PASSED
Verifying Package: glibc-devel-2.17 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack glibc-devel(x86_64)-2.17-157.el7 glibc-devel(x86_64)-2.17 passed
jill glibc-devel(x86_64)-2.17-157.el7 glibc-devel(x86_64)-2.17 passed
Verifying Package: glibc-devel-2.17 (x86_64) ...PASSED
Verifying Package: libaio-0.3.109 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack libaio(x86_64)-0.3.109-13.el7 libaio(x86_64)-0.3.109 passed
jill libaio(x86_64)-0.3.109-13.el7 libaio(x86_64)-0.3.109 passed
Verifying Package: libaio-0.3.109 (x86_64) ...PASSED
Verifying Package: libaio-devel-0.3.109 (x86_64) ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack libaio-devel(x86_64)-0.3.109-13.el7 libaio-devel(x86_64)-0.3.109 passed
jill libaio-devel(x86_64)-0.3.109-13.el7 libaio-devel(x86_64)-0.3.109 passed
Verifying Package: libaio-devel-0.3.109 (x86_64) ...PASSED
Verifying Package: nfs-utils-1.2.3-15 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack nfs-utils-1.3.0-0.33.0.1.el7 nfs-utils-1.2.3-15 passed
jill nfs-utils-1.3.0-0.33.0.1.el7 nfs-utils-1.2.3-15 passed
Verifying Package: nfs-utils-1.2.3-15 ...PASSED
Verifying Package: smartmontools-6.2-4 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack smartmontools-6.2-7.el7 smartmontools-6.2-4 passed
jill smartmontools-6.2-7.el7 smartmontools-6.2-4 passed
Verifying Package: smartmontools-6.2-4 ...PASSED
Verifying Package: net-tools-2.0-0.17 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack net-tools-2.0-0.17.20131004git.el7 net-tools-2.0-0.17 passed
jill net-tools-2.0-0.17.20131004git.el7 net-tools-2.0-0.17 passed
Verifying Package: net-tools-2.0-0.17 ...PASSED
Verifying Users With Same UID: 0 ...PASSED
Verifying Current Group ID ...PASSED
Verifying Root user consistency ...
Node Name Status
------------------------------------ ------------------------
jack passed
jill passed
Verifying Root user consistency ...PASSED
Verifying correctness of ASM disk group files ownership ...PASSED
Verifying selectivity of ASM discovery string ...PASSED
Verifying ASM spare parameters ...PASSED
Verifying Disk group ASM compatibility setting ...PASSED
Verifying Package: cvuqdisk-1.0.10-1 ...
Node Name Available Required Status
------------ ------------------------ ------------------------ ----------
jack cvuqdisk-1.0.10-1 cvuqdisk-1.0.10-1 passed
jill cvuqdisk-1.0.10-1 cvuqdisk-1.0.10-1 passed
Verifying Package: cvuqdisk-1.0.10-1 ...PASSED
Verifying Host name ...PASSED
Verifying Node Connectivity ...
Verifying Hosts File ...
Node Name Status
------------------------------------ ------------------------
jack passed
jill passed
Verifying Hosts File ...PASSED
Interface information for node "jack"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
enp0s3 172.168.190.1 172.168.0.0 0.0.0.0 172.168.190.100 08:00:27:23:74:81 1500
enp0s3 172.168.190.3 172.168.0.0 0.0.0.0 172.168.190.100 08:00:27:23:74:81 1500
enp0s3 172.168.190.20 172.168.0.0 0.0.0.0 172.168.190.100 08:00:27:23:74:81 1500
enp0s3 172.168.190.30 172.168.0.0 0.0.0.0 172.168.190.100 08:00:27:23:74:81 1500
enp0s8 192.168.190.1 192.168.190.0 0.0.0.0 172.168.190.100 08:00:27:33:DF:CA 1500
Interface information for node "jill"
Name IP Address Subnet Gateway Def. Gateway HW Address MTU
------ --------------- --------------- --------------- --------------- ----------------- ------
enp0s3 172.168.190.2 172.168.0.0 0.0.0.0 172.168.190.100 08:00:27:A8:FB:52 1500
enp0s3 172.168.190.4 172.168.0.0 0.0.0.0 172.168.190.100 08:00:27:A8:FB:52 1500
enp0s3 172.168.190.10 172.168.0.0 0.0.0.0 172.168.190.100 08:00:27:A8:FB:52 1500
enp0s8 192.168.190.2 192.168.190.0 0.0.0.0 172.168.190.100 08:00:27:0C:71:64 1500
Check: MTU consistency on the private interfaces of subnet "192.168.190.0"
Node Name IP Address Subnet MTU
---------------- ------------ ------------ ------------ ----------------
jack enp0s8 192.168.190.1 192.168.190.0 1500
jill enp0s8 192.168.190.2 192.168.190.0 1500
Check: MTU consistency of the subnet "172.168.0.0".
Node Name IP Address Subnet MTU
---------------- ------------ ------------ ------------ ----------------
jack enp0s3 172.168.190.1 172.168.0.0 1500
jack enp0s3 172.168.190.3 172.168.0.0 1500
jack enp0s3 172.168.190.20 172.168.0.0 1500
jack enp0s3 172.168.190.30 172.168.0.0 1500
jill enp0s3 172.168.190.2 172.168.0.0 1500
jill enp0s3 172.168.190.4 172.168.0.0 1500
jill enp0s3 172.168.190.10 172.168.0.0 1500
Source Destination Connected?
------------------------------ ------------------------------ ----------------
jack[enp0s3:172.168.190.1] jack[enp0s3:172.168.190.3] yes
jack[enp0s3:172.168.190.1] jack[enp0s3:172.168.190.20] yes
jack[enp0s3:172.168.190.1] jack[enp0s3:172.168.190.30] yes
jack[enp0s3:172.168.190.1] jill[enp0s3:172.168.190.2] yes
jack[enp0s3:172.168.190.1] jill[enp0s3:172.168.190.4] yes
jack[enp0s3:172.168.190.1] jill[enp0s3:172.168.190.10] yes
jack[enp0s3:172.168.190.3] jack[enp0s3:172.168.190.20] yes
jack[enp0s3:172.168.190.3] jack[enp0s3:172.168.190.30] yes
jack[enp0s3:172.168.190.3] jill[enp0s3:172.168.190.2] yes
jack[enp0s3:172.168.190.3] jill[enp0s3:172.168.190.4] yes
jack[enp0s3:172.168.190.3] jill[enp0s3:172.168.190.10] yes
jack[enp0s3:172.168.190.20] jack[enp0s3:172.168.190.30] yes
jack[enp0s3:172.168.190.20] jill[enp0s3:172.168.190.2] yes
jack[enp0s3:172.168.190.20] jill[enp0s3:172.168.190.4] yes
jack[enp0s3:172.168.190.20] jill[enp0s3:172.168.190.10] yes
jack[enp0s3:172.168.190.30] jill[enp0s3:172.168.190.2] yes
jack[enp0s3:172.168.190.30] jill[enp0s3:172.168.190.4] yes
jack[enp0s3:172.168.190.30] jill[enp0s3:172.168.190.10] yes
jill[enp0s3:172.168.190.2] jill[enp0s3:172.168.190.4] yes
jill[enp0s3:172.168.190.2] jill[enp0s3:172.168.190.10] yes
jill[enp0s3:172.168.190.4] jill[enp0s3:172.168.190.10] yes
Source Destination Connected?
------------------------------ ------------------------------ ----------------
jack[enp0s8:192.168.190.1] jill[enp0s8:192.168.190.2] yes
Verifying Check that maximum (MTU) size packet goes through subnet ...PASSED
Verifying subnet mask consistency for subnet "172.168.0.0" ...PASSED
Verifying subnet mask consistency for subnet "192.168.190.0" ...PASSED
Verifying Node Connectivity ...PASSED
Verifying Multicast or broadcast check ...
Checking subnet "192.168.190.0" for multicast communication with multicast group "224.0.0.251"
Verifying Multicast or broadcast check ...PASSED
Verifying ASMLib installation and configuration verification. ...
Verifying '/etc/init.d/oracleasm' ...PASSED
Verifying '/dev/oracleasm' ...PASSED
Verifying '/etc/sysconfig/oracleasm' ...PASSED
Node Name Status
------------------------------------ ------------------------
jack passed
jill passed
Verifying ASMLib installation and configuration verification. ...PASSED
Verifying ASM Integrity ...PASSED
Verifying ACFS Driver Checks ...PASSED
Verifying OCR Integrity ...PASSED
Verifying Network Time Protocol (NTP) ...
Verifying '/etc/ntp.conf' ...
Node Name File exists?
------------------------------------ ------------------------
jack yes
jill yes
Verifying '/etc/ntp.conf' ...PASSED
Verifying '/etc/chrony.conf' ...
Node Name File exists?
------------------------------------ ------------------------
jack yes
jill yes
Verifying '/etc/chrony.conf' ...PASSED
Verifying '/var/run/chronyd.pid' ...
Node Name File exists?
------------------------------------ ------------------------
jack yes
jill yes
Verifying '/var/run/chronyd.pid' ...PASSED
Verifying Daemon 'chronyd' ...
Node Name Running?
------------------------------------ ------------------------
jack yes
jill yes
Verifying Daemon 'chronyd' ...PASSED
Verifying NTP daemon or service using UDP port 123 ...
Node Name Port Open?
------------------------------------ ------------------------
jack yes
jill yes
Verifying NTP daemon or service using UDP port 123 ...PASSED
Verifying chrony daemon is synchronized with at least one external time source ...FAILED (PRVG-13606)
Verifying Network Time Protocol (NTP) ...FAILED (PRVG-13606)
Verifying Same core file name pattern ...PASSED
Verifying User Mask ...
Node Name Available Required Comment
------------ ------------------------ ------------------------ ----------
jack 0022 0022 passed
jill 0022 0022 passed
Verifying User Mask ...PASSED
Verifying User Not In Group "root": oracle ...
Node Name Status Comment
------------ ------------------------ ------------------------
jack passed does not exist
jill passed does not exist
Verifying User Not In Group "root": oracle ...PASSED
Verifying Time zone consistency ...PASSED
Verifying Time offset between nodes ...PASSED
Verifying VIP Subnet configuration check ...PASSED
Verifying Voting Disk ...PASSED
Verifying resolv.conf Integrity ...
Node Name Status
------------------------------------ ------------------------
jack passed
jill passed
checking response for name "jack" from each of the name servers specified in
"/etc/resolv.conf"
Node Name Source Comment Status
------------ ------------------------ ------------------------ ----------
jack 172.168.190.1 IPv4 passed
jack 172.168.190.2 IPv4 passed
checking response for name "jill" from each of the name servers specified in
"/etc/resolv.conf"
Node Name Source Comment Status
------------ ------------------------ ------------------------ ----------
jill 172.168.190.2 IPv4 passed
jill 172.168.190.1 IPv4 passed
Verifying resolv.conf Integrity ...PASSED
Verifying DNS/NIS name service ...PASSED
Verifying Daemon "avahi-daemon" not configured and running ...
Node Name Configured Status
------------ ------------------------ ------------------------
jack no passed
jill no passed
Node Name Running? Status
------------ ------------------------ ------------------------
jack no passed
jill no passed
Verifying Daemon "avahi-daemon" not configured and running ...PASSED
Verifying Daemon "proxyt" not configured and running ...
Node Name Configured Status
------------ ------------------------ ------------------------
jack no passed
jill no passed
Node Name Running? Status
------------ ------------------------ ------------------------
jack no passed
jill no passed
Verifying Daemon "proxyt" not configured and running ...PASSED
Verifying loopback network interface address ...PASSED
Verifying Privileged group consistency for upgrade ...PASSED
Verifying CRS user Consistency for upgrade ...PASSED
Verifying Clusterware Version Consistency ...
Verifying cluster upgrade state ...PASSED
Verifying Clusterware Version Consistency ...PASSED
Verifying Check incorrectly sized ASM Disks ...PASSED
Verifying Network configuration consistency checks ...PASSED
Verifying /boot mount ...PASSED
Verifying OLR Integrity ...PASSED
Verifying Verify that the ASM instance was configured using an existing ASM parameter file. ...PASSED
Verifying User Equivalence ...PASSED
Verifying RPM Package Manager database ...INFORMATION (PRVG-11250)
Verifying Network interface bonding status of private interconnect network interfaces ...PASSED
Verifying /dev/shm mounted as temporary file system ...PASSED
Verifying File system mount options for path /var ...PASSED
Verifying DefaultTasksMax parameter ...PASSED
Verifying zeroconf check ...PASSED
Verifying ASM Filter Driver configuration ...PASSED
Verifying Systemd login manager IPC parameter ...PASSED
Pre-check for cluster services setup was unsuccessful.
Checks did not pass for the following nodes:
jack,jill
Failures were encountered during execution of CVU verification request "stage -pre crsinst".
Verifying Physical Memory ...FAILED
jack: PRVF-7530 : Sufficient physical memory is not available on node "jack"
[Required physical memory = 8GB (8388608.0KB)]
jill: PRVF-7530 : Sufficient physical memory is not available on node "jill"
[Required physical memory = 8GB (8388608.0KB)]
Verifying Network Time Protocol (NTP) ...FAILED
Verifying chrony daemon is synchronized with at least one external time
source ...FAILED
jack: PRVG-13606 : chrony daemon is not synchronized with any external time
source on node "jack".
jill: PRVG-13606 : chrony daemon is not synchronized with any external time
source on node "jill".
Verifying RPM Package Manager database ...INFORMATION
PRVG-11250 : The check "RPM Package Manager database" was not performed because
it needs 'root' user privileges.
CVU operation performed: stage -pre crsinst
Date: May 25, 2021 8:14:57 PM
CVU home: /u01/app/19c/grid/
User: oracle
Here in the home setup, we are ignoring the errors received
Step 1.3.7 – Using Dry-Run Upgrade Mode to Check System Upgrade Readiness
Dry run phase will not do any changes to the existing grid setup. It will just check the system readiness.
As per oracle note: Dry run does below activities:
• Validates storage and network configuration for the new release
• Checks if the system meets the software and hardware requirements for the new release
• Checks for the patch requirements and apply necessary patches before starting the upgrade
• Writes system configuration issues or errors in the gridSetupActions
Run as grid owner ( oracle)
• unset ORACLE_BASE
unset ORACLE_HOME
unset ORACLE_SID
[oracle@jack ~]$ cd /u01/app/19c/grid
[oracle@jack grid]$ ./gridSetup.sh –dryRunForUpgrade
Click Upgrade Oracle Grid Infrastructure and hit Next
Select the cluster nodes and hit Next
Deselect the EM registration option and hit Next
Set the Oracle Base and hit Next
We are going ahead with Automatically running the Configuration scripts. If you want to run this manually, please deselect the option and hit Next
The prerequisite checks will be performed, which will give you the results of Preupgrade actions (if any) to be performed
In the below screenshot, these were some pre-requirements which needs to be met before the actual upgrade. This is just an example, you may receive some other warnings, which needs to be resolved first
The Dry Run Upgrade is running which will be like an actual upgrade, without actually upgrading the GRID Infrastructure
The Dry Run Upgrade is successful
Before we start upgrade , please note 19c upgrade will need some 32gb free space in grid diskgroup . Ensure to check free space rightly from asmcm>lsdg under usable free ms , rather checking from sqlplus . Its better to have 40gb free space in grid home before we start upgrade
After upgrade its also important to track that your backups are Successful and user upgrade their client to 19c
ASMCMD> lsdg
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED NORMAL N 512 512 4096 4194304 204784 131656 51196 40230 0 Y DATA/
MOUNTED NORMAL N 512 512 4096 4194304 40952 38944 0 19472 0 N FRA/
Phase 2 – Upgrade Grid Infrastructure to 19c
Step 2.1. – Verify no active rebalance is running
Query gv$asm_operation to verify no active rebalance is running. A rebalance is running when the result of the following query is not equal to zero :
SYS@+ASM1> select count(*) from gv$asm_operation;
COUNT(*)
----------
0
Step 2.2 ACTUAL UPGRADE:
Now we will proceed with the actual upgrade in a rolling mode .
Run as grid owner ( oracle )
unset ORACLE_BASE
unset ORACLE_HOME
unset ORACLE_SID
[oracle@jack ~]$ cd /u01/app/19c/grid
[oracle@jack grid]$ ./gridSetup.sh
Click Upgrade Oracle Grid Infrastructure and hit Next
Select the cluster nodes and hit Next
Deselect the EM registration option and hit Next
Set the Oracle Base and hit Next
We are going ahead with Automatically running the Configuration scripts. If you want to run this manually, please deselect the option and hit Next
You can perform rolling upgrades of Oracle Grid Infrastructure in batches.
We select different batches here because between when we move from Batch 1 to Batch 2, we can move services from the node currently still running the previous release to the upgraded node in a rolling manner, so that services are not affected by the upgrade process.
You can use root user automation to automate running the rootupgrade.sh script during the upgrade. When you use root user automation, you can divide the nodes into groups, or batches, and start upgrades of these batches. Between batches, you can move services from nodes running the previous release to the upgraded nodes, so that services are not affected by the upgrade. Oracle recommends that you use root automation, and allow the rootupgrade.sh script to stop and start instances automatically. You can also continue to run root scripts manually.
The following restrictions apply when selecting nodes in batches for upgrade:
- You can pool nodes in batches for upgrade, up to a maximum of three batches.
- The local node, where Oracle Universal Installer (OUI) is running, must be upgraded in batch one.
The prerequisite checks will be performed, which will give you the results of Preupgrade actions (if any) to be performed
The below error needs to be ignored for your home setup. In your work environment, this shouldnt be ignored
The response file will be created.. Click on SUBMIT button to continue with the upgrade
Click on Execute Now, once it asks for Batch 2 process execution
The Grid Upgrade is successful
Step 2.3 Check the crs
[root@jack ~]# /u01/app/19c/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.GHCHKPT.advm
OFFLINE OFFLINE jack STABLE
OFFLINE OFFLINE jill STABLE
ora.LISTENER.lsnr
ONLINE ONLINE jack STABLE
ONLINE ONLINE jill STABLE
ora.chad
ONLINE ONLINE jack STABLE
ONLINE ONLINE jill STABLE
ora.data.ghchkpt.acfs
OFFLINE OFFLINE jack volume /opt/oracle/r
hp_images/chkbase is
unmounted,STABLE
OFFLINE OFFLINE jill volume /opt/oracle/r
hp_images/chkbase is
unmounted,STABLE
ora.helper
OFFLINE OFFLINE jack STABLE
OFFLINE OFFLINE jill STABLE
ora.net1.network
ONLINE ONLINE jack STABLE
ONLINE ONLINE jill STABLE
ora.ons
ONLINE ONLINE jack STABLE
ONLINE ONLINE jill STABLE
ora.proxy_advm
OFFLINE OFFLINE jack STABLE
OFFLINE OFFLINE jill STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup)
1 ONLINE ONLINE jack STABLE
2 ONLINE ONLINE jill STABLE
3 OFFLINE OFFLINE STABLE
ora.DATA.dg(ora.asmgroup)
1 ONLINE ONLINE jack STABLE
2 ONLINE ONLINE jill STABLE
3 OFFLINE OFFLINE STABLE
ora.FRA.dg(ora.asmgroup)
1 ONLINE ONLINE jack STABLE
2 ONLINE ONLINE jill STABLE
3 OFFLINE OFFLINE STABLE
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE jill STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE jack STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE jack STABLE
ora.MGMTLSNR
1 ONLINE ONLINE jack 169.254.31.101 192.1
68.190.1,STABLE
ora.asm(ora.asmgroup)
1 ONLINE ONLINE jack Started,STABLE
2 ONLINE ONLINE jill Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.asmnet1.asmnetwork(ora.asmgroup)
1 ONLINE ONLINE jack STABLE
2 ONLINE ONLINE jill STABLE
3 ONLINE OFFLINE STABLE
ora.cvu
1 ONLINE ONLINE jill STABLE
ora.infradb.db
1 ONLINE ONLINE jack Open,HOME=/u01/app/oracle/product/12c/db_1,STABLE
2 ONLINE ONLINE jill Open,HOME=/u01/app/o
racle/product/12c/db
_1,STABLE
ora.jack.vip
1 ONLINE ONLINE jack STABLE
ora.jill.vip
1 ONLINE ONLINE jill STABLE
ora.mgmtdb
1 ONLINE ONLINE jack Open,STABLE
ora.qosmserver
1 ONLINE ONLINE jill STABLE
ora.rhpserver
1 OFFLINE OFFLINE STABLE
ora.scan1.vip
1 ONLINE ONLINE jill STABLE
ora.scan2.vip
1 ONLINE ONLINE jack STABLE
ora.scan3.vip
1 ONLINE ONLINE jack STABLE
Step 2.4 CRS Version
[root@jack ~]# /u01/app/19c/grid/bin/crsctl query crs softwareversion -all
Oracle Clusterware version on node [jack] is [19.0.0.0.0]
Oracle Clusterware version on node [jill] is [19.0.0.0.0]
[root@jack ~]# /u01/app/19c/grid/bin/crsctl query crs activeversion -f
Oracle Clusterware active version on the cluster is [19.0.0.0.0]. The cluster upgrade state is [UPGRADE FINAL]. The cluster active patch level is [724960844].
[root@jack ~]# /u01/app/19c/grid/bin/crsctl query crs activeversion
Oracle Clusterware active version on the cluster is [19.0.0.0.0]
[oracle@jill OPatch]$ pwdx 3988
3988: /u01/app/19c/grid/dbs
[oracle@jill OPatch]$ pwdx 13162
13162: /u01/app/19c/grid/dbs
[oracle@jill OPatch]$ pwdx 13413
13413: /u01/app/oracle/product/12c/db_1/dbs
Phase 3 – Install Database 19c Software
Step 3.1 – Create the new Oracle DB Home directory on all primary and standby database server nodes
[oracle@jack ~]$ cd /u01/app/oracle/product/
[oracle@jack product]$ mkdir -p 19c/db_1
Step 3.2 – Unzip the software to the database home
[oracle@jack db_1]$ pwd
/u01/app/oracle/product/19c/db_1
[oracle@jack db_1]$ unzip LINUX.X64_193000_db_home.zip
Step 3.3 – Install 19c software
Select configuration options from the installer screens
1.On “Select Configuration Option” screen, select ‘Setup Software only’, and then click Next.
2.On “Select Database Installation Option”, select “Oracle Real Application Clusters database installation” and click Next
3. On “Select List of Nodes” screen, verify all database servers in your cluster are present in the list and are selected, and then click Next.
4. On “Select Database Edition”, select ‘Enterprise Edition’, then click Next.
5. On “Specify Installation Location” screen, choose “Oracle Base” and change the software location. The ORACLE_HOME directory can’t be chosen. It shows software location: /u01/app/oracle/product/19c/db_1 as the Software Location for the Database from where you started runInstaller from, and then click Next.
6. On “Operating System Groups” screen, verify group names, and then click Next.
7. On “Root script execution” screen, do not check the box. Keep root execution in your own control.
8. On “Prerequisite Checks” screen, verify there are no failed checks or warnings.
9. On “Summary” screen, verify information presented about installation, and then click Install.
10. On “Execute Configuration scripts screen, execute root.sh on each database server as instructed, and then click OK.
11. On “Finish screen”, click Close.
Phase 4 – Upgrade Database to 19c
Step 4.1 – Analyze the Database to Upgrade with the Pre-Upgrade Information Tool
Oracle Database 19c uses the preupgrade.jar, Pre-Upgrade Information Tool. The Pre-Upgrade Information Tool is provided with the 19c software. Run this tool to analyze the 11.2.0.4, 12.1.0.2, 12.1.0.2, 12.2.0.1 or 18c databases prior to upgrade.
Step 4.2 – Run the DBUA
cd $ORACLE_HOME/bin
[oracle@jack bin]$ ./dbua
Logs directory: /u01/app/oracle/oraclebase19c/cfgtoollogs/dbua/upgrade2021-06-04_06-53-28PM
Select Database to be upgraded
Choose the database to be upgraded and provide the Username as SYS and provide SYS password
Prerequisite check will be performed
The preupgrade utility tool needs to be run manually, if we are proceeding with Manual upgrade. But since this document is with GUI upgrade, so this will be taken care by the DBUA Tool
Select upgrade Options
Here we are enabling Parallel upgrade,upgrading Timezone and recompiling INVALID objects
Select Recovery Options
You can choose either to restore from a current backup, or a backup you have taken earlier or with Guaranteed restore point.
Here we are proceeding with Guaranteed restore point
Configure Management
I wont be configuring Enterprise Manager
Summary
Hit Finish
Progress
Check the progress over here
Upgrade completed
Check the upgrade log from the DBUA screen
Phase 5 – Post-upgrade Steps
Step 5.1 Validate the database
[oracle@jack ~]$ cd $ORACLE_HOME
[oracle@jack db_1]$ cd bin
[oracle@jack bin]$ ./srvctl config database -d infradb
Database unique name: infradb
Database name: infradb
Oracle home: /u01/app/oracle/product/19c/db_1
Oracle user: oracle
Spfile: +DATA/INFRADB/PARAMETERFILE/spfileinfradb.ora
Password file: +DATA/INFRADB/PASSWORD/pwdinfradb.285.1073336979
Domain:
Start options: open
Stop options: immediate
Database role: PRIMARY
Management policy: AUTOMATIC
Server pools:
Disk Groups: FRA,DATA
Mount point paths:
Services:
Type: RAC
Start concurrency:
Stop concurrency:
OSDBA group: oinstall
OSOPER group: oinstall
Database instances: infradb1,infradb2
Configured nodes: jack,jill
CSS critical: no
CPU count: 0
Memory target: 0
Maximum memory: 0
Default network number for database services:
Database is administrator managed
SQL> select banner_full from gv$version;
BANNER_FULL
----------------------------------------------------------------------------------------------------------------------------------------------------------------
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.3.0.0.0
SQL> select count(*) from dba_objects where status='INVALID';
COUNT(*)
----------
0
SQL> select comp_name,version,status from dba_registry;
COMP_NAME VERSION STATUS
---------------------------------------- ------------------------------ --------------------------------------------
Oracle Database Catalog Views 19.0.0.0.0 VALID
Oracle Database Packages and Types 19.0.0.0.0 VALID
JServer JAVA Virtual Machine 19.0.0.0.0 VALID
Oracle XDK 19.0.0.0.0 VALID
Oracle Database Java Packages 19.0.0.0.0 VALID
OLAP Analytic Workspace 19.0.0.0.0 VALID
Oracle Real Application Clusters 19.0.0.0.0 VALID
Oracle XML Database 19.0.0.0.0 VALID
Oracle Workspace Manager 19.0.0.0.0 VALID
Oracle Text 19.0.0.0.0 VALID
Oracle Multimedia 19.0.0.0.0 VALID
Spatial 19.0.0.0.0 VALID
Oracle OLAP API 19.0.0.0.0 VALID
Oracle Label Security 19.0.0.0.0 VALID
Oracle Database Vault 19.0.0.0.0 VALID
15 rows selected.
Step 5.2 Remove Oracle 12c Grid and Database binaries
Please refer to my article for steps
Remove Oracle 19c Grid Infrastructure
Issues Faced :
1. OUI-10166: The permission 0755 cannot be set for the file rootconfig.sh
While starting the actual upgrade, If you are seeing the error. as
OUI-10166: The permissions 0755 cannot be set for the file $GRID_HOME/crs/config/rootconfig.sh. OUI-10175: The effective user ID does not match the owner of the file or the process is not the user user.
Cause :
This happens after the Dry Run Upgrade due to a bug 29671933
SOLUTION:
Check the owner of the file. $GRID_HOME/crs/config/rootconfig.sh . If it is owned by root , then change it to oracle and rerun the gridsetup.sh script.
chown oracle:oinstall rootconfig.sh
2. Opatchauto doesn’t work while applying the patches
[root@jack OPatch]# opatchauto apply /u01/software/28553832
System initialization log file is /u01/app/12c/grid/cfgtoollogs/opatchautodb/systemconfig2021-05-21_12-25-04PM.log.
Session log file is /u01/app/12c/grid/cfgtoollogs/opatchauto/opatchauto2021-05-21_12-25-51PM.log
OPATCHAUTO-68021: Missing required argument(s).
OPATCHAUTO-68021: The following argument(s) are required: [-wallet]
OPATCHAUTO-68021: Provide the required argument(s).
Oracle OPatchAuto Version 13.9.0.2.0
Copyright (c) 2016, Oracle Corporation. All rights reserved.
Solution : Ensure you have the latest Opatch version present. In README, its given OPatch version : 12.2.0.1.5 or higher is rewuired. I had 12.2.0.1.6, but still faced the issue. To resovle this, the latest Opatch version needs to be installed