Description:-
This article we are going to see steps used to apply the latest oracle 19c Database Release Update 19.5.0.0.0
The environment is 2 Node RAC database.
We will also check on some errors in Oracle Readme file, which needs to be taken care for this patch.
Please click on the INDEX and browse for more interesting posts.
For more posts regarding Oracle 19c RAC, please click on the links below
Installation of ORACLE 19c RAC Servers using Virtual Box
Installation of 19c Grid and RAC database using Virtual Box
Steps to fix a corrupted SPFile in ASM with 2 node RAC
Change SCAN Name in Oracle 19c
Steps :
1. Review readme file
2. Download patch p30116789_190000_Linux-x86-64
3. Make sure the opatch version is minimum 12.2.0.1.17 for both your Grid and Database homes
[oracle@Jack]$ cd $GRID_HOME/OPatch [oracle@Jack OPatch]$ ./opatch version OPatch Version: 12.2.0.1.17 OPatch succeeded. [oracle@Jack]$ cd $ORACLE_HOME/OPatch [oracle@Jack OPatch]$ ./opatch version OPatch Version: 12.2.0.1.17 OPatch succeeded.
4. Validation of Oracle Inventory
Before beginning patch application, check the consistency of inventory information for GI home and each database home to be patched. Run the following commands as respective Oracle home owner to check the consistency.
[oracle@JACK patch]$ $GRID_HOME/OPatch/opatch lsinventory -detail -oh $GRID_HOME [oracle@JACK patch]$ $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME
5. Unzip the Patch
To apply the patch, it must be accessible from all nodes in the Oracle cluster. Download the patch and unzip it to a shared location, this is called the <UNZIPPED_PATCH_LOCATION>. This directory must be empty and not be /tmp. Additionally, the directory should have read permission for the ORA_INSTALL group.
$ cd <UNZIPPED_PATCH_LOCATION>
Check that the directory is empty.
$ ls
Unzip the patch as grid home owner except for installations that do not have any grid homes. For installations where this patch will be applied to the Database home only, the patch needs to be unzipped as the database home owner.
$ unzip p30116789_190000_Linux-x86-64.zip
6. Run OPatch Conflict Check
Determine whether any currently installed one-off patches conflict with this patch 30116789 from both Grid and Database Home as follows:
For Grid Infrastructure Home, as home user:
In here, the README has a documentary error :
% $GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir <UNZIPPED_PATCH_LOCATION>/30116789/ ---> The subdirectories need to mentioned in here. And not the Main directory
Run the below commands excluding the above :
% $GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir <UNZIPPED_PATCH_LOCATION>/30116789/30125133 % $GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir <UNZIPPED_PATCH_LOCATION>/30116789/30122149 % $GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir <UNZIPPED_PATCH_LOCATION>/30116789/30122167 % $GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir <UNZIPPED_PATCH_LOCATION>/30116789/29401763 % $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir <UNZIPPED_PATCH_LOCATION>/30116789/30125133 % $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir <UNZIPPED_PATCH_LOCATION>/30116789/30122149
Showing an example below for GRID Home :
[oracle@JACK patch]$ $GRID_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /oracle/software/patch/30116789/30125133 Oracle Interim Patch Installer version 12.2.0.1.17 Copyright (c) 2021, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /u01/app/oracle/product/19c/db_1 Central Inventory : /oracle/oraInventory from : /u01/app/oracle/product/19c/db_1/oraInst.loc OPatch version : 12.2.0.1.17 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19c/db_1/cfgtoollogs/opatch/opatch2021-03-01_09-18-24PM_1.log Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed. OPatch succeeded.
Showing an example below for Database Home :
[oracle@JACK patch]$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /oracle/software/patch/30116789/30125133 Oracle Interim Patch Installer version 12.2.0.1.17 Copyright (c) 2021, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /u01/app/oracle/product/19c/db_1 Central Inventory : /oracle/oraInventory from : /u01/app/oracle/product/19c/db_1/oraInst.loc OPatch version : 12.2.0.1.17 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19c/db_1/cfgtoollogs/opatch/opatch2021-03-01_09-20-47PM_1.log Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed. OPatch succeeded.
Note: When OPatch starts, it validates the patch and ensures that there are no conflicts with the software already installed in the ORACLE_HOME. OPatch categorizes conflicts into the following types:
Conflicts with a patch already applied to the ORACLE_HOME.
In this case, stop the patch installation and contact Oracle Support Services.
Conflicts with subset patch already applied to the ORACLE_HOME.
In this case, continue with the patch installation because as the new patch contains all the fixes from the existing patch in the ORACLE_HOME. And, in any case, the subset patch will automatically be rolled back prior to the installation of the new patch.
7. Run OPatch SystemSpace Check
Check if enough free space is available on the ORACLE_HOME filesystem for the patches to be applied as given below:
For Grid Infrastructure Home, as home user:
If you go by the README file, please remove this entry from the text file, as its a documentary error from Oracle :
<UNZIPPED_PATCH_LOCATION>/30116789/
vi /tmp/patch_list_gihome.txt /oracle/software/patch/30116789/30125133 /oracle/software/patch/30116789/30122149 /oracle/software/patch/30116789/30122167 /oracle/software/patch/30116789/29401763
Run the opatch command to check if enough free space is available in the Grid Infrastructure Home:
[oracle@JACK patch]$ $GRID_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_gihome.txt Oracle Interim Patch Installer version 12.2.0.1.17 Copyright (c) 2021, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /u01/app/oracle/product/19c/db_1 Central Inventory : /oracle/oraInventory from : /u01/app/oracle/product/19c/db_1/oraInst.loc OPatch version : 12.2.0.1.17 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19c/db_1/cfgtoollogs/opatch/opatch2021-03-01_09-29-05PM_1.log Invoking prereq "checksystemspace" Prereq "checkSystemSpace" passed. OPatch succeeded.
Run the opatch command to check if enough free space is available in the Database home, as home user:
Create file /tmp/patch_list_dbhome.txt with the following content:
vi /tmp/patch_list_dbhome.txt /oracle/software/patch/30116789/30125133 /oracle/software/patch/30116789/30122149 [oracle@JACK patch]$ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_dbhome.txt Oracle Interim Patch Installer version 12.2.0.1.17 Copyright (c) 2021, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /u01/app/oracle/product/19c/db_1 Central Inventory : /oracle/oraInventory from : /u01/app/oracle/product/19c/db_1/oraInst.loc OPatch version : 12.2.0.1.17 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19c/db_1/cfgtoollogs/opatch/opatch2021-03-01_09-32-19PM_1.log Invoking prereq "checksystemspace" Prereq "checkSystemSpace" passed. OPatch succeeded.
The command output reports pass and fail messages as per the system space availability:
If OPatch reports Prereq “checkSystemSpace” failed., then cleanup the system space as the required amount of space is not available.
If OPatch reports Prereq “checkSystemSpace” passed., then no action is needed. Proceed with patch installation.
8. opatchauto
The Opatch utility has automated the patch application for the Oracle Grid Infrastructure (GI) home and the Oracle RAC database homes. It operates by querying existing configurations and automating the steps required for patching each Oracle RAC database home of same version and the GI home.
The utility must be executed by an operating system (OS) user with root privileges, and it must be executed on each node in the cluster if the GI home or Oracle RAC database home is in non-shared storage. The utility should not be run in parallel on the cluster nodes.
Depending on command line options specified, one invocation of opatchauto can patch the GI home, Oracle RAC database homes, or both GI and Oracle RAC database homes of the same Oracle release version as the patch. You can also roll back the patch with the same selectivity.
Add the directory containing the opatchauto to the $PATH environment variable. For example:
# export PATH=$PATH:<GI_HOME>/OPatch
To patch the GI home and all Oracle RAC database homes of the same version:
# opatchauto apply <UNZIPPED_PATCH_LOCATION>/30116789
To patch only the GI home:
# opatchauto apply <UNZIPPED_PATCH_LOCATION>/30116789 -oh <GI_HOME>
To patch one or more Oracle RAC database homes:
# opatchauto apply <UNZIPPED_PATCH_LOCATION>/30116789 -oh <oracle_home1_path>,<oracle_home2_path>
To roll back the patch from the GI home and each Oracle RAC database home:
# opatchauto rollback <UNZIPPED_PATCH_LOCATION>/30116789
To roll back the patch from the GI home:
# opatchauto rollback <UNZIPPED_PATCH_LOCATION>/30116789 -oh <path to GI home>
To roll back the patch from the Oracle RAC database home:
# opatchauto rollback <UNZIPPED_PATCH_LOCATION>/30116789 -oh <oracle_home1_path>,<oracle_home2_path>
Here we are going with Opatchauto apply for both Grid and Database homes in a single command.
NOTE : If your database isnt created, then this command wont work for your database home and you need to separately run the patch for Grid and Database homes
[root@JACK ~]# export PATH=$PATH:/u01/app/19c/grid/OPatch
[root@JACK ~]# opatchauto apply /oracle/software/patch/30116789
OPatchauto session is initiated at Wed Mar 1 21:41:48 2021
System initialization log file is /u01/app/19c/grid/cfgtoollogs/opatchautodb/systemconfig2021-03-01_09-41-53PM.log.
Session log file is /u01/app/19c/grid/cfgtoollogs/opatchauto/opatchauto2021-03-01_09-43-24PM.log
The id for this session is P9O6
Executing OPatch prereq operations to verify patch applicability on home /u01/app/19c/grid
Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19c/db_1
Patch applicability verified successfully on home /u01/app/oracle/product/19c/db_1
Patch applicability verified successfully on home /u01/app/19c/grid
Verifying SQL patch applicability on home /u01/app/oracle/product/19c/db_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19c/db_1
Preparing to bring down database service on home /u01/app/oracle/product/19c/db_1
Successfully prepared home /u01/app/oracle/product/19c/db_1 to bring down database service
Bringing down CRS service on home /u01/app/19c/grid
CRS service brought down successfully on home /u01/app/19c/grid
Performing prepatch operation on home /u01/app/oracle/product/19c/db_1
Perpatch operation completed successfully on home /u01/app/oracle/product/19c/db_1
Start applying binary patch on home /u01/app/oracle/product/19c/db_1
Binary patch applied successfully on home /u01/app/oracle/product/19c/db_1
Performing postpatch operation on home /u01/app/oracle/product/19c/db_1
Postpatch operation completed successfully on home /u01/app/oracle/product/19c/db_1
Start applying binary patch on home /u01/app/19c/grid
Binary patch applied successfully on home /u01/app/19c/grid
Starting CRS service on home /u01/app/19c/grid
CRS service started successfully on home /u01/app/19c/grid
Preparing home /u01/app/oracle/product/19c/db_1 after database service restarted
No step execution required.........
Trying to apply SQL patch on home /u01/app/oracle/product/19c/db_1
SQL patch applied successfully on home /u01/app/oracle/product/19c/db_1
OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:
Host:JACK
RAC Home:/u01/app/oracle/product/19c/db_1
Version:19.0.0.0.0
Summary:
==Following patches were SKIPPED:
Patch: /oracle/software/patch/30116789/30122167
Reason: This patch is not applicable to this specified target type - "rac_database"
Patch: /oracle/software/patch/30116789/29401763
Reason: This patch is not applicable to this specified target type - "rac_database"
==Following patches were SUCCESSFULLY applied:
Patch: /oracle/software/patch/30116789/30122149
Log: /u01/app/oracle/product/19c/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-01_09-56-29PM_1.log
Patch: /oracle/software/patch/30116789/30125133
Log: /u01/app/oracle/product/19c/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-01_09-56-29PM_1.log
Host:JACK
CRS Home:/u01/app/19c/grid
Version:19.0.0.0.0
Summary:
==Following patches were SKIPPED:
Patch: /oracle/software/patch/30116789/29401763
Reason: This patch is already been applied, so not going to apply again.
==Following patches were SUCCESSFULLY applied:
Patch: /oracle/software/patch/30116789/30122149
Log: /u01/app/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-01_09-58-43PM_1.log
Patch: /oracle/software/patch/30116789/30122167
Log: /u01/app/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-01_09-58-43PM_1.log
Patch: /oracle/software/patch/30116789/30125133
Log: /u01/app/19c/grid/cfgtoollogs/opatchauto/core/opatch/opatch2021-03-01_09-58-43PM_1.log
OPatchauto session completed at Wed Mar 1 22:10:20 2021
Time taken to complete the session 18 minutes, 39 seconds
9. Check whether patching applied successfully or not
Check ASM Version
[oracle@JACK patch]$ asmcmd -V asmcmd version 19.5.0.0.0
Check Database Version
[oracle@JACK patch]$ sqlplus '/as sysdba' SQL*Plus: Release 19.0.0.0.0 - Production on Wed Mar 1 22:13:25 2021 Version 19.5.0.0.0 Copyright (c) 1982, 2019, Oracle. All rights reserved. Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.5.0.0.0 SQL> Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.5.0.0.0
Check Database Verbose :
[oracle@JACK patch]$ $ORACLE_HOME/OPatch/datapatch -verbose SQL Patching tool version 19.5.0.0.0 Production on Wed Mar 1 22:15:24 2021 Copyright (c) 2012, 2019, Oracle. All rights reserved. Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_25903_2021_03_01_22_15_24/sqlpatch_invocation.log Connecting to database...OK Gathering database info...done Bootstrapping registry and package to current versions...done Determining current state...done Current state of interim SQL patches: No interim patches found Current state of release update SQL patches: Binary registry: 19.3.0.0.0 Release_Update 190410122720: Not installed on nodes JACK 19.5.0.0.0 Release_Update 190909180549: Not installed on nodes JILL SQL registry: Applied 19.3.0.0.0 Release_Update 190410122720 successfully on 28-FEB-21 11.23.20.760178 PM Adding patches to installation queue and performing prereq checks...done Installation queue: No interim patches need to be rolled back No release update patches need to be installed No interim patches need to be applied SQL Patching tool complete on Wed Mar 1 22:13:44 2021
10. Repeat the above steps for other Nodes