Oracle RAC 12.2.0.1 Patch 30920127(Apr 2020)

Environment introduction:

Oracle RAC version: 12.2.0.1 (two nodes)

Operating system version: CentOS 7.4 64bit

1, Patch environment preparation

1.1 upload installation package

p6880880_122010_Linux-x86-64

p30920127_122010_Linux-x86-64

1.2 ensure the ipatch utility version

Patch 30920127 requires the Opatch tool to be 12.2.0.1.19 and higher (both gird and Oracle users need to confirm)

1.2.1 backup the original ipatch utility

Executed by root, both nodes need to execute

mv /u01/app/12.2.0/grid/OPatch /u01/app/12.2.0/grid/OPatch_bak mv /u01/app/oracle/product/12.2.0/db_1/OPatch /u01/app/oracle/product/12.2.0/db_1/OPatch_bak

1.2.2 install a new ipatch utility

Executed by root, both nodes need to execute

unzip /tmp/p6880880_122010_Linux-x86-64.zip -d /u01/app/12.2.0/grid unzip /tmp/p6880880_122010_Linux-x86-64.zip -d /u01/app/oracle/product/12.2.0/db_1/ chown grid:oinstall -R /u01/app/12.2.0/grid/OPatch chown oracle:oinstall -R /u01/app/oracle/product/12.2.0/db_1/OPatch

1.2.3 confirmation version

grid user execution, both nodes need to execute

$ORACLE_HOME/OPatch/opatch version

1.2 verify Oracle Inventory and save (compare after patching)

Oracle and grid users execute on two nodes respectively,

$ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME

1.3 decompress the patch package to the specified directory

root executes on two nodes

unzip /tmp/p30920127_122010_Linux-x86-64.zip -d /PSU/ chmod 777 -R /PSU/

1.4 conflict check

Execute at two nodes to ensure no patch conflict

Check grid patch (grid user)

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /PSU/30920127/30886680 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /PSU/30920127/30882603 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /PSU/30920127/30869447 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /PSU/30920127/26839277 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /PSU/30920127/30888810

Check db patch (oracle user)

$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /PSU/30920127/30886680 $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /PSU/30920127/30882603

1.5 space inspection

Check the grid space (grid user) to make sure there is enough space

cat /tmp/patch_list_gihome.txt /PSU/30920127/30886680 /PSU/30920127/30882603 /PSU/30920127/30869447 /PSU/30920127/26839277 /PSU/30920127/30888810 $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_gihome.txt

Check the db space (oracle user) to make sure there is enough space

cat /tmp/patch_list_dbhome.txt /PSU/30920127/30886680 /PSU/30920127/30882603 $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_dbhome.txt

2, Install patches

2.1 patching

Both Grid home and db home are installed on the corresponding nodes, and ACFS is not configured

The root user executes on two nodes respectively. It needs to wait until one node finishes executing and confirms the success before executing on the second node

# /u01/app/12.2.0/grid/OPatch/opatchauto apply /PSU/30920127 OPatchauto session is initiated at Tue May 5 07:40:47 2020 System initialization log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-05-05_07-40-53AM.log. Session log file is /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/opatchauto2020-05-05_07-43-06AM.log The id for this session is WLQR Executing OPatch prereq operations to verify patch applicability on home /u01/app/12.2.0/grid Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/12.2.0/db_1 Patch applicability verified successfully on home /u01/app/oracle/product/12.2.0/db_1 Patch applicability verified successfully on home /u01/app/12.2.0/grid Verifying SQL patch applicability on home /u01/app/oracle/product/12.2.0/db_1 SQL patch applicability verified successfully on home /u01/app/oracle/product/12.2.0/db_1 Preparing to bring down database service on home /u01/app/oracle/product/12.2.0/db_1 Successfully prepared home /u01/app/oracle/product/12.2.0/db_1 to bring down database service Bringing down CRS service on home /u01/app/12.2.0/grid Prepatch operation log file location: /u01/app/grid/crsdata/rac1/crsconfig/crspatch_rac1_2020-05-05_07-43-54AM.log CRS service brought down successfully on home /u01/app/12.2.0/grid Performing prepatch operation on home /u01/app/oracle/product/12.2.0/db_1 Perpatch operation completed successfully on home /u01/app/oracle/product/12.2.0/db_1 Start applying binary patch on home /u01/app/oracle/product/12.2.0/db_1 Binary patch applied successfully on home /u01/app/oracle/product/12.2.0/db_1 Performing postpatch operation on home /u01/app/oracle/product/12.2.0/db_1 Postpatch operation completed successfully on home /u01/app/oracle/product/12.2.0/db_1 Start applying binary patch on home /u01/app/12.2.0/grid Binary patch applied successfully on home /u01/app/12.2.0/grid Starting CRS service on home /u01/app/12.2.0/grid Postpatch operation log file location: /u01/app/grid/crsdata/rac1/crsconfig/crspatch_rac1_2020-05-05_08-03-46AM.log CRS service started successfully on home /u01/app/12.2.0/grid Preparing home /u01/app/oracle/product/12.2.0/db_1 after database service restarted No step execution required......... Trying to apply SQL patch on home /u01/app/oracle/product/12.2.0/db_1 SQL patch applied successfully on home /u01/app/oracle/product/12.2.0/db_1 OPatchAuto successful. --------------------------------Summary-------------------------------- Patching is completed successfully. Please find the summary as follows: Host:rac1 RAC Home:/u01/app/oracle/product/12.2.0/db_1 Version:12.2.0.1.0 Summary: ==Following patches were SKIPPED: Patch: /PSU/30920127/30869447 Reason: This patch is not applicable to this specified target type - "rac_database" Patch: /PSU/30920127/26839277 Reason: This patch is not applicable to this specified target type - "rac_database" Patch: /PSU/30920127/30888810 Reason: This patch is not applicable to this specified target type - "rac_database" ==Following patches were SUCCESSFULLY applied: Patch: /PSU/30920127/30882603 Log: /u01/app/oracle/product/12.2.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-05-05_07-45-26AM_1.log Patch: /PSU/30920127/30886680 Log: /u01/app/oracle/product/12.2.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-05-05_07-45-26AM_1.log Host:rac1 CRS Home:/u01/app/12.2.0/grid Version:12.2.0.1.0 Summary: ==Following patches were SKIPPED: Patch: /PSU/30920127/26839277 Reason: This patch is already been applied, so not going to apply again. ==Following patches were SUCCESSFULLY applied: Patch: /PSU/30920127/30869447 Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-05-05_07-52-27AM_1.log Patch: /PSU/30920127/30882603 Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-05-05_07-52-27AM_1.log Patch: /PSU/30920127/30886680 Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-05-05_07-52-27AM_1.log Patch: /PSU/30920127/30888810 Log: /u01/app/12.2.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-05-05_07-52-27AM_1.log Patching session reported following warning(s): _________________________________________________ [WARNING] The database instance 'orcl1' from '/u01/app/oracle/product/12.2.0/db_1', in host'rac1' is not running. SQL changes, if any, will not be applied. To apply. the SQL changes, bring up the database instance and run the command manually from any one node (run as oracle). Refer to the readme to get the correct steps for applying the sql changes. OPatchauto session completed at Tue May 5 08:17:22 2020 Time taken to complete the session 36 minutes, 35 seconds

2.2 execute the corresponding sql

Because it is an sql statement, it only needs to be executed in one node (oracle user)

Our environment is Standalone DB, no CDB/PDB

After the patch is completed, the database has been started

% cd $ORACLE_HOME/OPatch % ./datapatch -verbose

Check the patch log information to confirm that the log is useless

$ORACLE_BASE/cfgtoollogs/sqlpatch/30886680/

Execute the corresponding sql

cd $ORACLE_HOME/rdbms/admin sqlplus /nolog SQL> CONNECT / AS SYSDBA SQL> @utlrp.sql SQL > @dbmsjdev.sql SQL > exec dbms_java_dev.disable

If the rman environment is configured, UPGRADE CATALOG is also required;

3, Verify patch set and cluster status

3.1 confirm patch set

grid and oracle users execute in two nodes respectively, and compare with the previous one

$ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME /u01/12.2.0/grid/bin/crsctl query crs releasepatch

3.2 confirm cluster status

crsctl stat res -t

4, Problems encountered

4.1 CRS-6706 Patch leve inconsistency

1.Run the following command as the root user to complete the patching set up behind the scenes:

#GI_HOME/bin:> ./clscfg -localpatch

2.Run the following command as the root user to lock the GI home:

#GI_HOME/crs/install:> ./rootcrs.sh -lock

3.Run the following command as the root user to start the GI:

#GI_HOME/bin:> ./crsctl start crs

4.2 GSMADMIN_INTERNAL.DBMS_GSM_POOLADMIN is invalid

Error message:

Error at line 503: Warning: Package Body created with compilation errors. Error at line 6801: Warning: Package Body created with compilation errors. Error at line 6808: 4675/4 PL/SQL: Statement ignored Error at line 6809: 4675/23 PLS-00302: component 'ISNONORACLECLOUD' must be declared Error at line 6831: 4675/4 PL/SQL: Statement ignored Error at line 6832: 4675/23 PLS-00302: component 'ISNONORACLECLOUD' must be declared Error at line 7485: Warning: Trigger created with compilation errors. Error at line 7492: 0/0 PLS-00905: object GSMADMIN_INTERNAL.DBMS_GSM_POOLADMIN is invalid Error at line 7509: Warning: Trigger created with compilation errors. Error at line 7516: 0/0 PLS-00905: object GSMADMIN_INTERNAL.DBMS_GSM_POOLADMIN is invalid Error at line 7523: Warning: Trigger created with compilation errors. Error at line 7530: 0/0 PLS-00905: object GSMADMIN_INTERNAL.DBMS_GSM_POOLADMIN is invalid

resolvent:

GDS objects are invalid - gsmain? Interface (document ID 2425776.1)

Gsmain and lbacsys objects invalid after create new 12C database (document ID 2284611.1)

Execute following steps to compile the invalid objects:

$sqlplus "/as sysdba" SQL> spool fix_cat_logfile.txt SQL> startup upgrade SQL > @?/rdbms/admin/catalog.sql SQL > @?/rdbms/admin/catproc.sql SQL > @?/rdbms/admin/utlrp.sql SQL> spool off

Oracle RAC 12.2.0.1 Patch 30920127(Apr 2020)