Tips and Tricks for the Oracle DBA: 2022

ISSUE:

12.2.9 adop prepare failed with the error below despite updated ETCC having been successfully ran:

....................................................................................................................

Validating system setup.

Node registry is valid.

[ERROR] ETCC not run in the database node data16

The EBS Technology Codelevel Checker needs to be run on the database node.

It is available as Patch 17537119.

Encountered the above errors when performing database validations.

Resolve the above errors and restart adop.

....................................................................................................................

CAUSE:

Possible mismatch between FND_NODES and APPLSYS.TXK_TCC_RESULTS.

Failure occurred in my freshly cloned environment. In my case, the "domain" had been appended to the node_name.

SQL> SELECT NODE_NAME,DATABASE_NAME FROM APPLSYS.TXK_TCC_RESULTS;

NODE_NAME DATABASE_NAME

-------------------- ----------------------------------------------------------------

data16.xyzdomain.com dbproject

app010 dbproject

SQL> SELECT NODE_NAME,NODE_ID,HOST FROM FND_NODES;

NODE_NAME NODE_ID HOST

-------------------- ---------- --------------------

DATA16 12125 data16

APP010 12126 app010

AUTHENTICATION 12127

FIX:

#1: Backup the APPLSYS.TXK_TCC_RESULTS

create table APPS.TXK_TCC_RESULTS_backup as select * from APPLSYS.TXK_TCC_RESULTS;

#2: Update the APPLSYS.TXK_TCC_RESULTS to correct the NODE_NAME value:

update APPLSYS.TXK_TCC_RESULTS set node_name='DATA16' where node_name ='data16.xyzdomain.com';

commit;

#3: Verify:

SQL> SELECT NODE_NAME,DATABASE_NAME FROM APPLSYS.TXK_TCC_RESULTS;

NODE_NAME DATABASE_NAME

-------------------- ----------------------------------------------------------------

DATA16 dbproject

app010 dbproject

Proceeded with adop prepare.

On : 12.2.9 version, Patch Application Issues: fs_clone fails with following error:

###ERROR:

................................................................................................................................

Lines #(188-190):

TIME : Fri April 15 13:53:08 2022

FUNCTION: main::removeDirectory [ Level 1 ]

ERRORMSG: Failed to delete the directory /uproject1/ora/uproject/fs2/EBSapps/comn.

/uproject1/ora/uproject/fs_ne/EBSapps/log/adop/35/20220415_135028/adop.log:

................................................................................................................................

*******FATAL ERROR*******

PROGRAM : (/uproject1/ora/uproject/fs1/EBSapps/appl/ad/12.0.0/patch/115/bin/txkADOPPreparePhaseSynchronize.pl)

TIME : Fri April 15 13:53:08 2022

FUNCTION: main::removeDirectory [ Level 1 ]

ERRORMSG: Failed to delete the directory /uproject1/ora/uproject/fs2/EBSapps/comn.

"txkADOPPreparePhaseSynchronize.log" 190L, 5846C

[uproject@project bin]$

[uproject@project bin]$ echo $CONTEXT_FILE

/uproject1/ora/uproject/fs1/inst/apps/ccproject_project/appl/admin/ccproject_project.xml

[uproject@project bin]$

................................................................................................................................

####CAUSE

Fs_clone is unable to delete directory due to Patch file system (FS2 in tis case) *comn* directory being in use.

................................................................................................................................

[uproject@project TXK_SYNC_create]$ lsof | grep fs2

lsof: WARNING: can't stat() cifs file system /winshare/cotfin

Output information may be incomplete.

java 6111 uproject txt REG 0,52 7710 4269120606 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/bin/java (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 255773 4268870522 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/i386/libsunec.so (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 63439194 4266306476 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/rt.jar (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 110222 4268870521 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/i386/libnet.so (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 283852 4269013831 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/ext/sunjce_provider.jar (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 247427 4269013832 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/ext/sunpkcs11.jar (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 118834 4268946548 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/.nfs00000000fe72f474000135ac (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 661748 4268946551 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/.nfs00000000fe72f477000135ad (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 82198 4266103557 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/i386/libnio.so (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 111314 4266103563 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/i386/libzip.so (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 35388 4269013834 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/ext/sunec.jar (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 186081 4266749473 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/i386/.nfs00000000fe516e21000135ae (192.168.50.200:/project_app02)

java 6111 uproject mem REG 0,52 11758682 4269276099 /uproject1/ora/uproject/fs2/EBSapps/comn/util/jdk32/jre/lib/i386/server/.nfs00000000fe77fbc3000135af (192.168.50.200:/project_app02)

................................................................................................................................

#### SOLUTION

Killed the process that has the directory locked.

................................................................................................................................

[uproject@project TXK_SYNC_create]$

[uproject@project TXK_SYNC_create]$ ps -aef | grep 6111

uproject 6111 1 0 Apr27 ? 00:43:39 java -jar bin/zEnterpriseConnector.jar -s

kill -9 6111

[uproject@project bin]$

[uproject@project bin]$ lsof | grep fs2 | wc -l

lsof: WARNING: can't stat() cifs file system /winshare/cotfin

Output information may be incomplete.

[uproject@project bin]$

................................................................................................................................

I then restarted adop by running fs_clone with the force=yes option

................................................................................................................................

adop phase=fs_clone force=yes

................................................................................................................................

Tips and Tricks for the Oracle DBA

Thursday, October 6, 2022

ADOP prepare fails with "[ERROR] ETCC not run in the database"

Thursday, May 19, 2022

ADOP: Fs_clone fails with error Failed to delete the directory

About Me

Archive