...
Thu Mar 3 14:12:50 2011
Errors in file /oracle/admin/ORCL/udump/orcl_ora_15969.trc:
ORA-00600: internal error code, arguments: [kjpcre2], [], [], [], [], [], [], []
Thu Mar 3 14:12:50 2011
Errors in file /oracle/admin/ORCL/udump/orcl_ora_15969.trc:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [kjpcre2], [], [], [], [], [], [], []
...
Thu Mar 3 17:50:43 2012
Errors in file /oracle/admin/ORCL/udump/orcl_ora_15979.trc:
ORA-00600: internal error code, arguments: [kjpcre2], [], [], [], [], [], [], []
Thu Mar 3 17:50:43 2012
Errors in file /oracle/admin/ORCL/udump/orcl_ora_15979.trc:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [kjpcre2], [], [], [], [], [], [], []
...ORA-00600 is a general and internal error code which will show the arguments to indicate the causes of the problem. The argument "kjpcre2" indicates that PMON takes too long to clean up a dead process. But which processes are dead?
SQL> select inst_id, spid, username, program from gv$process where spid in ('15979','15969');
INST_ID SPID USERNAM PROGRAM
---------- ------------ ------- ---------------------------------------
1 15969 oracle oracle@ORCL (D000)
1 15979 oracle oracle@ORCL (D002)You can see they are dispatchers. But, is there any message in the alert log when the two dispatchers dead? Yes, we found the message as below:
..
Thu Mar 3 13:11:08 2011
found dead dispatcher 'D002', pid = (48, 3)
...
Thu Mar 3 13:18:41 2011
found dead dispatcher 'D000', pid = (46, 3)
...At last, we know the whole story, someone was trying to kill hung sessions on DB-level but failed, so he killed the related dispatchers on OS-level to force the other sessions go another circuits. But now, what conditions of the two dead dispatchers they are in?
SQL> select name, status, accept, spid from gv$dispatcher vd, gv$process vp where vd.inst_id =1 and vd.paddr = vp.addr order by name;
NAME STATUS ACCEPT SPID
---- ---------------- ------ ------------
D000 REFUSE NO 15969
D001 WAIT YES 15973
D002 REFUSE NO 15979
D003 WAIT YES 15983
D004 WAIT YES 15989The dead processes are noted as "REFUSE" and "ACCEPT" is NO.
PMON in 9i has a bug which fails to clean up a dead process automatically under some conditions, you can do it manually to shutdown the dead processes explicitly, then PMON will restart the two processes automatically:
SQL> alter system shutdown immediate 'D000';
SQL> alter system shutdown immediate 'D002';If PMON did not restart the dispatchers, you can re-enable dispathers by this:
SQL> ALTER SYSTEM SET DISPATCHERS="(ADDRESS=(PROTOCOL=TCP)(CONNECTIONS=500)(HOST=ORCL.EXAMPLE.COM))(DISPATCHERS=5)(SERVICE=ORCL)" SCOPE=MEMORY;Theoretically, PMON will clean up dead process automatically, but it sometimes fails to do it in 9i databases like the error message in the alert log:...
Thu Mar 3 14:12:50 2011
Errors in file /oracle/admin/ORCL/udump/orcl_ora_15969.trc:
ORA-00600: internal error code, arguments: [kjpcre2], [], [], [], [], [], [], []
Thu Mar 3 14:12:50 2011
Errors in file /oracle/admin/ORCL/udump/orcl_ora_15969.trc:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [kjpcre2], [], [], [], [], [], [], []
...
Thu Mar 3 17:50:43 2012
Errors in file /oracle/admin/primdb/udump/orcl_ora_15979.trc:
ORA-00600: internal error code, arguments: [kjpcre2], [], [], [], [], [], [], []
Thu Mar 3 17:50:43 2012
Errors in file /oracle/admin/primdb/udump/orcl_ora_15979.trc:
ORA-00603: ORACLE server session terminated by fatal error
ORA-00600: internal error code, arguments: [kjpcre2], [], [], [], [], [], [], []
...ORA-00600 is a general and internal error code which will show the arguments to indicate the causes of the problem. The argument "kjpcre2" indicates that PMON takes too long to clean up a dead process. But which processes are dead?
SQL> select inst_id, spid, username, program from gv$process where spid in ('15979','15969');
INST_ID SPID USERNAM PROGRAM
---------- ------------ ------- ---------------------------------------
1 15969 oracle oracle@primdb (D000)
1 15979 oracle oracle@primdb (D002)You can see they are dispatchers. But, is there any message in the alert log when the two dispatchers dead? Yes, we found the message as below:
..
Thu Mar 3 13:11:08 2011
found dead dispatcher 'D002', pid = (48, 3)
...
Thu Mar 3 13:18:41 2011
found dead dispatcher 'D000', pid = (46, 3)
...At last, we know the whole story, someone was trying to kill hung sessions on DB-level but failed, so he killed the related dispatchers on OS-level to force the other sessions go another circuits. But now, what conditions of the two dead dispatchers they are in?
SQL> select name, status, accept, spid from gv$dispatcher vd, gv$process vp where vd.inst_id =1 and vd.paddr = vp.addr order by name;
NAME STATUS ACCEPT SPID
---- ---------------- ------ ------------
D000 REFUSE NO 15969
D001 WAIT YES 15973
D002 REFUSE NO 15979
D003 WAIT YES 15983
D004 WAIT YES 15989The dead processes are noted as "REFUSE" and "ACCEPT" is NO.
PMON in 9i has a bug which fails to clean up a dead process automatically under some conditions, you can do it manually to shutdown the dead processes explicitly, then PMON will restart the two processes automatically:
SQL> alter system shutdown immediate 'D000';
SQL> alter system shutdown immediate 'D002';If PMON did not restart the dispatchers, you can re-enable dispathers by this:
SQL> ALTER SYSTEM SET DISPATCHERS="(ADDRESS=(PROTOCOL=TCP)(CONNECTIONS=500)(HOST=ORCL.EXAMPLE.COM))(DISPATCHERS=5)(SERVICE=ORCL)" SCOPE=MEMORY;