Data Guard

How to Resolve Standby ORA-00600 [3020]

ORA-00600 [3020]

Found MRP shutdown with ORA-00600 [3020] in alert log of a standby database due to inconsistency of a newly created data file.

Wed Nov 21 18:32:12 2018
Errors in file /oracle/admin/ORCL/bdump/orcl_mrp0_4589393.trc:
ORA-00600: internal error code, arguments: [3020], [1786773511], [1], [37836], [320123], [72], [], []
ORA-10567: Redo is inconsistent with data block (file# 854, block# 7)
ORA-10564: tablespace UNDO
ORA-01110: data file 854: '/oradata/ORCL/UNDO12.dbf'
ORA-10560: block type 'KTFB Bitmapped File Space Bitmap'
Recovery interrupted.
Recovered data files restored to a consistent state at change 145333907627.
MRP0: Background Media Recovery process shutdown

It looks like that the newly created data file UNDO12.dbf is somewhat inconsistent with the redo logs of standby database.

Workaround

This problem was reported as a bug 3615851 and it’s fixed in 9.2.0.6. But I don’t think you would patch your pretty old database (9.2.0.4 or 9.2.0.5) for solving this problem.

My workaround is pretty easy, that’s to copy the data file from the primary to the standby server. Then start managed recovery process (MRP).

Begin BACKUP Mode

You have to get into BEGIN BACKUP Mode only on this tablespace UNDO in order to get the consistent and useful data file before shipping to the standby database.

SQL> alter tablespace undo begin backup;

Tablespace altered.

SQL> !

Now we have to be quick!

Copy the Data File

$ cp -p /oradata/ORCL/UNDO12.dbf /ready/to/ftp/UNDO12.dbf
$ exit

Please note that, remote ftp may be very time-consuming, so here I did is a local copy that can shorten the hot backup mode.

End BACKUP Mode

Once the local copy is completed, we have to end the hot backup mode as soon as possible in case of anther archiving problem.

SQL> alter tablespace undo end backup;

Tablespace altered.

SQL> column file# format 999;
SQL> column status format a10;
SQL> column change# format 9999999999999;
SQL> select distinct t.name tablespace, b.status, b.change# from v$backup b inner join v$datafile d on b.file# = d.file# inner join v$tablespace t on d.ts# = t.ts# where t.name = 'UNDO';

TABLESPACE           STATUS            CHANGE#
-------------------- ---------- --------------
UNDO                 NOT ACTIVE   276205972967

FTP the Data File to the Standby Database

Now you can ftp, rcp, sftp or scp this file to the standby server and make sure that it is successfully shipped to the destination.

Start MRP

SQL> alter database recover managed standby database disconnect from session;

Database altered.

SQL> select sysdate, process, status, sequence#, block#, blocks from v$managed_standby;

SYSDATE             PROCESS   STATUS        SEQUENCE#     BLOCK#     BLOCKS
------------------- --------- ------------ ---------- ---------- ----------
2018-11-21 19:48:02 MRP0      APPLYING_LOG      37857    1358758    2047998
2018-11-21 19:48:02 RFS       WRITING           37921     659456    2047998
2018-11-21 19:48:02 RFS       WRITING           37924     675840    2047998

It’s back. MRP0 starts to apply new archived logs. I see no more ORA-00600 [3020] in the alert log.

Leave a Reply

Your email address will not be published. Required fields are marked *