How to Resolve Srvctl Start Database ORA-00304: requested INSTANCE_NUMBER is busy

Srvctl Start Database ORA-00304

Saw some errors when I tried to startup a cluster database by srvctl.

[oracle@primary01 ~]$ srvctl start database -d compdb
PRCR-1079 : Failed to start resource ora.compdb.db
CRS-5017: The resource action "ora.compdb.db start" encountered the following error:
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3640
Additional information: 671785331
Process ID: 0
Session ID: 0 Serial number: 0
. For details refer to "(:CLSN00107:)" in "/u01/app/grid/diag/crs/primary02/crs/trace/crsd_oraagent_oracle.trc".

CRS-2674: Start of 'ora.compdb.db' on 'primary02' failed
CRS-2632: There are no more servers to try to place resource 'ora.compdb.db' on that would satisfy its placement policy

Eventually, only one RAC node was running, the other node failed to be started up. More specifically, if NODE 1’s instance was online, then NODE 2’s instance was offline, or vice versa.

Let’s inspect the log for more details:

[oracle@primary02 ~]$ vi /u01/app/grid/diag/crs/primary02/crs/trace/crsd_oraagent_oracle.trc
...
2019-06-27 12:04:42.803845 :CLSDYNAM:123709184: [ora.compdb.db]{1:56587:1766} [start] ORA-00304: requested INSTANCE_NUMBER is busy

2019-06-27 12:04:42.803999 :CLSDYNAM:123709184: [ora.compdb.db]{1:56587:1766} [start] InstAgent::start: 1 errcode 304
2019-06-27 12:04:42.804059 :CLSDYNAM:123709184: [ora.compdb.db]{1:56587:1766} [start] InstAgent::start: ORA-304 instance busy  m_instanceType 1 m_lastOCIError 304
2019-06-27 12:04:42.804113 :CLSDYNAM:123709184: [ora.compdb.db]{1:56587:1766} [start] InstAgent::start: ORA-304 instance busy  dbType RAC
2019-06-27 12:04:42.804659 :CLSDYNAM:123709184: [ora.compdb.db]{1:56587:1766} [start] InstConnection::connectInt: server not attached
2019-06-27 12:04:42.820420 :CLSDYNAM:123709184: [ora.compdb.db]{1:56587:1766} [start] ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3640
Additional information: 671785331
Process ID: 0
Session ID: 0 Serial number: 0

Solution

Did you see ORA-00304? It told us that “requested INSTANCE_NUMBER is busy”. As we know, INSTANCE_NUMBER is a database parameter used for RAC. Next, let’s output a PFILE from SPFILE in order to inspect parameters of the cluster database.

SQL> create pfile='/tmp/pfile.ora' from spfile;

File created.

[oracle@primary01 ~]$ vi /tmp/pfile.ora
...
*.instance_number=1
...
*.thread=1
*.undo_tablespace='UNDOTBS1'

As we can see, those parameters with wild cards above should be used for single-instance databases, not for clustered databases. For RAC database, they should be prefixed with INSTANCE_NAME. For example:

PRIMDB1.instance_number=1
PRIMDB2.instance_number=2
...
PRIMDB1.thread=1
PRIMDB2.thread=2
PRIMDB1.undo_tablespace='UNDOTBS1'
PRIMDB2.undo_tablespace='UNDOTBS2'

So I created a new shared SPFILE correctly from the modified PFILE for the cluster database, then restart it again by srvctl. I solved the problem.

In summary, the key actions to this problem are:

  1. Inspect the log.
  2. Inspect the parameters.

Leave a Reply

Your email address will not be published. Required fields are marked *