Views expressed here are solely that of my own. Please make sure that you test the code/queries that appear on my blog before applying them to the production environment.

Wednesday, March 7, 2012

PRKP-1001 and CRS-0215 errors in an Oracle 10gR2 RAC database

I just installed a test Oracle 10gR2 RAC database and I am getting the below error when I try to start the instance on first node by using srvctl utility. I do not get any error when I try to startup the instance in SQL Plus.

This is the current state and I want to open instance on first node, first component in the below list.
[root@nldevrac01]:/oracle/crshome1/bin > crs_stat -t
Name           Type           Target    State     Host        
------------------------------------------------------------
ora....P1.inst application    ONLINE    OFFLINE               
ora....P2.inst application    ONLINE    OFFLINE               
ora.COREP.db   application    ONLINE    OFFLINE               
ora....SM1.asm application    ONLINE    ONLINE    nldevrac01  
ora....01.lsnr application    ONLINE    ONLINE    nldevrac01  
ora....c01.gsd application    ONLINE    ONLINE    nldevrac01  
ora....c01.ons application    ONLINE    ONLINE    nldevrac01  
ora....c01.vip application    ONLINE    ONLINE    nldevrac01  
ora....SM2.asm application    ONLINE    OFFLINE               
ora....02.lsnr application    ONLINE    OFFLINE               
ora....c02.gsd application    OFFLINE   OFFLINE               
ora....c02.ons application    ONLINE    OFFLINE               
ora....c02.vip application    ONLINE    ONLINE    nldevrac01  
[root@nldevrac01]:/oracle/crshome1/bin >

When I try to start it by using srvctl, it gives the below error.
[root@nldevrac01]:/oracle/crshome1/bin > srvctl start instance -d corep -i corep1
PRKP-1001 : Error starting instance COREP1 on node nldevrac01
CRS-0215: Could not start resource 'ora.COREP.COREP1.inst'.
[root@nldevrac01]:/oracle/crshome1/bin > 

Then I searched for the solution to this error, I started by checking the related log file to this error. I found the below error in one of the log files.
[oracle@nldevrac01]:/oracle/orahome1/log/nldevrac01/racg > vi imonCOREP.log
...
Oracle Database 10g RACG Release 10.2.0.4.0 Production Copyright 1996, 2005, Oracle.  All rights reserved.
SQL*Plus: Release 10.2.0.4.0 - Production on Tue Mar 6 17:21:34 2012
Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.
Enter user-name: Connected to an idle instance.
SQL> ORA-00119: invalid specification for system parameter LOCAL_LISTENER
ORA-00132: syntax error or unresolved network name 'LISTENER_NLDEVRAC01'
ORA-01078: failure in processing system parameters
SQL> Disconnected

SQL*Plus: Release 10.2.0.4.0 - Production on Tue Mar 6 17:23:31 2012
Copyright (c) 1982, 2007, Oracle.  All Rights Reserved.
Enter user-name: Connected to an idle instance.
SQL> ORACLE instance shut down.
SQL> Disconnected

Searching in the internet for a solution to this error I found the below solution and it worked for me. The problem was that srvctl utility was not aware of the TNS_ADMIN parameter setting in Oracle owner user's ".profile" file.
It's actually set to the below value but srvctl is unaware of it.

[oracle@nldevrac01]:/oracle > echo $TNS_ADMIN
/oracle/asmhome1/network/admin
[oracle@nldevrac01]:/oracle >

Solution :
SRVCTL works from the Oracle Cluster Registry information and does not know the TNS_ADMIN environment setting for the database and instances. Solving this problem is adding the TNS_NAMES environment setting to the Oracle Cluster Registry for the database and instances. The SRVCTL setenv statements to add the attribute TNS_ADMIN to the Oracle Cluster Rergistry have only to be executed from one node only.

Set this parameter
[oracle@nldevrac01]:/oracle > srvctl getenv database -d corep
[oracle@nldevrac01]:/oracle > srvctl setenv db -d corep -t TNS_ADMIN=/oracle/asmhome1/network/admin
[oracle@nldevrac01]:/oracle > srvctl getenv database -d corep
TNS_ADMIN=/oracle/asmhome1/network/admin

Try to start the instance now
[oracle@nldevrac01]:/oracle > srvctl start instance -d corep -i corep1                   

Check the final stat and I saw that the instance on first node and database was started successfully after setting the above parameter, first and third component below.
[oracle@nldevrac01]:/oracle > crs_stat -t
Name           Type           Target    State     Host        
------------------------------------------------------------
ora....P1.inst application    ONLINE    ONLINE    nldevrac01  
ora....P2.inst application    ONLINE    OFFLINE               
ora.COREP.db   application    ONLINE    ONLINE    nldevrac01  
ora....SM1.asm application    ONLINE    ONLINE    nldevrac01  
ora....01.lsnr application    ONLINE    ONLINE    nldevrac01  
ora....c01.gsd application    ONLINE    ONLINE    nldevrac01  
ora....c01.ons application    ONLINE    ONLINE    nldevrac01  
ora....c01.vip application    ONLINE    ONLINE    nldevrac01  
ora....SM2.asm application    ONLINE    OFFLINE               
ora....02.lsnr application    ONLINE    OFFLINE               
ora....c02.gsd application    ONLINE    OFFLINE               
ora....c02.ons application    ONLINE    OFFLINE               
ora....c02.vip application    ONLINE    ONLINE    nldevrac01  
[oracle@nldevrac01]:/oracle > 

This was a quick solution for my case, I hope it can help to other people who will get the same error in their configurations.

4 comments:

Navin Bansal said...
This comment has been removed by the author.
Navin Bansal said...

Hello There,
Thanks for sharing this article. I have also the same issue in my 2 node RAC. I tried the same. For me echo $TNS_ADMIN=/etc
I set the same then i tried this. But still getting the same issue. Do you have another solution for this? I can't reboot CRS as in the same CRS i have running my Prod live DB.
Any help on this would be greatly appreciated.
Thanks
Navin

Enrique Ortiz said...

thanks for sharing these article , it solves us some RAC problems

Ural Ural said...

Hi guys, you're all welcome to my blog page, I am happy if you get anything useful from my blog.

Take care,
Ural