UNKNOWN State Of RAC Resources

While Checking the status of database resources, ASM was shown as UNKNOWN on one node of a two node RAC.

$ crs_stat -t

Name           Type           Target    State     Host
------------------------------------------------------------
ora.orcl.db    application    ONLINE    ONLINE    rac1
ora....11.inst application    ONLINE    ONLINE    rac1
ora....SM1.asm application    ONLINE    ONLINE    rac1
ora....DC.lsnr application    ONLINE    ONLINE    rac1
ora....idc.gsd application    ONLINE    ONLINE    rac1
ora....idc.ons application    ONLINE    ONLINE    rac1
ora....idc.vip application    ONLINE    ONLINE    rac1
ora....SM2.asm application    ONLINE    UNKNOWN    rac2
ora....C2.lsnr application    ONLINE    ONLINE    rac2
ora....dc2.gsd application    ONLINE    ONLINE    rac2
ora....dc2.ons application    ONLINE    ONLINE    rac2
ora....dc2.vip application    ONLINE    ONLINE    rac2

Following error was coming while trying to start the +ASM2 instance with SRVCTL:

$srvctl start asm -n rac2

PRKS-1009 : Failed to start ASM instance "+ASM2" on node "rac2",
[CRS-0223: Resource 'ora.rac2.ASM2.asm' has placement error.]

While trying to start the same with crs_start :

$ crs_start -f ora.rac2.ASM2.asm

CRS-1028: Dependency analysis failed because of:
'Resource in UNKNOWN state: ora.rac2.ASM2.asm'
CRS-0223: Resource 'ora.rac2.ASM2.asm' has placement error

There are two ways to come out of this UNKNOWN state of resources:
1. Start the resource from sqlplus
2. Use crs_stop -f to clear the state of database resources.

$ export ORACLE_HOME=+ASM2
$ sqlplus "/ as sysdba"
SQL>startup 
Diskgroup mounted

It will go fine and the +ASM2 instnace will be started.

$ crs_stop -f ora.rac2.ASM2.asm

This will clear the UNKNOWN state and will make the resource as OFFLINE.

Now start the resource as:

$ srvctl start asm -n rac2

After using this check the status :

$ crs_stat -t

In case of listener resource, if starting listener using srvctl results in following error

CRS-0215: Could not start resource 'ora.dev-101.LISTENER_DEV-101.lsnr'.

This can be resolved by removing listener resource and adding it back. Perform following action using root user

#crs_unregister ora.dev-101.LISTENER_DEV-101.lsnr
#crs_unregister ora.dev-102.LISTENER_DEV-102.lsnr

Then recreate the listener using silent mode as oracle user

$netca /silent /responsefile $ORACLE_HOME/network/install/netca_typ.rsp /nodeinfo dev-101,dev-102

Above command can result in error like below

Exception in thread "main" java.lang.UnsatisfiedLinkError: /home/oracle/product/10.2/jdk/jre/lib/i386/libawt.so: libXp.so.6: cannot open shared object file: No such file or directory
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1586)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1503)
	at java.lang.Runtime.loadLibrary0(Runtime.java:788)
	at java.lang.System.loadLibrary(System.java:834)
	at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:50)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.awt.NativeLibLoader.loadLibraries(NativeLibLoader.java:38)
	at sun.awt.DebugHelper.(DebugHelper.java:29)
	at java.awt.Component.(Component.java:506)

This can be resolved by installing xorg-x11-deprecated-libs rpm. (yum install xorg-x11-deprecated-libs)

One thought on “UNKNOWN State Of RAC Resources

Leave a Reply