11g

11.1.0.6 ASM installation on Solaris fails -II

Some time back, I had written about CSS service not starting in my post 11.1.0.6 ASM installation on Solaris fails -I

After doing some research, I came across Metalink Document

Note:397238.1 – How to Convert init.cssd as a SMF service for Solaris 10

This document talks about using Service Management Facility (SMF) which was introduced in Solaris 10. To configure it we have to download a zip file from the note and copy two files called initcssd to /lib/svc/method/initcssd and copy the second file initcssd.xml to /var/svc/manifest/site and some other steps (as listed in doc)

1) Install Oracle Software on Solaris 10

2) Download files from Note: 397238.1. Once done, modify the files accordingly and copy to the required location.

3) Do the configuration and then enable the service.

<strong># svcadm -v enable initcssd</strong>

After doing this, still service does not start. 

# ps -ef|grep css
    root 29137  3793   0 14:54:39 ?           0:00 /bin/sh /lib/svc/method/initcssd run
    root 29188 26874   0 14:54:50 pts/9       0:00 grep css

Note talks about checking the content of content of file /var/opt/oracle/scls_scr/<Your-hostname>/root/cssrun file.

But when I tried to check, I found that directory does not exists.

# cd /var/opt/oracle/scls_scr/
 cd: /var/opt/oracle/scls_scr/: No such file or directory
# cd /u03/app/oracle/product/11.1.0/db_1/bin/
<strong># ./crsctl start crs</strong>
Attempting to start Oracle Clusterware stack
Failure at scls_scr_create with code 1
Segmentation Fault (core dumped)

Actually this directory is created when we run ‘localconfig add’ which will configure the socket files and directories. 

<strong># ps -ef|grep css</strong>
    root 29137  3793   0 14:54:39 ?           0:00 /bin/sh /lib/svc/method/initcssd run
# pwd
/u03/app/oracle/product/11.1.0/db_1/bin
<strong># ./localconfig add</strong>
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Configuration for local CSS has been initialized

Cleaning up Network socket directories
Setting up Network socket directories
Adding to inittab
Startup will be queued to init within 30 seconds.
Checking the status of new Oracle init process...
Expecting the CRS daemons to be up within 600 seconds.
Cluster Synchronization Services is active on these nodes.
        testzone2
Cluster Synchronization Services is active on all the nodes.
Oracle CSS service is installed and running under init(1M)
<strong># ps -ef|grep css</strong>
  oracle 29137  3793   0 14:54:39 ?           0:00 /u03/app/oracle/product/11.1.0/db_1/bin/ocssd.bin

Now just for fun, I thought of disabling the SMF and trying the configuration again. 

# ./localconfig delete
Stopping Cluster Synchronization Services.
Shutting down the Cluster Synchronization Services daemon.
Shutdown request successfully issued.
Shutdown has begun. The daemons should exit soon.
Cleaning up Network socket directories

Disable the SMF
# svcadm -v disable initcssd
svc:/system/initcssd:default disabled.

Now again add CSS service

# ./localconfig add
Successfully accumulated necessary OCR keys.
Creating OCR keys for user \'root\', privgrp \'root\'..
Operation successful.
Configuration for local CSS has been initialized

Cleaning up Network socket directories
Setting up Network socket directories
Adding to inittab
Startup will be queued to init within 30 seconds.
Checking the status of new Oracle init process...
Expecting the CRS daemons to be up within 600 seconds.

Giving up: Oracle CSS stack appears NOT to be running.
Oracle CSS service would not start as installed
Automatic Storage Management(ASM) cannot be used until Oracle CSS service is started

Enable the initcssd SMF service

<strong># svcadm -v enable initcssd</strong>
svc:/system/initcssd:default enabled.
# ps -ef|grep css
  oracle  2589  3793   1 15:14:31 ?           0:00 /u03/app/oracle/product/11.1.0/db_1/bin/ocssd.bin

Cool!! CSS Service has started again 🙂

Note:397238.1 says that problem occurs only after rebooting the service and not during installation. May be document needs to be updated for Oracle 11g on Solaris 10. I would suggest anyone installing Oracle 11g on Solaris 10 should try using these steps. I will try to add a remark to the note. Apart from that Article is quite good.

24 June, 2008 by 11g 5

11.1.0.6 ASM installation on Solaris fails -I

Looks like that there is no easy way for me to install/configure Oracle components. And every install/configuration leads me to some or the other problems. Anyways I was trying to install Oracle Database 11g on a Solaris 10 but the CSS does not seems to come up. While issuing localconfig add as root user it gives following message and fails

-bash-3.00# ./localconfig add
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Configuration for local CSS has been initialized

Cleaning up Network socket directories
Setting up Network socket directories
Adding to inittab
Startup will be queued to init within 30 seconds.
Checking the status of new Oracle init process...
Expecting the CRS daemons to be up within 600 seconds.

Giving up: Oracle CSS stack appears NOT to be running.
Oracle CSS service would not start as installed
Automatic Storage Management(ASM) cannot be used until Oracle CSS service is started

On checking further for css log messages in $ORACLE_HOME/log/hostname/client , I found following messages

Oracle Database 11g CRS Release 11.1.0.6.0 - Production Copyright 1996, 2007 Oracle. All rights reserved.
2008-06-18 21:38:29.721: [ CSSCLNT][1]clsssInitNative: failed to connect to (ADDRESS=(PROTOCOL=ipc)(<strong>KEY=OCSSD_LL_test1zone2_</strong>))
, rc 9

On checking the /var/adm/messages file , found following errors

Jun 18 01:16:23 test1zone2 root: [ID 702911 user.error] Oracle Cluster Synchronization Service starting by user request.
Jun 18 01:16:25 test1zone2 root: [ID 702911 user.error] Cluster Ready Services completed waiting on dependencies.
Jun 18 01:16:35 test1zone2 last message repeated 9 times

/var/tmp/.oracle does not show any files created after running localconfig add. These are called Socket files and CSS uses these for communication.I suspect this to be a issue.
I checked our Linux machine (with RAC) and found that we have some files and one of them is named as

srwxrwxrwx  1 oracle oinstall 0 Jun 11 22:10 <strong>sOCSSD_LL_prod01_</strong>

This is quite similar to the error message in css.log files i.e KEY=OCSSD_LL_test1zone2_

As this is a test machine so I cant raise a ticket with Oracle. Currently posted a thread on Oracle Forum. Let’s see if someone is able to figure out something. I will keep you posted. If anyone is interested in reading how these sockets work then they can visit this article from Frits Hoogland.

18 June, 2008 by 11g 3

Allocation Unit and Extents In ASM

ASM was introduced with Oracle 10g and is used as a Volume Manager and a file system. It provides both mirroring and striping of the database files. To use ASM you need to create a Diskgroup and add disks/raw devices to the Diskgroup.Data is allocated in disks in terms of Extents. As per documentation

Every ASM disk is divided into allocation units (AU). An AU is the fundamental unit of allocation within a disk group. A file extent consists of one or more AU. An ASM file consists of one or more file extents.

In case of 10g,ASM divides files into 1 MB extents/AU’s and spreads each file’s extents evenly across all disks in the disk group.

Starting from 11g, we can create Diskgroups of varying AU sizes ranging from 1 MB to 64 MB in powers of two, such as, 1, 2, 4, 8, 16, 32, or 64. Each Allocation Unit consumes some amount of memory in ASM SGA for storing the metadata information. Very Large Databases (VLDB) which can have sizes in TeraBytes will have too many AU’s allocated and also memory requirement for ASM instance will also increase. This will also impact the performance of the application. As a result Variable Size extents concept has been introduced in 11g.

Extents and Allocation Units

Number of Extents Size
0 – 19999 1*AU
20000 – 39999 8*AU
40000 – 59999 64*AU

As shown in table, for first 20000 extents Extent size is always equal to AU. This keeps on increasing gradually.

The ASM coarse striping is always equal to the disk group AU size, but fine striping size always remains 128KB in any configuration . The AU size is determined at creation time with the allocation unit size (AU_SIZE>) disk group attribute. The values can be 1, 2, 4, 8, 16, 32, and 64 MB.

 CREATE DISKGROUP DATA1 EXTERNAL REDUNDANCY DISK '/dev/sda1' ATTRIBUTE 'au_size'='10M';

You can query ALLOCATION_UNIT_SIZE column in V$ASM_DISKGROUP.

Now all this is documented and if I do not mention anything else you will be angry at me 🙂

Now we know this is implemented from 11g. But does that mean it is not available in 10g??

No. It is available in 10g, but we need to play around with some Hidden Parameters.

Warning – You should try this out in Development Environment before trying in Production database. Also contact Oracle Support to validate if this can be implemented in your system and ensure that there are no reported issues/bugs.

<span style="color: #000099;"><span style="color: #3333ff;">SQL&gt; select nam.ksppinm NAME, val.KSPPSTVL VALUE from x$ksppi nam, x$ksppsv val where nam.indx = val.indx and nam.ksppinm like '%asm%';</span></span>

<span style="color: #000099;"><span style="color: #3333ff;">NAME                           VALUE
------------------------------ --------------------
asm_diskstring                 /dev/sda7*
_asm_disk_repair_time          14400
asm_diskgroups                 DATA
asm_power_limit                1
<strong>_asm_ausize                    1048576</strong>
_asm_blksize                   4096
_asm_acd_chunks                1
_asm_libraries                 ufs
_asm_maxio                     1048576
_asm_allow_only_raw_disks      TRUE
_asmlib_test                   0
_asm_allow_resilver_corruption FALSE
_asmsid                        asm
_asm_wait_time                 18
_asm_skip_resize_check         FALSE
_asm_stripewidth               8
<strong>_asm_stripesize                131072</strong>
_asm_droptimeout               60
_asm_emulmax                   10000
_asm_emultimeout               0
_asm_kfdpevent                 0</span>
</span>

We need to look at parameters _asm_ausize and _asm_stripesize which are set to 1 M and 128 K respectively.

In case you wish to create Diskgroup of say 10M Allocation Unit and want to increase the Stripe size to 1M,then set following parameter in pfile

<span style="color: #ff0000;"><strong>_asm_ausize=</strong></span><span style="color: #ff0000;"><strong>10485760
</strong></span><span style="color: #ff0000;"><strong>_asm_stripesize=</strong></span><span style="color: #ff0000;"><strong>1048576</strong></span>

Restart the ASM instance. Please note that these settings will be applicable only for new Diskgroups and will not modify the existing Diskgroups.

You will be required to change the template for the Diskgroup else all the datafiles will use COARSE attribute and will have stripe of 10M. You can find details in 10g SQL reference Guide

20 May, 2008 by 10g 11g 8

11G: OCR Manual Backups

OCR is a file that manages cluster and Oracle Real Application Clusters (Oracle RAC) database configuration information. The OCR contains information about the cluster node list, instance-to-node mapping information, and information about Oracle Clusterware resource profiles for applications

This is a Very important component along with Voting Disk and Loss of it will require you to re-install the Clusterware Software.

Oracle 10g provides Automatic Backup of OCR along with manually export of OCR files.

Starting from Oracle 11g, we can also take manual backups of OCR. So in 11g , we can have following backups

1) Automatic backups—Oracle Clusterware automatically creates OCR backups every four hours. At any one time, Oracle always retains the last three backup copies of the OCR. The CRSD process that creates the backups also creates and retains an OCR backup for each full day and at the end of each week. You cannot customize the backup frequencies or the number of files that Oracle retains.

We can view the backups using ocrconfig -showbackup auto command.

[oracle@prod01]/home/oracle>ocrconfig -showbackup auto
prod01     2008/05/14 19:23:22     /u01/app/11.1.0/crs/cdata/prod_cluster/backup00.ocr
prod01     2008/05/14 15:23:21     /u01/app/11.1.0/crs/cdata/prod_cluster/backup01.ocr
prod01     2008/05/14 11:23:21     /u01/app/11.1.0/crs/cdata/prod_cluster/backup02.ocr
prod01     2008/05/13 15:23:20     /u01/app/11.1.0/crs/cdata/prod_cluster/day.ocr
prod01     2008/05/04 01:52:33     /u01/app/11.1.0/crs/cdata/prod_cluster/week.ocr

2)Manual backups—You can use the ocrconfig -manualbackup command to force Oracle Clusterware to perform a backup of the OCR at any time, rather than wait for the automatic backup that occurs at 4-hour intervals. The -manualbackup option is especially useful when you to need to obtain a binary backup on demand, such as before you make changes to the OCR.

You need to be logged in as root user to take manual backups. Use ocrconfig –manualbackup command. To view the backups use ocrconfig –showbackup manual command


[oracle@prod01]/home/oracle>ocrconfig -showbackup manual
prod01     2008/05/01 12:21:29     /u01/app/11.1.0/crs/cdata/prod_cluster/backup_20080501_122129.ocr
prod02     2008/04/11 11:32:50     /u01/app/11.1.0/crs/cdata/prod_cluster/backup_20080411_113250.ocr
prod02     2008/03/27 17:25:23     /u01/app/11.1.0/crs/cdata/prod_cluster/backup_20080327_172523.ocr

3)Manual Export—You can Use the ocrconfig -export file_name command to export the OCR content to a file format.

For more details refer to 11g Documentation

15 May, 2008 by 11g 0

ASM Disk Discovery

While creating ASM diskgroup or adding new disk to a existing diskgroup, Disk should be visible in V$ASM_DISK.

ASM discovers and examines the contents of all of the disks that are in the paths that you designated with values in the ASM_DISKSTRING initialization parameter.

As per Oracle Docs, Disk discovery also occurs when you:

– Run the ALTER DISKGROUP…ADD DISK and ALTER DISKGROUP…RESIZE DISK commands

– Query the V$ASM_DISKGROUP and V$ASM_DISK views

Note: – You should try to use V$ASM_DISK_STAT to get faster results as access to this view does not lead to Disk Discovery.

While creating ASM Diskgroup, Disk Discovery is the most common issue. This is mostly due to Disk permission issue or incorrect setting for ASM_DISKSTRING parameter.

So as to check this , you can use KFOD utility provided by Oracle.

This can be found in $ASM_HOME/bin directory. Help can be seen using

 oracle@asm]/home/oracle&gt; kfod help=y

<span style="font-size:small;"><span style="font-family:Times New Roman;">_asm_a/llow_only_raw_disks<span>              </span>KFOD allow only raw devices [_asm_allow_</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">only_raw_disks=TRUE/(FALSE)]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">_asm_l/ibraries<span>         </span>ASM Libraries[_asm_libraries='lib1','lib2',...]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">_asms/id<span>                </span>ASM Instance[_asmsid=sid]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">a/sm_diskstring<span>         </span>ASM Diskstring [asm_diskstring='discoverystring', 'disco</span></span><span style="font-size:small;"><span style="font-family:Times New Roman;">verystring' ...]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">d/isks<span>          </span>Disks to discover [disks=raw,asm,all]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">g/roup<span>          </span>Group discover [group=controlfile]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">n/ohdr<span>          </span>KFOD header suppression [nohdr=TRUE/(FALSE)]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">o/p<span>             </span>KFOD options type [OP=DISKS/GROUPS/ALL]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">p/file<span>          </span>ASM parameter file [pfile='parameterfile']</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">s/tatus<span>    </span><span>     </span>Include disk header status [status=TRUE/(FALSE)]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">v/erbose<span>                </span>KFOD verbose errors [verbose=TRUE/(FALSE)]</span></span>

<span style="font-size:small;font-family:Times New Roman;">KFOD-01000: file not found</span>

 

To discover the disks , you need to use disks=all clause

 

[oracle@asm] /home/oracle&gt;kfod disks=all</span>
kfod disks=all
--------------------------------------------------------------------------------
 Disk          Size Path
==================================================================
   1:     345648 Mb /dev/sda7
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
==================================================================
     +ASM2 /u01/app/oracle/product/asm10.2
     +ASM1 /u01/app/oracle/product/asm10.2

As seen from help menu, we can use raw and asm as option. One more important parameter which can be used is asm_diskstring to make sure that problem is not with initialization parameter.

[oracle@asm]/home/oracle>kfod asm_diskstring='/raw/*' disks=all
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
====================================================================
     +ASM2 /u01/app/oracle/product/asm10.2
     +ASM1 /u01/app/oracle/product/asm10.2

By default asm_diskstring looks in all the directories.

9 May, 2008 by 10g 11g 2

Disclaimer!!

This blog reflect our own views and do not necessarily represent the views of our current or previous employers.

The contents of this blog are from our experience, you may use at your own risk, however you are strongly advised to cross reference with Product documentation and test before deploying to production environments.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 188 other subscribers