ASM

Allocation Unit and Extents In ASM

ASM was introduced with Oracle 10g and is used as a Volume Manager and a file system. It provides both mirroring and striping of the database files. To use ASM you need to create a Diskgroup and add disks/raw devices to the Diskgroup.Data is allocated in disks in terms of Extents. As per documentation

Every ASM disk is divided into allocation units (AU). An AU is the fundamental unit of allocation within a disk group. A file extent consists of one or more AU. An ASM file consists of one or more file extents.

In case of 10g,ASM divides files into 1 MB extents/AU’s and spreads each file’s extents evenly across all disks in the disk group.

Starting from 11g, we can create Diskgroups of varying AU sizes ranging from 1 MB to 64 MB in powers of two, such as, 1, 2, 4, 8, 16, 32, or 64. Each Allocation Unit consumes some amount of memory in ASM SGA for storing the metadata information. Very Large Databases (VLDB) which can have sizes in TeraBytes will have too many AU’s allocated and also memory requirement for ASM instance will also increase. This will also impact the performance of the application. As a result Variable Size extents concept has been introduced in 11g.

Extents and Allocation Units

Number of Extents Size
0 – 19999 1*AU
20000 – 39999 8*AU
40000 – 59999 64*AU

As shown in table, for first 20000 extents Extent size is always equal to AU. This keeps on increasing gradually.

The ASM coarse striping is always equal to the disk group AU size, but fine striping size always remains 128KB in any configuration . The AU size is determined at creation time with the allocation unit size (AU_SIZE>) disk group attribute. The values can be 1, 2, 4, 8, 16, 32, and 64 MB.

 CREATE DISKGROUP DATA1 EXTERNAL REDUNDANCY DISK '/dev/sda1' ATTRIBUTE 'au_size'='10M';

You can query ALLOCATION_UNIT_SIZE column in V$ASM_DISKGROUP.

Now all this is documented and if I do not mention anything else you will be angry at me 🙂

Now we know this is implemented from 11g. But does that mean it is not available in 10g??

No. It is available in 10g, but we need to play around with some Hidden Parameters.

Warning – You should try this out in Development Environment before trying in Production database. Also contact Oracle Support to validate if this can be implemented in your system and ensure that there are no reported issues/bugs.

<span style="color: #000099;"><span style="color: #3333ff;">SQL&gt; select nam.ksppinm NAME, val.KSPPSTVL VALUE from x$ksppi nam, x$ksppsv val where nam.indx = val.indx and nam.ksppinm like '%asm%';</span></span>

<span style="color: #000099;"><span style="color: #3333ff;">NAME                           VALUE
------------------------------ --------------------
asm_diskstring                 /dev/sda7*
_asm_disk_repair_time          14400
asm_diskgroups                 DATA
asm_power_limit                1
<strong>_asm_ausize                    1048576</strong>
_asm_blksize                   4096
_asm_acd_chunks                1
_asm_libraries                 ufs
_asm_maxio                     1048576
_asm_allow_only_raw_disks      TRUE
_asmlib_test                   0
_asm_allow_resilver_corruption FALSE
_asmsid                        asm
_asm_wait_time                 18
_asm_skip_resize_check         FALSE
_asm_stripewidth               8
<strong>_asm_stripesize                131072</strong>
_asm_droptimeout               60
_asm_emulmax                   10000
_asm_emultimeout               0
_asm_kfdpevent                 0</span>
</span>

We need to look at parameters _asm_ausize and _asm_stripesize which are set to 1 M and 128 K respectively.

In case you wish to create Diskgroup of say 10M Allocation Unit and want to increase the Stripe size to 1M,then set following parameter in pfile

<span style="color: #ff0000;"><strong>_asm_ausize=</strong></span><span style="color: #ff0000;"><strong>10485760
</strong></span><span style="color: #ff0000;"><strong>_asm_stripesize=</strong></span><span style="color: #ff0000;"><strong>1048576</strong></span>

Restart the ASM instance. Please note that these settings will be applicable only for new Diskgroups and will not modify the existing Diskgroups.

You will be required to change the template for the Diskgroup else all the datafiles will use COARSE attribute and will have stripe of 10M. You can find details in 10g SQL reference Guide

20 May, 2008 by 10g 11g 8

ORA-15063 – ASM Discovered Insufficient amount of Disks

Looking at the response generated by my earlier post on ASM, I thought of writing one more article.

Many of us (Working on ASM) would have encountered following errors during startup of ASM instance

ORA-15032: not all alterations performed
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "DATA1"

Corresponding ASM alert log shows

ERROR: no PST quorum in group 1: required 2, found 1
NOTE: cache dismounting group 1/0x94F1292B (DATA1)
NOTE: dbwr not being msg'd to dismount
ERROR: diskgroup DATA1 was not mounted

Like any other error , first approach to solving any “ORA” error should be look at its description

<span style="color: #ff0000;">/home/oracle&gt;oerr ora 15063
15063, 00000, "ASM discovered an insufficient number of disks for diskgroup \"%s\""
// *Cause:  ASM was unable to find a sufficient number of disks belonging to the
//          diskgroup to continue the operation.
// *Action: Check that the disks in the diskgroup are present and functioning,
//          that the owner of the ORACLE binary has read/write permission to
//          the disks, and that the ASM_DISKSTRING initialization parameter
//          has been set correctly.  Verify that ASM discovers the appropriate
//          disks by querying V$ASM_DISK from the ASM instance.
</span>

Now if you see the explanation, it clearly mentions that there are some disks missing in diskgroup which is not allowing the diskgroup to be mounted. It also gives the Action plan to be taken

1)Check that the disks in the diskgroup are present and functioning


To check this you need to query V$ASM_DISK and see if all the disks are visible and also MOUNT_STATUS shows as Member

SQL> SELECT GROUP_NUMBER,DISK_NUMBER,MOUNT_STATUS,HEADER_STATUS,STATE,NAME,PATH FROM V$ASM_DISK;

GROUP_NUMBER DISK_NUMBER MOUNT_S HEADER_STATU STATE    NAME       PATH
------------ ----------- ------- ------------ -------- ---------- --------------------
1           0 CACHED  MEMBER       NORMAL   DATA_0000  /dev/sda7

In case the disks are not member, you will most likely see the GROUP_NUMBER corresponding to that disk as 0.

2)Check that Owner of the ORACLE binary has read/write permission to the disks, and that the ASM_DISKSTRING initialization parameter has been set correctly.


You need to check the permissions for the disk and ensure that Oracle is owner

<span style="color: #3333ff;">$ls -ltr /dev/sda7
brw-rw----  1 <strong>oracle</strong> oinstall 8, 7 May  4 18:43 /dev/sda7</span>

You can also use dd command to see if the disk is accessible by Oracle user.e.g

$ id
uid=100(oracle) gid=100(oinstall)
$ dd if=/dev/rdsk/1 of=/dev/null bs=1024 count=100
dd: /dev/rdsk/1: open: Invalid argument

It should have shown something like

100+0 in
100+0 out

In case you are using ASMLIB . then you can use following command to see if disks are visible

$/etc/init.d/oracleasm listdisks

If it does not display any disk, then check if Oracle is having correct permissions

ls -ltr /dev/oracleasm/disks/*


To Check ASM_DISKSTRING parameter you can check previous post on ASM Disk Discovery

3)Verify that ASM discovers the appropriate disks by querying V$ASM_DISK from the ASM instance.

This is again similar to point 2 as in case the permissions are not set or ASM_DISKSTRING is not correctly set, it will not recognize the disk in V$ASM_DISK.

In addition to this , we can also use Oracle utility called kfed to check the problematic disk and verify if disk is part of ASM diskgroup. Kfed is not part of standard oracle installation and has to be generated. Please find below steps for same

For 10.2 above 

<span style="color: #3333ff;"><strong>$cd $ORACLE_HOME/rdbms/lib
$ make -f ins_rdbms.mk ikfed</strong> </span>

<span style="color: #3333ff;">
</span> <span style="color: #3333ff;">Linking KFED utility (kfed)
rm -f /u01/app/oracle/product/asm10.2/rdbms/lib/kfed
gcc -o /u01/app/oracle/product/asm10.2/rdbms/lib/kfed -L/u01/app/oracle/product/asm10.2/rdbms/lib/ -L/u01/app/oracle/product/asm10.2/lib/ -L/u01/app/oracle/product/asm10.2/lib/stubs/ -L/usr/lib -lirc /u01/app/oracle/product/asm10.2/lib/s0main.o /u01/app/oracle/product/asm10.2/rdbms/lib/sskfeded.o /u01/app/oracle/product/asm10.2/rdbms/lib/skfedpt.o /u01/app/oracle/product/asm10.2/rdbms/lib/defopt.o -ldbtools10 -lclntsh `cat /u01/app/oracle/product/asm10.2/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lnro10 `cat /u01/app/oracle/product/asm10.2/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lclient10 -lnnetd10 -lvsn10 -lcommon10 -lgeneric10 -lmm -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 `cat /u01/app/oracle/product/asm10.2/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lnro10 `cat /u01/app/oracle/product/asm10.2/lib/ldflags` -lnsslb10 -lncrypt10 -lnsgr10 -lnzjs10 -ln10 -lnnz10 -lnl10 -lclient10 -lnnetd10 -lvsn10 -lcommon10 -lgeneric10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 -lclient10 -lnnetd10 -lvsn10 -lcommon10 -lgeneric10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lcore10 -lsnls10 -lnls10 -lxml10 -lcore10 -lunls10 -lsnls10 -lnls10 -lcore10 -lnls10 `cat /u01/app/oracle/product/asm10.2/lib/sysliblist` -Wl,-rpath,/u01/app/oracle/product/asm10.2/lib -lm `cat /u01/app/oracle/product/asm10.2/lib/sysliblist` -ldl -lm -L/u01/app/oracle/product/asm10.2/lib
mv -f /u01/app/oracle/product/asm10.2/bin/kfed /u01/app/oracle/product/asm10.2/bin/kfedO
mv: cannot stat `/u01/app/oracle/product/asm10.2/bin/kfed': No such file or directory
make: [ikfed] Error 1 (ignored)
mv /u01/app/oracle/product/asm10.2/rdbms/lib/kfed /u01/app/oracle/product/asm10.2/bin/kfed
chmod 751 /u01/app/oracle/product/asm10.2/bin/kfed</span>

Now to use kfed, we need to use synatx as kfed read devicename 

<span style="color: #3333ff;">$ kfed read /dev/sda8
kfbh.endian:                          1 ; 0x000: 0x01
kfbh.hard:                          130 ; 0x001: 0x82
kfbh.type:                            1 ; 0x002: KFBTYP_DISKHEAD
kfbh.datfmt:                          1 ; 0x003: 0x01
kfbh.block.blk:                       0 ; 0x004: T=0 NUMB=0x0
kfbh.block.obj:              2147483648 ; 0x008: TYPE=0x8 NUMB=0x0
kfbh.check:                  2083552713 ; 0x00c: 0x7c307dc9
kfbh.fcn.base:                        0 ; 0x010: 0x00000000
kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000
kfbh.spare1:                          0 ; 0x018: 0x00000000
kfbh.spare2:                          0 ; 0x01c: 0x00000000
kfdhdb.driver.provstr:         ORCLDISK ; 0x000: length=8
kfdhdb.driver.reserved[0]:            0 ; 0x008: 0x00000000
kfdhdb.driver.reserved[1]:            0 ; 0x00c: 0x00000000
kfdhdb.driver.reserved[2]:            0 ; 0x010: 0x00000000
kfdhdb.driver.reserved[3]:            0 ; 0x014: 0x00000000
kfdhdb.driver.reserved[4]:            0 ; 0x018: 0x00000000
kfdhdb.driver.reserved[5]:            0 ; 0x01c: 0x00000000
kfdhdb.compat:                168820736 ; 0x020: 0x0a100000
kfdhdb.dsknum:                        0 ; 0x024: 0x0000
<strong>kfdhdb.grptyp:                        1 ; 0x026: KFDGTP_EXTERNAL
kfdhdb.hdrsts:                        3 ; 0x027: KFDHDR_MEMBER</strong>
</span> <strong><span style="color: #3333ff;">kfdhdb.dskname:               DATA_0000 ; 0x028: length=9
kfdhdb.grpname:                    DATA ; 0x048: length=4
kfdhdb.fgname:                DATA_0000 ; 0x068: length=9</span>
</strong>

I have made the important things as Bold (You can also say, things which I know. For rest need to check with Oracle Support 🙂 )
Lets verify the result with output from V$ASM_DISK and V$ASM_DISKGROUP

<span style="color: #3333ff;">select DG.GROUP_NUMBER "G.NO",DG.NAME,D.DISK_NUMBER,D.MOUNT_STATUS,
D.HEADER_STATUS,DG.TYPE,D.NAME,D.PATH FROM V$ASM_DISK D,V$ASM_DISKGROUP DG 
where DG.GROUP_NUMBER=D.GROUP_NUMBER;</span>

<span style="color: #3333ff;">
</span> <span style="color: #3333ff;"> G.NO NAME       DISK_NUMBER MOUNT_S HEADER_STATU TYPE   NAME       PATH
---------- ---------- ----------- ------- ------------ ------ ---------- ----------
1 DATA                 0 CACHED  MEMBER       EXTERN DATA_0000  /dev/sda8</span>

kfdhdb.grptyp: 1 ; 0x026: KFDGTP_EXTERNAL ->This indicates Redundancy for Group.Check TYPE in query output.


kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER – > This indicates Disk Header status. Here it indicates it is member of Group.

kfdhdb.dskname: DATA_0000 ; 0x028: length=9 -> This indicates Disk Name
kfdhdb.grpname: DATA ; 0x048: length=4 ->This indicates the Group Name for the disk.

kfdhdb.fgname: DATA_0000 ; 0x068: length=9 ->This indicates the Failure Group Name.


Going back to error, we saw

ERROR: no PST quorum in group 1: required 2, found 1

Here PST means Partnership Status Table. PST contains list of disks(partners) which are required to mount the diskgroup. In case you are using External Redundancy and ASM is not able to find one disk , then the diskgroup will fail to mount.

You can read more about PST in book Oracle Automatic Storage Management: Under-the-Hood & Practical Deployment Guide (Oracle)
by Nitin Vengurlekar (Author), Murali Vallath (Author), Rich Long

11g Update on Kfed

You are not required to generate kfed on 11g installation. It is part of the default oracle installation.
18 May, 2008 by 10g 5

ASM Disk Discovery

While creating ASM diskgroup or adding new disk to a existing diskgroup, Disk should be visible in V$ASM_DISK.

ASM discovers and examines the contents of all of the disks that are in the paths that you designated with values in the ASM_DISKSTRING initialization parameter.

As per Oracle Docs, Disk discovery also occurs when you:

– Run the ALTER DISKGROUP…ADD DISK and ALTER DISKGROUP…RESIZE DISK commands

– Query the V$ASM_DISKGROUP and V$ASM_DISK views

Note: – You should try to use V$ASM_DISK_STAT to get faster results as access to this view does not lead to Disk Discovery.

While creating ASM Diskgroup, Disk Discovery is the most common issue. This is mostly due to Disk permission issue or incorrect setting for ASM_DISKSTRING parameter.

So as to check this , you can use KFOD utility provided by Oracle.

This can be found in $ASM_HOME/bin directory. Help can be seen using

 oracle@asm]/home/oracle&gt; kfod help=y

<span style="font-size:small;"><span style="font-family:Times New Roman;">_asm_a/llow_only_raw_disks<span>              </span>KFOD allow only raw devices [_asm_allow_</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">only_raw_disks=TRUE/(FALSE)]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">_asm_l/ibraries<span>         </span>ASM Libraries[_asm_libraries='lib1','lib2',...]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">_asms/id<span>                </span>ASM Instance[_asmsid=sid]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">a/sm_diskstring<span>         </span>ASM Diskstring [asm_diskstring='discoverystring', 'disco</span></span><span style="font-size:small;"><span style="font-family:Times New Roman;">verystring' ...]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">d/isks<span>          </span>Disks to discover [disks=raw,asm,all]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">g/roup<span>          </span>Group discover [group=controlfile]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">n/ohdr<span>          </span>KFOD header suppression [nohdr=TRUE/(FALSE)]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">o/p<span>             </span>KFOD options type [OP=DISKS/GROUPS/ALL]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">p/file<span>          </span>ASM parameter file [pfile='parameterfile']</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">s/tatus<span>    </span><span>     </span>Include disk header status [status=TRUE/(FALSE)]</span></span>

<span style="font-size:small;"><span style="font-family:Times New Roman;">v/erbose<span>                </span>KFOD verbose errors [verbose=TRUE/(FALSE)]</span></span>

<span style="font-size:small;font-family:Times New Roman;">KFOD-01000: file not found</span>

 

To discover the disks , you need to use disks=all clause

 

[oracle@asm] /home/oracle&gt;kfod disks=all</span>
kfod disks=all
--------------------------------------------------------------------------------
 Disk          Size Path
==================================================================
   1:     345648 Mb /dev/sda7
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
==================================================================
     +ASM2 /u01/app/oracle/product/asm10.2
     +ASM1 /u01/app/oracle/product/asm10.2

As seen from help menu, we can use raw and asm as option. One more important parameter which can be used is asm_diskstring to make sure that problem is not with initialization parameter.

[oracle@asm]/home/oracle>kfod asm_diskstring='/raw/*' disks=all
--------------------------------------------------------------------------------
ORACLE_SID ORACLE_HOME
====================================================================
     +ASM2 /u01/app/oracle/product/asm10.2
     +ASM1 /u01/app/oracle/product/asm10.2

By default asm_diskstring looks in all the directories.

9 May, 2008 by 10g 11g 2

Disclaimer!!

This blog reflect our own views and do not necessarily represent the views of our current or previous employers.

The contents of this blog are from our experience, you may use at your own risk, however you are strongly advised to cross reference with Product documentation and test before deploying to production environments.

Subscribe to Blog via Email

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 188 other subscribers