database

ORA-00600 [12333]

ORA-00600 with [12333] is an error that I have encountered on multiple occasions. This post is to shed some light on what this error is all about.

To start with let us try to understand when does oracle report ora-00600 error. An ora-00600 error is raised in the exception handler section of Oracle’s c-program code. i.e. Coders have forseen certain exceptional situation that are potential threats to data integrity or memory integrity and written appropriate exceptional handlers to report a ora-00600 error with appropriate information about the exception condition.

Details like [12333] indicate or provide hints as to what caused the exception. ora-00600 [12333] is reported when the server recieves data from a client and the server cannot recognize the data format. This error mostly is because of issues with network and does not indicate any data corruption.

Few Common Causes & Suggestions:
1. An incompatible client software can cause such an internal error. Metalink document 207303.1 explains the supported combinations of clients and Server release.

2. An incompatible NLS settings on Client. Check NLS settings especially ORA_NLS33 for 9i, ORA_NLS10 for 10g.

3. Apart from this, it could be because of network issues, client software, TIMEOUT setting, etc. Try to narrow down the error to a particular client machine/ client software.

4. This can be because of an Oracle bug also. Refer to metalink document 428629.1 for list of known bugs with ora-00600 [12333].

As a last resort you can check this with Oracle Support for any new issues.

20 May, 2008 by Leo 10g 0

10.2.0.4 on AIX5L (64-Bit) is Out

As usual Laurent has been first to locate that 10.2.0.4 on AIX5L (64-Bit) is Out.

Patchset size is whopping 1.9G which I believe is the maximum size for Oracle Patchset till date. Looks like 10.2.0.3 on AIX had lot of bugs or it could be that DATABASE REPLAY feature needed more lines of code in AIX 🙂

I am not sure why Oracle has named this feature as DATABASE REPLAY on 10.2.0.4. It should have been named as DATABASE CAPTURE.

Anyways I was trying to be the first person to post release of 10.2.0.4 Patchset , but I guess Laurent has written some Package which sends a SMS that patchset is out 🙂 I need to start learning PL/SQL.

17 May, 2008 by Amit Bansal 10g 3

ORA-07445 [opidsa()+480] – Metalink Alert for 10.2.0.3

Oracle has reported a new Alert for user’s who have upgraded their databases to 10.2.0.3 patchset on windows.

User processes dumps core dumps with following errors

ORA-07445: exception encountered: core dump [<strong>opidsa()</strong>+480]
[SIGSEGV] [Address not mapped to object] [0x000000000] [] []

ORA-07445: exception encountered: core dump [ACCESS_VIOLATION]
 [_opidsa+360] [PC:0x2080540] [ADDR:0x0] [UNABLE_TO_READ] []

Patch 5648872 can be applied to resolve the issue. For more information, refer to
Metalink Note:418531.1-ORA-7445[opidsa] after Applying the 10.2.0.3 Patch Set

This issue is fixed in 10.2.0.4 Patchset

13 May, 2008 by Amit Bansal 10g 0

Significance of LD_LIBRARY_PATH

Few days back, I wanted to remove a instance semaphore as they were not cleared automatically due to abnormal shutdown. As there were few other databases running on the server , so I thought of capturing the semaphore id to remove them manually using “ipcrm” .
For this purpose, we are required to use sysresv utility. Upon execution of the command, I received following errors

$/u01/app/oracle/product/10.2.0/bin>sysresv
sysresv: error while loading shared libraries: libclntsh.so.10.1: cannot open shared object file: No such file or directory

Now this was a new installation so I was bit scared if there was something wrong with the installation. Searching the web pointed me to articles which mentioned about using user which is not in correct group.
But in my case it was properly set and part of dba/oinstall group. Further research led me to Puschitz webpage (Very useful site if you are planning to install Database)which described that this problem is caused when environment variable LD_LIBRARY_PATH is not set.
So I set the value of LD_LIBRARY_PATH and then tried again

<strong>$</strong>export LD_LIBRARY_PATH=$ORACLE_HOME/lib
<strong>$</strong>sysresv
IPC Resources for ORACLE_SID "orcl10" :
Shared Memory:
ID KEY
131074 0x0291b1b8
Semaphores:
ID KEY
229377 0xc116e83c
262146 0xc116e83d
294915 0xc116e83e
Oracle Instance alive for sid "orcl10"

This worked !!!
My issue was solved but I must admit that I was not aware of what this variable did. I had seen this Environment variable set in my previous company, but somehow I didn’t give importance to it.
Searching on web, I came across Oracle Forum thread
which stated that
“The LD_LIBRARY_PATH environment variable contains a colon-separated list of directories in which the linker will search for library (.a) files. If you want the linker to search additional libraries, you can add their names to LD_LIBRARY_PATH.”

One more article gave insight into how application looks for library files.

http://docsun.cites.uiuc.edu/sun_docs/C/solaris_9/SUNWdev/LLM/p4.html
The runtime linker only looks in one default location for dependencies. This location is /usr/lib when processing 32-bit objects, and /usr/lib/64 when processing 64-bit objects. All other directories to be searched must be added to the runtime linker’s search path explicitly.
On searching metalink for further information on this parameter, I came across

Subject: Solaris: Setting LD_LIBRARY_PATH in 10gR1 and higher
Doc ID: Note:271451.1

According to this note, there are two more variables LD_LIBRARY_PATH_32 and LD_LIBRARY_PATH_64 which can be set to point to 32-Bit and 64-Bit libraries.
Further it mentioned that it is Not Recommended to set LD_LIBRARY_PATH unless application (Oracle or 3rd-party) requires it and recommended using LD_LIBRARY_PATH_32 and LD_LIBRARY_PATH_64.
This is done so that 32-Bit programs can use the correct libraries and similarly 64-bit programs can use 64-Bit library files.

So next time, you receive some library error, then do not panic 🙂 and see if parameter LD_LIBRARY_PATH or LD_LIBRARY_PATH_32 or LD_LIBRARY_PATH_64 is set.

Note:- In case you are using non-oracle user and see that you are getting same error after setting the variable,then check if the file ibclntsh.so.10.1exists and non-oracle user’s have read/execute permission over it. If no then relink Oracle using relink all command

12 May, 2008 by Amit Bansal 10g 11g 0

CPU (Critical Patch Update) and N-apply

OVERVIEW

Oracle has been regularly releasing security fixes for all it’s products to take care of any security vulnerability. Since 2005 Oracle formulated a policy that it will release CPU patch ( bundle of security fixes) every quarter (January, April, July, and October).

CPU includes fixes for various Oracle products including Database. Initially Oracle decided to include security fixes, fixes that conflict with security fix and fixes for bugs that affect wide range of users.

With each CPU Oracle provides a brief overview of fixes included, but does not elaborate on the vulnerabilities as this information could be misused. CPU patches are applied using Opatch utility.

CONFLICTS – SUPER SET – SUBSET

Before moving any further let us understand what a patch conflict is. As we all know, Oracle is bunch of c-programs. Any patch makes one or more changes to the c program files. Two patches can make changes to the same or different sets of c-code. When a patch shares one or more files with an existing patch then it is known as CONFLICT.

If a old patch makes all the changes to the c-code as compared to the new patch plus a set of additional changes, then the old patch is the SUPERSET of new patch. On the contrary if an existing patch contains only some of the changes compared to the new patch being installed, then it is called a SUBSET. While apply any new patch, Opatch checks for these 3 conditions.

In case of a superset situation, Opatch does not install the new patch and in case of subset old patch is rolled back and new patch is installed. When opatch detects a conflict scenario, user can decide which patch to keep and which patch to rollback.

N-APPLY

While applying a cpu patch, users can face a patch conflict with an all ready applied patch. You can request Oracle for a merge patch. CPU patches are cumulative i.e. fixes included in Jan fix will be part of April fix.

To take care of all the conflicts reported by various users, Oracle releases CPU super merge patches mid-way between 2 CPU patch releases. CPU super merge patches are merge of a CPU patch and all the reported patches conflicting with a given CPU patch.

This entire process was time consuming. Conflicts also prevent application of CPU patches. To overcome this, Oracle has come up with a new concept of packaging cpu patches for 10.2.0.3 (not all platforms) and upwards since July 2007. With this new process Oracle has stopped including non-security fixes in CPUs. Oracle has come up with N-apply concept.

With the new N-apply concept, a cpu is divided into one or more molecules. Molecule is set of one or more bug fixes. This feature is specially use full when users encounter conflicts. Conflicts can be resolved at the molecular level. This provides us a technique to apply a part of cpu fixes while patch conflicts are resolved.

To make use of the n-apply feature ensure that you have latest version of opatch, run the command:

opatch napply -skip_subset -skip_duplicate

OPatch parameter -skip_subset, which is specified at CPU patch install, enables OPatch to determine that a merge patch that includes a molecule is already installed in the Oracle home, and to then skip application of the molecule patch.

OPatch parameter, -skip_duplicate, provides the additional benefit of detecting when a molecule patch has already been applied, as in the case of a previous CPU, and to skip application of it. This reduces the length of time required to do the n-apply CPU installation and minimizes the overall change to the Oracle home.

4 May, 2008 by Leo 10g 1

Understanding of Oracle 11g Automatic Diagnostic Repository – Part 1

INTRODUCTION

In an endeavour to move towards self managed database and provide better support solutions, Oracle 11g introduces another new feature in 11g called the Automatic Diagnostic Repository or ADR. As the name implies ADR is a automatically managed file based repository of diagnostic information such as trace files, core dumps, alert log, etc… ADR is like a mini database managing traces of multiple instances and multiple prouducts (database, ASM, CRS, etc…) with metadata of these traces being maintained under a unified directory structure. ADR promises to use consistent diagnostic data structures across products. Set of tools bundled with ADR facilitate us in analyzing diagnostic data.

As an outcome of this feature:
– Critical errors captured in the diagnostic repository show up as alerts in EM. In cases where the issue is a known issue, Oracle recommends corrective actions to be implemented by DBA.
– It is intended to make it easy for DBAs to consolidate and transfer diagnostic data to Oracle support, thus facilitating faster resolution of the issue.

ARCHITECTURE

Starting with Oracle 11g R1, traditional *_DUMP_DEST parameters are ignored and the trace location is decided based on DIAGNOSTIC_DEST parameter. If this parameter is not set explicitly, it defaults to ORACLE_BASE environment variable if this is set, if this is not set, then it defaults to ORACLE_HOME/log.

DIAGNOSTIC_DEST is the ADR root directory also known as ADR base.

Each instance of each product stores diagnostic data in its own ADR home directory. For example, in a Real Application Clusters (RAC) environment with shared storage and ASM, each database instance and each ASM instance have a home directory within ADR.

Under ADR base are multiple ADR homes. ADR home is the root directory for all traces within an instance of a product. i.e. Each instance of a product has an ADR home. Directroy structure of ADR is pictorial represented as below:

Under the ADR base there will be a folder “diag”. Within this folder there will be seperate folders for each product like rdbsm, asm, crs, clients, lsnrctl, netcman, etc… Each of these folders will have seperate folders for every instance of the product.

For an instance of Oracle database, the structure is as depicted. All traces that used to be within USER_DUMP_DEST and BACKGROUND_DUMP_DEST are with trace folder. There is a seperate folder for alert log. A copy of the alert log in XML format is mainted in the alert folder.

Problem and Incident are two terms used frequently in ADR. All errors like the ora-600, ora-7445 or any other error like ora-470 that crashes database instance are termed as critical error. A critical error is termed as a Problem and is given a problem id by ADR based on it’s symptoms. Each occurrence of a critical error is termed as an Incident.

Incident directory contains multiple subdirectories, where each subdirectory is named for a particular incident, and where each contains dumps pertaining only to that incident.

Metadata directory contains ADR’s metadata.

HM directory contains health monitor reports which are part of ADR.

ADRCI – ADR command interpreter and EM support work bench are the two interfaces to ADR.

We will discuss these tools in our next post.

4 May, 2008 by Leo 11g 0

database

ORA-00600 [12333]

10.2.0.4 on AIX5L (64-Bit) is Out

ORA-07445 [opidsa()+480] – Metalink Alert for 10.2.0.3

Significance of LD_LIBRARY_PATH

CPU (Critical Patch Update) and N-apply

Understanding of Oracle 11g Automatic Diagnostic Repository – Part 1

Disclaimer!!

Recent Comments

Trending

Subscribe to Blog via Email