Setting Shared Memory :
Shared memory allows processes to access common structures and data by placing them in shared memory segments. It’s the fastest form of Interprocess Communication (IPC) available since no kernel involvement occurs when data is passed between the processes. In fact, data does not need to be copied between the processes.
Oracle uses shared memory segments for the Shared Global Area (SGA) which is an area of memory that is shared by Oracle processes. The size of the SGA has a significant impact to Oracle’s performance since it holds database buffer cache and much more.
To see all shared memory settings, execute:
$ ipcs -lm
This parameter defines the maximum size in bytes of a single shared memory segment that a Linux process can allocate in its virtual address space. For example, if you use the RHEL 3 smp kernel on a 32-bit platform (x86), then the virtual address space for a user process is 3 GB. If you use the RHEL 3 hugemem kernel on a 32-bit platform (x86), then the virtual address space for a user process is almost 4GB. Hence, setting SHMMAX to 4 GB – 1 byte (4294967295 bytes) on a smp kernel on a 32-bit architecture won’t increase the maximum size of a shared memory segment to 4 GB -1. Even setting SHMMAX to 4 GB – 1 byte using the hugemem kernel on a 32-bit architecture won’t enable a process to get such a large shared memory segment. In fact, the upper limit for a shared memory segment for an Oracle 10g R1 SGA using the hugemem kernel is roughly 3.42 GB (~3.67 billion bytes) since virtual address space is also needed for other things like shared libraries. This means if you have three 2 GB shared memory segments on a 32-bit system, no process can attach to more than one shared memory segment at a time. Also note if you set SHMMAX to 4294967296 bytes (4*1024*1024*1024=4GB) on a 32-bit system, then SHMMAX will essentially bet set to 0 bytes since it wraps around the 4GB value. This means that SHMMAX should not exceed 4294967295 on a 32-bit system. On x86-64 platforms, SHMMAX can be much larger than 4GB since the virtual address space is not limited by 32 bits.
Since the SGA is comprised of shared memory, SHMMAX can potentially limit the size of the SGA. SHMMAX should be slightly larger than the SGA size. If SHMMAX is too small, you can get error messages similar to this one:
ORA-27123: unable to attach to shared memory segment
It is highly recommended that the shared memory fits into the Big Pages or Huge Pages pool, see Large Memory Optimization (Big Pages, Huge Pages).
To increase the default maximum SGA size on x86 RHEL 2.1 systems without VLM, refer to Growing the Oracle SGA to 2.7 GB in x86 RHEL 2.1 Without VLM.
To increase the default maximum SGA size on x86 RHEL 3/4 systems without VLM, refer to Growing the Oracle SGA to 2.7/3.42 GB in x86 RHEL 3/4 Without VLM.
To determine the maximum size of a shared memory segment, run:
# cat /proc/sys/kernel/shmmax
2147483648
The default shared memory limit for SHMMAX can be changed in the proc file system without reboot:
# echo 2147483648 > /proc/sys/kernel/shmmax
Alternatively, you can use sysctl(8) to change it:
# sysctl -w kernel.shmmax=2147483648
To make a change permanent, add the following line to the file /etc/sysctl.conf (your setting may vary). This file is used during the boot process.
# echo "kernel.shmmax=2147483648" >> /etc/sysctl.conf
This parameter sets the system wide maximum number of shared memory segments.
Oracle recommends SHMMNI to be at least 4096 for Oracle 10g. For Oracle 9i on x86 the recommended minimum setting is lower. Since these recommendations are minimum settings, it’s best to set it always to at least 4096 for 9i and 10g databases on x86 and x86-64 platforms.
To determine the system wide maximum number of shared memory segments, run:
# cat /proc/sys/kernel/shmmni
4096
The default shared memory limit for SHMMNI can be changed in the proc file system without reboot:
# echo 4096 > /proc/sys/kernel/shmmni
Alternatively, you can use sysctl(8)
to change it:
# sysctl -w kernel.shmmni=4096
To make a change permanent, add the following line to the file /etc/sysctl.conf. This file is used during the boot process.
# echo "kernel.shmmni=4096" >> /etc/sysctl.conf
This parameter sets the total amount of shared memory pages that can be used system wide. Hence, SHMALL should always be at least ceil(shmmax/PAGE_SIZE).
The default size for SHMALL in RHEL 3/4 and 2.1 is 2097152 which is also Oracle’s recommended minimum setting for 9i and 10g on x86 and x86-64 platforms. In most cases this setting should be sufficient since it means that the total amount of shared memory available on the system is 2097152*4096 bytes (shmall*PAGE_SIZE) which is 8 GB. PAGE_SIZE is usually 4096 bytes unless you use Big Pages or Huge Pages which supports the configuration of larger memory pages.
If you are not sure what the default PAGE_SIZE is on your Linux system, you can run the following command:
$ getconf PAGE_SIZE
4096
To determine the system wide maximum number of shared memory pages, run:
# cat /proc/sys/kernel/shmall
2097152
The default shared memory limit for SHMALL can be changed in the proc file system without reboot:
# echo 2097152 > /proc/sys/kernel/shmall
Alternatively, you can use sysctl(8) to change it:
# sysctl -w kernel.shmall=2097152
To make a change permanent, add the following line to the file /etc/sysctl.conf. This file is used during the boot process.
# echo "kernel.shmall=2097152" >> /etc/sysctl.conf
Sometimes after an instance crash you may have to remove Oracle’s shared memory segment(s) manually.
To see all shared memory segments that are allocated on the system, execute:
$ ipcs -m
------ Shared Memory Segments --------
key shmid owner perms bytes nattch status
0x8f6e2129 98305 oracle 600 77694523 0
0x2f629238 65536 oracle 640 2736783360 35
0x00000000 32768 oracle 640 2736783360 0 dest
In this example you can see that three shared memory segments have been allocated. The output also shows that shmid 32768 is an abandoned shared memory segment from a past ungraceful Oracle shutdown. Status “dest” means that this memory segment is marked to be destroyed. To find out more about this shared memory segment you can run:
$ ipcs -m -i 32768
Shared memory Segment shmid=32768
uid=500 gid=501 cuid=500 cgid=501
mode=0640 access_perms=0640
bytes=2736783360 lpid=3688 cpid=3652 nattch=0
att_time=Sat Oct 29 13:36:52 2005
det_time=Sat Oct 29 13:36:52 2005
change_time=Sat Oct 29 11:21:06 2005
To remove the shared memory segment, you could copy/paste shmid and execute:
$ ipcrm shm 32768
Another approach to remove shared memory is to use Oracle’s sysresv utility. Here are a few self explanatory examples on how to use sysresv:
Checking Oracle’s IPC resources:
$ sysresv
IPC Resources for ORACLE_SID "orcl" :
Shared Memory
ID KEY
No shared memory segments used
Semaphores:
ID KEY
No semaphore resources used
Oracle Instance not alive for sid "orcl"
$
Instance is up and running:
$ sysresv -i IPC Resources for ORACLE_SID "orcl" : Shared Memory: ID KEY 2818058 0xdc70f4e4 Semaphores: ID KEY 688128 0xb11a5934 Oracle Instance alive for sid "orcl" SYSRESV-005: Warning Instance maybe alive - aborting remove for sid "orcl" $
Instance has crashed and resources were not released:
$ sysresv -i IPC Resources for ORACLE_SID "orcl" : Shared Memory: ID KEY 32768 0xdc70f4e4 Semaphores: ID KEY 98304 0xb11a5934 Oracle Instance not alive for sid "orcl" Remove ipc resources for sid "orcl" (y/n)?y Done removing ipc resources for sid "orcl" $