How to enable DNFS and set hugepages

Configuring HugePages for Oracle on Linux (x86-64)


The steps in this section are for configuring HugePages on a 64-bit Oracle Linux system running one or more Oracle Database instances.

To configure HugePages:Verify that the soft and hard values in kilobytes of memlock that are configured in /etc/security/limits.conf are slightly smaller than the amount of installed memory. For example, if the system has 64GB of RAM, the values shown here would be appropriate:

  1. soft memlock 60397977
hard memlock 60397977

  2. Log in as the Oracle account owner (usually oracle) and use the following command to verify the value of memlock:

    $ ulimit -l
60397977

  3. If your system is running Oracle Database 11g or later, disable AMM by setting the values of both of the initialization parameters memory_target and memory_max_target to 0.

    If you start the Oracle Database instances with a server parameter file, which is the default if you created the database with the Database Configuration Assistant (DBCA), enter the following commands at the SQL prompt:

    SQL> alter system set memory_target=0;
System altered.
SQL> alter system set memory_max_target=0;
System altered.

    If you start the Oracle Database instances with a text initialization parameter file, manually edit the file so that it contains the following entries:

    memory_target = 0
memory_max_target = 0

  4. Verify that all the Oracle Database instances are running (including any Automatic Storage Management (ASM) instances) as they would run on the production system.

  5. Create the file hugepages_settings.sh with the following content (taken from the My Oracle Support (MOS) note 401749.1).

    #!/bin/bash
#
# hugepages_settings.sh
#
# Linux bash script to compute values for the
# recommended HugePages/HugeTLB configuration
#
# Note: This script does calculation for all shared memory
# segments available when the script is run, no matter it
# is an Oracle RDBMS shared memory segment or not.
# Check for the kernel version
KERN=`uname -r | awk -F. '{ printf("%d.%d\n",$1,$2); }'`
# Find out the HugePage size
HPG_SZ=`grep Hugepagesize /proc/meminfo | awk {'print $2'}`
# Start from 1 pages to be on the safe side and guarantee 1 free HugePage
NUM_PG=1
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk {'print $5'} | grep "[0-9][0-9]*"`
do
   MIN_PG=`echo "$SEG_BYTES/($HPG_SZ*1024)" | bc -q`
   if [ $MIN_PG -gt 0 ]; then
      NUM_PG=`echo "$NUM_PG+$MIN_PG+1" | bc -q`
   fi
done
# Finish with results
case $KERN in
   '2.4') HUGETLB_POOL=`echo "$NUM_PG*$HPG_SZ/1024" | bc -q`;
          echo "Recommended setting: vm.hugetlb_pool = $HUGETLB_POOL" ;;
   '2.6') echo "Recommended setting: vm.nr_hugepages = $NUM_PG" ;;
    *) echo "Unrecognized kernel version $KERN. Exiting." ;;
esac
# End

  6. Make the file executable, and run it to calculate the recommended value for the vm.nr_hugepages kernel parameter.

    $ chmod u+x ./hugepages_setting.sh  
$ ./hugepages_settings.sh
.
.
.
Recommended setting: vm.nr_hugepages = 22960

  7. As root, edit the file /etc/sysctl.conf and set the value of the vm.nr_hugepages parameter to the recommended value.

    vm.nr_hugepages = 22960

  8. Stop all the database instances and reboot the system.

After rebooting the system, verify that the database instances (including any ASM instances) have started, and use the following command to display the state of the huge pages.

# grep ^Huge /proc/meminfo
HugePages_Total:   22960
HugePages_Free:     2056
HugePages_Rsvd:     2016
HugePages_Surp:        0
Hugepagesize:       2048 kB

The value of HugePages_Free should be smaller than that of HugePages_Total, and the value of HugePages_Rsvd should be greater than zero. As the database instances allocate pages dynamically and proactively as required, the sum of the Hugepages_Free and HugePages_Rsvd values is likely to be smaller than the total SGA size.

If you subsequenty change the amount of system memory, add or remove any database instances, or change the size of the SGA for a database instance, use hugepages_settings.sh to recalculate the value of vm.nr_hugepages, readjust the setting in /etc/sysctl.conf, and reboot the system.

Below steps are in short need to follow:


First of all shutdown database and listener.

Following steps would be required:

chmod root:root $ORACLE_HOME/bin/oradism

chmod 4755 $ORACLE_HOME/bin/oradism

cd $ORACLE_HOME/rdbms/lib

make -f ins_rdbms.mk dnfs_on

Below steps for configure Huge pages:

$cat /etc/secuirity/limit.conf | grep oracle |grep memlock

oracle soft memlock 6293504

oracle hard memlock 6293504

Need to tweak value to 839066     .

DB bounce is required.



Enabling and Disabling Direct NFS Client Control of NFS

Use these commands to enable or disable Direct NFS Client Oracle Disk Manager Control of NFS:

By default, Direct NFS Client is installed in a disabled state with single-instance Oracle Database installations. Before enabling Direct NFS, you must configure an oranfstab file.

Enabling Direct NFS Client Control of NFS

  1. Change the directory to $ORACLE_HOME/rdbms/lib.

  2. Enter the following command:


make -f ins_rdbms.mk dnfs_on

  Disabling Direct NFS Client Control of NFS

  1. Log in as the Oracle software installation owner, and disable Direct NFS Client using the following commands:

         cd $ORACLE_HOME/rdbms/lib

          make -f ins_rdbms.mk dnfs_off


     2. Remove the oranfstab file.


If you remove an NFS path that an Oracle Database is using, then you must restart the database for the change to take effect.


Verifying the use of Oracle Direct NFS client

1) If dNFS is enabled, the alert.log will show the following entry when the database is started.

Oracle instance running with ODM: Oracle Direct NFS ODM Library Version 4.0

2) Query the dNFS server information from v$dnfs_servers view inside the database.

SQL> select svrname, dirname, mntport, nfsport, wtmax, rtmax from v$dnfs_servers;

SVRNAME            DIRNAME                 MNTPORT    NFSPORT      WTMAX      RTMAX
------------------ -------------------- ---------- ---------- ---------- ----------
fb-dnfs-test-02    /rman01                    2049       2049     524288      524288

Even though dNFS is enabled, Oracle only mounts the volume/filesystem and opens the files when they are accessed. If no data files are accessed, then the above view will return no rows.

How to troubleshoot oranfstab issues

Please make sure the svrname from the v$dnfs_servers show the server name from the oranfstab file.  If not, generally there is some misconfiguration in the oranfstab file.  Look for any dangling entries and clean up.

For example, in the following output the SVRNAME is not correct as it should be the actual server name from the oranfstab for all the exports under the given server but we see the IP addresses of the paths.  This means the oranfstab is not configured correctly.

SQL> SELECT svrname, dirname, mntport, nfsport, wtmax, rtmax FROM v$dnfs_servers;

Server                    Export Name                                                         MNTPORT    NFSPORT      WTMAX      RTMAX  
------------------------- ---------------------------------------------------------------- ---------- ---------- ---------- ----------  
192.168.1.100            /orcldata01                                                             2049       2049     524288     524288  
192.168.1.101            /orcldata02                                                             2049       2049     524288     524288 
192.168.1.102            /redo01                                                                2049       2049     524288     524288

Looking at the oranfstab file, we see the following. 

server: cd-dnfs-01
local: 192.168.1.50
path: 192.168.1.100
local: 192.168.1.50
path: 192.168.1.101
local: 192.168.1.50
path: 192.168.1.102
local: 192.168.1.50
nfs_version: nfsv3
export: /orcldata01 mount: /u02
export: /orcldata02 mount: /u03
export: /redo01    mount: /u04

After rearranging them and removing the dangling local entry from the oranfstab file and restarting the database shows the expected output.

server: cd-dnfs-01
local: 192.168.1.50 path: 192.168.1.100
local: 192.168.1.50 path: 192.168.1.101
local: 192.168.1.50 path: 192.168.1.102

nfs_version: nfsv3
export: /orcldata01 mount: /u02
export: /orcldata02 mount: /u03
export: /redo01    mount: /u04
SQL> SELECT svrname, dirname, mntport, nfsport, wtmax, rtmax FROM v$dnfs_servers;

Server                    Export Name                                                         MNTPORT    NFSPORT      WTMAX      RTMAX 
------------------------- ---------------------------------------------------------------- ---------- ---------- ---------- ---------- 
cd-dnfs-01                /orcldata01                                                             2049       2049     524288     524288  
cd-dnfs-01                /orcldata02                                                             2049       2049     524288     524288
cd-dnfs-01                /redo01                                                                2049       2049     524288     524288

Comments

Post a Comment

Popular posts from this blog

How to drop index and before dropping it how to get the DDL.

  Oracle DROP INDEX statement DROP INDEX [schema_name.]index_name; DECLARE index_count INTEGER; BEGIN SELECT COUNT(*) INTO index_count FROM USER_INDEXES WHERE INDEX_NAME = 'index_name' ; IF index_count > 0 THEN EXECUTE IMMEDIATE 'DROP INDEX index_name' ; END IF; END; / In case want to check whether index exists and then delete it. If try to drop a non-existing, you will get the following error: SQL Error : ORA -01418 : specified index does not exist In SQLplus, set these before running the commands set long 100000 set longchunksize 100000 select DBMS_METADATA.GET_DDL( 'INDEX' , 'index_name') from all_indexes where owner in ( USER , 'USER_OTHER_THAN_LOGGED_IN_USER' ); Use below script to fetch metadata other information. SELECT DBMS_METADATA,GET_GRANTED_DDL('SYSTEM_GRANT','SSO******') from dual;
Read more

PRVG-11250 : The check "RPM Package Manager database" was not performed because

Following message reported from cluvfy while performing pre-requisite for Clusterware Installation:  Verifying RPM Package Manager database ...INFORMATION PRVG-11250 : The check "RPM Package Manager database" was not performed because it needs 'root' user privileges. SOLUTION Execute runcluvfy manually with "-method root" option and enter root password when prompted : Example: $/runcluvfy.sh stage -pre crsinst -n <HOSTNAME1>,<HOSTNAME2> -method root Enter "ROOT" password: For reference metalink doc id :  2548970.1
Read more

ORA-00257:archiver error, connect internal only until freed

  Issue: Application log shows below error and users are unable to connect to the database. 0RA-00257:archiver error, connect internal only until freed (OR) 0RA-00257:archiver error, connect internal only until freed ORA-16020: less destinations available than specified by LOG_ARCHIVE_MIN_SUCCEED_DEST Fix: 1) Check if there are any errors for the archive destination(s). Make sure that the number of VALID archive destinations is greater than or equal to the value specified by LOG_ARCHIVE_MIN_SUCCEED_DEST initialization parameter. set pages 1000 col dest_name for a30 col destination for a60 SELECT dest_id, dest_name, binding, status, destination, error FROM v$archive_dest; SHOW PARAMETER log_archive_min_succeed_dest 2) If space is full in one or more of the archive destinations (or if destination is not available), take any of the following steps: 2.a) Manually move the archives to another location and delete them from archive destination.   (OR) 2.b) Change the archive des...
Read more

Verifying Daemon “Avahi-Daemon” Not Configured And Running …FAILED (PRVG-1360)

  Execute "/tmp/CVU_19.0.0.0.0_grid/runfixup.sh" as root user on nodes "shsebs7idb1" to perform the fix up operations manually Press ENTER key to continue after execution of "/tmp/CVU_19.0.0.0.0_grid/runfixup.sh" has completed on nodes "shsebs7idb1" Fix: Daemon "avahi-daemon" not configured and running   Node Name                             Status   ------------------------------------  ------------------------   shsebs7idb1                           failed ERROR: shsebs7idb1: PRVG-9023 : Manual fix up command "/tmp/CVU_19.0.0.0.0_grid/runfixup.sh" was not issued by root user on node "shsebs7idb1" Result: "Daemon "avahi-daemon" not configured and running" could not be fixed on nodes "shsebs7idb1" SOLUTION: To fix this error, stop and disable avahi-daemon service. 1. Check avahi-daemon status: [root@shsebs7idb1 ~]# ls -...
Read more

Linux OL7/RHEL7: PRVE-0421 : No entry exists in /etc/fstab for mounting /dev/shm

  This is due to below bugs: OL7 -  Unpublished Bug 21441387- 12.2/MAIN/OL7.1: CVU REPORTS /DEV/SHM NOT MOUNTED WHEN IT IS MOUNTED  21441387.8 RHEL 7 - Unpublished Bug 25907259 : RHEL7.2: CVU REPORTS /DEV/SHM NOT MOUNTED WHEN IT IS MOUNTED  25907259.8 As a work-around in OL7: In OL7, the /dev/shm is not mounted in /etc/fstab on a fresh system, it is done in: [root@testc140 u02]# ll /usr/lib/dracut/modules.d/99base/init.sh -rwxr-xr-x. 1 root root 11379 Mar 26 10:14 vi /usr/lib/dracut/modules.d/99base/init.sh and comment the below lines 62 if ! ismounted /dev/shm; then 63 mkdir -m 0755 /dev/shm 64 mount -t tmpfs -o mode=1777,nosuid,nodev,strictatime tmpfs /dev/shm >/dev/null 65 fi so, this validation is not really valid for OL7.    We can also safely ignore this message. For reference metalink id: -  Doc ID 2065603.1
Read more

SKIP DNS RESLOV.CONF CHECK DURING RAC CONFIGURATION

 shsebs7idb1: PRVG-2064 : There are no configured name servers in the file              '/etc/resolv.conf' on the nodes "shsebs7idb1" Solution: [root@shsebs7idb2 ~]# ls -ltr /etc/resolv.conf -rw-r--r--. 1 root root 78 Jul  3 11:02 /etc/resolv.conf [root@shsebs7idb2 ~]# mv /etc/resolv.conf /etc/resolv.conf.sid Rename resolv.conf to resolv.conf.sid for example and re run the installation again . It  worked for me
Read more

CPU Patch Analysis

 Oracle CPU patch analysis Identifying the Latest Critical Patch Update for Oracle E-Business Suite Release 12.2 (Doc ID 2484000.1) Database Quick-Link If you only care about the database patches, this MOS note steps you through really easily. Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patch sets and Base Releases (Doc ID 2118136.2) https://www.oracle.com/security-alerts/ Critical Patch Updates Critical Patch Updates provide security patches for supported Oracle on-premises products. They are available to customers with valid support contracts. Starting in April 2022, Critical Patch Updates are released on the third Tuesday of January, April, July, and October (They were previously published on the Tuesday closest to the 17th day of January, April, July, and October). The next four dates are: 16 April 2024 16 July 2024 15 October 2024 21 January 2025 A pre-release announcement will be published on the Thursday preceding eac...
Read more

How to write to a CSV file using Oracle SQL*Plus

  SPOOL command  The  SPOOL  command is  unavailable  in the browser-based SQL*Plus version,  iSQL*Plus . To generate files while using iSQL*Plus, change the necessary preference settings to directly output to a file. This is accomplished using the  SPOOL  statement. While  SPOOL  is  active , SQL*PLus will store the output of any query to the specified file. Therefore, the next command to enter is  spool : spool filepath Skipping ahead slightly,  after  your query is inserted, you also need to halt  spool  so the file output is closed by using the  spool off  command: spool off Insert the query  The last step after the settings are modified and  spool  is running is to insert your query. For our simple example, we’re outputting all books from our  books  table. SELECT   title,   author FROM   authors; Don’t forget the semi...
Read more

How to troubleshoot Long Running Concurrent Request in EBS 12.2

 The following points need to answer before jumping into troubleshooting: Are there any recent code changes done in the concurrent program?           Check long_running_sid and verify the recent code change of last_ddl_timestamp after a             discussion with the developer. Was this running long in the last few run as well, or was this time only?            ==> Check the program history. How much time does it take to complete?           Once again program history can give us some idea. Are these jobs fetching higher data compared to the last run           ===> Confirm on data change . Query Plan change will indicate this issue. Does this job runs at any specific time or it can be run anytime           ===> The nature of the job can help to understand the job's nature. Does this job fetch data using DB Link...
Read more

How To Manage Space of The FRA in the Oracle DB

  Delete Old files connect with the target database using RMAN RMAN target / Delete the old files RMAN> delete obsolete; Remove Restore Points SQL> SELECT name, storage_size FROM gv$restore_point; Drop the restore point SQL> DROP restore point RP; Delete the archive logs RMAN> DELETE archivelog ALL Increase the Space SQL> ALTER SYSTEM SET db_recovery_file_dest_size = 20G scope=both; Disable the Flashback ALTER DATABASE flashback off;
Read more