A note on how to fix SYSAUX Block Corruption

 Identify Objects in SYSAUX Tablespace

set linesize 120

set pagesize 100

 

COLUMN "Item" FORMAT A25

COLUMN "Space Used (GB)" FORMAT 999.99

COLUMN "Schema" FORMAT A25

COLUMN "Move Procedure" FORMAT A40

 

SELECT occupant_name "Item", space_usage_kbytes/1048576 "Space Used (GB)", schema_name "Schema", move_procedure "Move Procedure"

FROM v$sysaux_occupants

ORDER BY 1;

 

Item                      Space Used (GB) Schema                    Move Procedure
------------------------- --------------- ------------------------- ----------------------------------------
AO                                    .00 SYS                       DBMS_AW.MOVE_AWMETA
EM                                    .08 SYSMAN                    emd_maintenance.move_em_tblspc
EM_MONITORING_USER                    .00 DBSNMP
EXPRESSION_FILTER                     .00 EXFSYS
JOB_SCHEDULER                         .00 SYS
LOGMNR                                .01 SYSTEM                    SYS.DBMS_LOGMNR_D.SET_TABLESPACE
LOGSTDBY                              .00 SYSTEM                    SYS.DBMS_LOGSTDBY.SET_TABLESPACE
ODM                                   .00 DMSYS                     MOVE_ODM
ORDIM                                 .00 ORDSYS
ORDIM/PLUGINS                         .00 ORDPLUGINS
ORDIM/SQLMM                           .00 SI_INFORMTN_SCHEMA
SDO                                   .00 MDSYS                     MDSYS.MOVE_SDO
SM/ADVISOR                            .02 SYS
SM/AWR                                .15 SYS
SM/OPTSTAT                          11.44 SYS
SM/OTHER                              .02 SYS
STATSPACK                             .00 PERFSTAT
STREAMS                               .00 SYS
TEXT                                  .00 CTXSYS                    DRI_MOVE_CTXSYS
TSM                                   .00 TSMSYS
ULTRASEARCH                           .00 WKSYS                     MOVE_WK
ULTRASEARCH_DEMO_USER                 .00 WK_TEST                   MOVE_WK
WM                                    .00 WMSYS                     DBMS_WM.move_proc
XDB                                   .00 XDB                       XDB.DBMS_XDB.MOVEXDB_TABLESPACE
XSAMD                                 .00 OLAPSYS                   DBMS_AMD.Move_OLAP_Catalog
XSOQHIST                              .00 SYS                       DBMS_XSOQ.OlapiMoveProc

 

How long old stats are kept

select dbms_stats.get_stats_history_retention from dual;

 

Set retention of old stats to 10 days

exec dbms_stats.alter_stats_history_retention(10);

 

--Completely remove old statistics (This is the fastest option. Acts like a truncate command)

EXEC DBMS_STATS.PURGE_STATS(DBMS_STATS.PURGE_ALL);

 

Purge stats older than 10 days (best to do this in stages if there is a lot of data (sysdate-30,sydate-25 etc)

exec DBMS_STATS.PURGE_STATS(SYSDATE-10);

 

 

Show available stats that have not been purged

select dbms_stats.get_stats_history_availability from dual;

 

Show how big the tables are and rebuild after stats have been purged

col Mb form 9,999,999

col SEGMENT_NAME form a40

col SEGMENT_TYPE form a6

set lines 120

 

select sum(bytes/1024/1024) Mb, segment_name,segment_type from dba_segments

where tablespace_name = 'SYSAUX'

and segment_name like 'WRI$_OPTSTAT%'

and segment_type='TABLE'

group by segment_name,segment_type order by 1 asc;

 

 

        MB SEGMENT_NAME                             SEGMEN
---------- ---------------------------------------- ------
         0 WRI$_OPTSTAT_OPR                         TABLE
         0 WRI$_OPTSTAT_AUX_HISTORY                 TABLE
        88 WRI$_OPTSTAT_TAB_HISTORY                 TABLE
       126 WRI$_OPTSTAT_IND_HISTORY                 TABLE
       158 WRI$_OPTSTAT_HISTGRM_HISTORY             TABLE
     4,482 WRI$_OPTSTAT_HISTHEAD_HISTORY            TABLE

 

 

Show how big the indexes are ready for a rebuild after stats have been purged

col Mb form 9,999,999

col SEGMENT_NAME form a40

col SEGMENT_TYPE form a6

set lines 120

 

select sum(bytes/1024/1024) Mb, segment_name,segment_type from dba_segments

where tablespace_name = 'SYSAUX'

and segment_name like 'WRI$_OPTSTAT%'

and segment_type='INDEX'

group by segment_name,segment_type order by 1 asc;

 

        MB SEGMENT_NAME                             SEGMEN
---------- ---------------------------------------- ------
         0 WRH$_OPTIMIZER_ENV_PK                    INDEX
         0 I_WRI$_OPTSTAT_OPR_STIME                 INDEX
         0 I_WRI$_OPTSTAT_AUX_ST                    INDEX
        88 I_WRI$_OPTSTAT_TAB_ST                    INDEX
       105 I_WRI$_OPTSTAT_IND_ST                    INDEX
       105 I_WRI$_OPTSTAT_H_ST                      INDEX
       195 I_WRI$_OPTSTAT_TAB_OBJ#_ST               INDEX
       213 I_WRI$_OPTSTAT_H_OBJ#_ICOL#_ST           INDEX
       214 I_WRI$_OPTSTAT_IND_OBJ#_ST               INDEX
     2,055 I_WRI$_OPTSTAT_HH_ST                     INDEX
     3,883 I_WRI$_OPTSTAT_HH_OBJ_ICOL_ST            INDEX

 

 

Run the rebuild table commands – note that this does cause any gather_stats jobs to fail

alter table WRI$_OPTSTAT_TAB_HISTORY  move tablespace sysaux;
alter table WRI$_OPTSTAT_IND_HISTORY  move tablespace sysaux;
alter table WRI$_OPTSTAT_HISTHEAD_HISTORY  move tablespace sysaux;
alter table WRI$_OPTSTAT_HISTGRM_HISTORY  move tablespace sysaux;
alter table WRI$_OPTSTAT_AUX_HISTORY  move tablespace sysaux;
alter table WRI$_OPTSTAT_OPR  move tablespace sysaux;
alter table WRH$_OPTIMIZER_ENV  move tablespace sysaux;

Alter index SYS.I_WRI$_OPTSTAT_IND_ST rebuild TABLESPACE SYSAUX;
Alter index SYS.I_WRI$_OPTSTAT_IND_OBJ#_ST rebuild TABLESPACE SYSAUX;
Alter index SYS.I_WRI$_OPTSTAT_HH_ST rebuild TABLESPACE SYSAUX;
Alter index SYS.I_WRI$_OPTSTAT_HH_OBJ_ICOL_ST rebuild TABLESPACE SYSAUX;
Alter index SYS.I_WRI$_OPTSTAT_TAB_ST rebuild TABLESPACE SYSAUX;
Alter index SYS.I_WRI$_OPTSTAT_TAB_OBJ#_ST rebuild TABLESPACE SYSAUX;
Alter index SYS.I_WRI$_OPTSTAT_OPR_STIME rebuild TABLESPACE SYSAUX;

 

 

Script to generate rebuild statements

select 'alter index '||segment_name||'  rebuild online parallel (degree 14);'

from dba_segments

where tablespace_name = 'SYSAUX'

and segment_name like '%OPT%' and segment_type='INDEX'

 

Once completed it is best to check that the indexes (indices) are usable

col INDEX_NAME form a40

select  di.index_name,di.index_type,di.status 

from  dba_indexes di , dba_tables dt

where  di.tablespace_name = 'SYSAUX'

and dt.table_name = di.table_name

and di.table_name like '%OPT%'

order by 1 asc;

 


INDEX_NAME                     INDEX_TYPE                  STATUS
------------------------------ --------------------------- --------
I_WRI$_OPTSTAT_AUX_ST          FUNCTION-BASED NORMAL       VALID
I_WRI$_OPTSTAT_HH_OBJ_ICOL_ST  FUNCTION-BASED NORMAL       VALID
I_WRI$_OPTSTAT_HH_ST           FUNCTION-BASED NORMAL       VALID
I_WRI$_OPTSTAT_H_OBJ#_ICOL#_ST FUNCTION-BASED NORMAL       VALID
I_WRI$_OPTSTAT_H_ST            FUNCTION-BASED NORMAL       VALID
I_WRI$_OPTSTAT_IND_OBJ#_ST     FUNCTION-BASED NORMAL       VALID
I_WRI$_OPTSTAT_IND_ST          FUNCTION-BASED NORMAL       VALID
I_WRI$_OPTSTAT_OPR_STIME       FUNCTION-BASED NORMAL       VALID
I_WRI$_OPTSTAT_TAB_OBJ#_ST     FUNCTION-BASED NORMAL       VALID
I_WRI$_OPTSTAT_TAB_ST          FUNCTION-BASED NORMAL       VALID
WRH$_OPTIMIZER_ENV_PK          NORMAL                      VALID

 

 

After we clear statistics history, we can optionally rebuild %OPT% tables and indexes in SYSAUX.

--First check status of all %OPT% indexes. They must be VALID before rebuild.

select  di.index_name,di.index_type,di.status  from  dba_indexes di , dba_tables dt

where  di.tablespace_name = 'SYSAUX' and dt.table_name = di.table_name

and di.table_name like '%OPT%' order by 1 asc;

 

--Rebuild tables.

select 'alter table '||segment_name||'  move tablespace SYSAUX parallel 8;' from dba_segments where tablespace_name = 'SYSAUX'

and segment_name like '%OPT%' and segment_type='TABLE';

 

--Rebuild indexes.

select 'alter index '||segment_name||'  rebuild online parallel 8;' from dba_segments where tablespace_name = 'SYSAUX'

and segment_name like '%OPT%' and segment_type='INDEX';

 

--Check status of all %OPT% indexes. They must be VALID after rebuild.

select  di.index_name,di.index_type,di.status  from  dba_indexes di , dba_tables dt

where  di.tablespace_name = 'SYSAUX' and dt.table_name = di.table_name

and di.table_name like '%OPT%' order by 1 asc;

 

 

 

If above removal process taking too much of time then you can run below two sqls as sysdba to drop the old AWR’s and rebuild the repositories. This process is very fast.

SQL> connect / as sysdba
SQL> @?/rdbms/admin/catnoawr.sql
SQL> @?/rdbms/admin/catawrtb.sql

Comments

Post a Comment

Popular posts from this blog

How to drop index and before dropping it how to get the DDL.

  Oracle DROP INDEX statement DROP INDEX [schema_name.]index_name; DECLARE index_count INTEGER; BEGIN SELECT COUNT(*) INTO index_count FROM USER_INDEXES WHERE INDEX_NAME = 'index_name' ; IF index_count > 0 THEN EXECUTE IMMEDIATE 'DROP INDEX index_name' ; END IF; END; / In case want to check whether index exists and then delete it. If try to drop a non-existing, you will get the following error: SQL Error : ORA -01418 : specified index does not exist In SQLplus, set these before running the commands set long 100000 set longchunksize 100000 select DBMS_METADATA.GET_DDL( 'INDEX' , 'index_name') from all_indexes where owner in ( USER , 'USER_OTHER_THAN_LOGGED_IN_USER' ); Use below script to fetch metadata other information. SELECT DBMS_METADATA,GET_GRANTED_DDL('SYSTEM_GRANT','SSO******') from dual;
Read more

PRVG-11250 : The check "RPM Package Manager database" was not performed because

Following message reported from cluvfy while performing pre-requisite for Clusterware Installation:  Verifying RPM Package Manager database ...INFORMATION PRVG-11250 : The check "RPM Package Manager database" was not performed because it needs 'root' user privileges. SOLUTION Execute runcluvfy manually with "-method root" option and enter root password when prompted : Example: $/runcluvfy.sh stage -pre crsinst -n <HOSTNAME1>,<HOSTNAME2> -method root Enter "ROOT" password: For reference metalink doc id :  2548970.1
Read more

ORA-00257:archiver error, connect internal only until freed

  Issue: Application log shows below error and users are unable to connect to the database. 0RA-00257:archiver error, connect internal only until freed (OR) 0RA-00257:archiver error, connect internal only until freed ORA-16020: less destinations available than specified by LOG_ARCHIVE_MIN_SUCCEED_DEST Fix: 1) Check if there are any errors for the archive destination(s). Make sure that the number of VALID archive destinations is greater than or equal to the value specified by LOG_ARCHIVE_MIN_SUCCEED_DEST initialization parameter. set pages 1000 col dest_name for a30 col destination for a60 SELECT dest_id, dest_name, binding, status, destination, error FROM v$archive_dest; SHOW PARAMETER log_archive_min_succeed_dest 2) If space is full in one or more of the archive destinations (or if destination is not available), take any of the following steps: 2.a) Manually move the archives to another location and delete them from archive destination.   (OR) 2.b) Change the archive des...
Read more

Verifying Daemon “Avahi-Daemon” Not Configured And Running …FAILED (PRVG-1360)

  Execute "/tmp/CVU_19.0.0.0.0_grid/runfixup.sh" as root user on nodes "shsebs7idb1" to perform the fix up operations manually Press ENTER key to continue after execution of "/tmp/CVU_19.0.0.0.0_grid/runfixup.sh" has completed on nodes "shsebs7idb1" Fix: Daemon "avahi-daemon" not configured and running   Node Name                             Status   ------------------------------------  ------------------------   shsebs7idb1                           failed ERROR: shsebs7idb1: PRVG-9023 : Manual fix up command "/tmp/CVU_19.0.0.0.0_grid/runfixup.sh" was not issued by root user on node "shsebs7idb1" Result: "Daemon "avahi-daemon" not configured and running" could not be fixed on nodes "shsebs7idb1" SOLUTION: To fix this error, stop and disable avahi-daemon service. 1. Check avahi-daemon status: [root@shsebs7idb1 ~]# ls -...
Read more

Linux OL7/RHEL7: PRVE-0421 : No entry exists in /etc/fstab for mounting /dev/shm

  This is due to below bugs: OL7 -  Unpublished Bug 21441387- 12.2/MAIN/OL7.1: CVU REPORTS /DEV/SHM NOT MOUNTED WHEN IT IS MOUNTED  21441387.8 RHEL 7 - Unpublished Bug 25907259 : RHEL7.2: CVU REPORTS /DEV/SHM NOT MOUNTED WHEN IT IS MOUNTED  25907259.8 As a work-around in OL7: In OL7, the /dev/shm is not mounted in /etc/fstab on a fresh system, it is done in: [root@testc140 u02]# ll /usr/lib/dracut/modules.d/99base/init.sh -rwxr-xr-x. 1 root root 11379 Mar 26 10:14 vi /usr/lib/dracut/modules.d/99base/init.sh and comment the below lines 62 if ! ismounted /dev/shm; then 63 mkdir -m 0755 /dev/shm 64 mount -t tmpfs -o mode=1777,nosuid,nodev,strictatime tmpfs /dev/shm >/dev/null 65 fi so, this validation is not really valid for OL7.    We can also safely ignore this message. For reference metalink id: -  Doc ID 2065603.1
Read more

SKIP DNS RESLOV.CONF CHECK DURING RAC CONFIGURATION

 shsebs7idb1: PRVG-2064 : There are no configured name servers in the file              '/etc/resolv.conf' on the nodes "shsebs7idb1" Solution: [root@shsebs7idb2 ~]# ls -ltr /etc/resolv.conf -rw-r--r--. 1 root root 78 Jul  3 11:02 /etc/resolv.conf [root@shsebs7idb2 ~]# mv /etc/resolv.conf /etc/resolv.conf.sid Rename resolv.conf to resolv.conf.sid for example and re run the installation again . It  worked for me
Read more

CPU Patch Analysis

 Oracle CPU patch analysis Identifying the Latest Critical Patch Update for Oracle E-Business Suite Release 12.2 (Doc ID 2484000.1) Database Quick-Link If you only care about the database patches, this MOS note steps you through really easily. Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patch sets and Base Releases (Doc ID 2118136.2) https://www.oracle.com/security-alerts/ Critical Patch Updates Critical Patch Updates provide security patches for supported Oracle on-premises products. They are available to customers with valid support contracts. Starting in April 2022, Critical Patch Updates are released on the third Tuesday of January, April, July, and October (They were previously published on the Tuesday closest to the 17th day of January, April, July, and October). The next four dates are: 16 April 2024 16 July 2024 15 October 2024 21 January 2025 A pre-release announcement will be published on the Thursday preceding eac...
Read more

How to write to a CSV file using Oracle SQL*Plus

  SPOOL command  The  SPOOL  command is  unavailable  in the browser-based SQL*Plus version,  iSQL*Plus . To generate files while using iSQL*Plus, change the necessary preference settings to directly output to a file. This is accomplished using the  SPOOL  statement. While  SPOOL  is  active , SQL*PLus will store the output of any query to the specified file. Therefore, the next command to enter is  spool : spool filepath Skipping ahead slightly,  after  your query is inserted, you also need to halt  spool  so the file output is closed by using the  spool off  command: spool off Insert the query  The last step after the settings are modified and  spool  is running is to insert your query. For our simple example, we’re outputting all books from our  books  table. SELECT   title,   author FROM   authors; Don’t forget the semi...
Read more

How to troubleshoot Long Running Concurrent Request in EBS 12.2

 The following points need to answer before jumping into troubleshooting: Are there any recent code changes done in the concurrent program?           Check long_running_sid and verify the recent code change of last_ddl_timestamp after a             discussion with the developer. Was this running long in the last few run as well, or was this time only?            ==> Check the program history. How much time does it take to complete?           Once again program history can give us some idea. Are these jobs fetching higher data compared to the last run           ===> Confirm on data change . Query Plan change will indicate this issue. Does this job runs at any specific time or it can be run anytime           ===> The nature of the job can help to understand the job's nature. Does this job fetch data using DB Link...
Read more

How To Manage Space of The FRA in the Oracle DB

  Delete Old files connect with the target database using RMAN RMAN target / Delete the old files RMAN> delete obsolete; Remove Restore Points SQL> SELECT name, storage_size FROM gv$restore_point; Drop the restore point SQL> DROP restore point RP; Delete the archive logs RMAN> DELETE archivelog ALL Increase the Space SQL> ALTER SYSTEM SET db_recovery_file_dest_size = 20G scope=both; Disable the Flashback ALTER DATABASE flashback off;
Read more