How to troubleshoot Output Post Processor Issues

 The Output Post Processor (OPP) is designed to support XML Publisher as a post-processing action for concurrent requests. If a request is submitted with an XML Publisher template specified as a layout for the concurrent request output, then after the concurrent manager finishes running the concurrent program, it will contact the OPP to apply the XML Publisher template and create the final output.

Memory Usage Per Process:

The maximum amount of memory or maximum Java heap size a single OPP process can use is by default set to 512MB. This value is needed by the Loader Data File: $FND_TOP/patch/115/import/US/afoppsrv.ldt which specifies that the DEVELOPER_PARAMETERS is “J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx512m”.

How to determine the current Java heap size:

SELECT service_id, service_handle, developer_parameters
FROM fnd_cp_services
WHERE service_id = (SELECT manager_type
FROM fnd_concurrent_queues
WHERE concurrent_queue_name = ‘FNDCPOPP’);



Increase the maximum Java heap size for the OPP

To 1024MB (1GB):

UPDATE fnd_cp_services
SET developer_parameters =
'J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx1024m'
WHERE service_id = (SELECT manager_type
FROM fnd_concurrent_queues
WHERE concurrent_queue_name = 'FNDCPOPP');

To 2048MB (2GB):

 UPDATE fnd_cp_services SET developer_parameters = 'J:oracle.apps.fnd.cp.gsf.GSMServiceController:-mx2048m' WHERE service_id = (SELECT manager_type FROM fnd_concurrent_queues WHERE concurrent_queue_name = 'FNDCPOPP');

Then we need to bounce the OPP to get it into effect

Recreate the OPP queue

The OPP queue can be Recreated using the $FND_TOP/patch/115/sql/afopp002.sql file as an ‘APPLSYS’ user. On running the script you will be prompted for username and password.

Profile options related to OPP

There are 2 new that can be used to control the timeouts

Profile Option : Concurrent: OPP Response Timeout
Internal Name : CONC_PP_RESPONSE_TIMEOUT
Description: Specifies the amount of time a manager waits for OPP to respond to its request for post-processing.

Profile Option : Concurrent:OPP Process Timeout
Internal Name : CONC_PP_PROCESS_TIMEOUT
Description: Specifies the amount of time the manager waits for the OPP to actually process the request.

How to check OPP logfiles

To investigate XML issues or other publishing problems, often the OPP log file is needed. Below are the steps to find logfile from the Front end

  1. Login to the application as SYSADMIN  
  2. Responsibility: System Administrator
  3. Concurrent –> Manager –> Administration
  4. Select the Output Post Processor
  5. Click on the Processes button
  6. Select the Concurrent Process which was active during the time that the request ran
  7. Click on the Manager Log button to open the Output Post Processor log file

Directly from the file system based after identifying the corresponding OPP log file name using the following SQL statement:

SELECT fcpp.concurrent_request_id req_id, fcp.node_name, fcp.logfile_name
FROM fnd_conc_pp_actions fcpp, fnd_concurrent_processes fcp
WHERE fcpp.processor_id = fcp.concurrent_process_id
AND fcpp.action_type = 6
AND fcpp.concurrent_request_id = &&request_id

Threads and Processes for Output post-processor

OPP Service is a multi-threaded process.

OPP processes and threads can be seen from Concurrent Manager define form

1) Logon to Applications with “System Administrator” responsibility

2) Concurrent -> Manager -> Define

3) Query for “Output Post Processor”

4) Click on “Work Shifts” and This will show the number of processes

One service instance of the OPP service is by default. This seeded OPP service instance has one work shift with one process

We can see the parameter of the threads as below:

oracle.apps.fnd.cp.opp.OPPServiceThread:2:0:max_threads=5

General Issues with OPP

 CONC-PP NO RESPONSE:

Setting the profile option Concurrent: OPP Response Timeout to a higher value may be necessary.

Solution:

By default, a timeout will occur if it takes longer than 120 seconds (2 min.) for the Output Post Processor to pick up the request from the Concurrent Manager process. Increase the same if required.

CONC-PP TIMEOUT:

By default, a timeout will occur if it takes longer than 300 seconds (5 min.) for the XML Publisher engine to generate the output file. We will need to increase the timeout if necessary 

java.lang.OutOfMemoryError error

A high number of threads increases the risk of running into java.lang.OutOfMemoryError errors during peak loads. So Limit  the number of threads per OPP process

For the specific concurrent program which is showing OPP related out of memory issues, think of increasing run time memory

Go to “System Administrator” responsibility.

Concurrent – > Program – > Define.

Search for the concurrent program

Set the Options field to -Xmx1024M

Save.

Bounce the Concurrent Manager for the effects to take place.

How to restart OOP Services



Using System Administrator responsibility navigate to

Concurrent–>Manager–>Administer ->Output Post Processor


Deactivate and Activate

Alternatively, click on Restart.

If not stopped/deactivated check the processes and kill and try to start


ps -ef |grep java | grep -v grep | grep FNDOPP

Comments

Post a Comment

Popular posts from this blog

How to drop index and before dropping it how to get the DDL.

  Oracle DROP INDEX statement DROP INDEX [schema_name.]index_name; DECLARE index_count INTEGER; BEGIN SELECT COUNT(*) INTO index_count FROM USER_INDEXES WHERE INDEX_NAME = 'index_name' ; IF index_count > 0 THEN EXECUTE IMMEDIATE 'DROP INDEX index_name' ; END IF; END; / In case want to check whether index exists and then delete it. If try to drop a non-existing, you will get the following error: SQL Error : ORA -01418 : specified index does not exist In SQLplus, set these before running the commands set long 100000 set longchunksize 100000 select DBMS_METADATA.GET_DDL( 'INDEX' , 'index_name') from all_indexes where owner in ( USER , 'USER_OTHER_THAN_LOGGED_IN_USER' ); Use below script to fetch metadata other information. SELECT DBMS_METADATA,GET_GRANTED_DDL('SYSTEM_GRANT','SSO******') from dual;
Read more

PRVG-11250 : The check "RPM Package Manager database" was not performed because

Following message reported from cluvfy while performing pre-requisite for Clusterware Installation:  Verifying RPM Package Manager database ...INFORMATION PRVG-11250 : The check "RPM Package Manager database" was not performed because it needs 'root' user privileges. SOLUTION Execute runcluvfy manually with "-method root" option and enter root password when prompted : Example: $/runcluvfy.sh stage -pre crsinst -n <HOSTNAME1>,<HOSTNAME2> -method root Enter "ROOT" password: For reference metalink doc id :  2548970.1
Read more

How to troubleshoot Long Running Concurrent Request in EBS 12.2

 The following points need to answer before jumping into troubleshooting: Are there any recent code changes done in the concurrent program?           Check long_running_sid and verify the recent code change of last_ddl_timestamp after a             discussion with the developer. Was this running long in the last few run as well, or was this time only?            ==> Check the program history. How much time does it take to complete?           Once again program history can give us some idea. Are these jobs fetching higher data compared to the last run           ===> Confirm on data change . Query Plan change will indicate this issue. Does this job runs at any specific time or it can be run anytime           ===> The nature of the job can help to understand the job's nature. Does this job fetch data using DB Link...
Read more

ORA-00257:archiver error, connect internal only until freed

  Issue: Application log shows below error and users are unable to connect to the database. 0RA-00257:archiver error, connect internal only until freed (OR) 0RA-00257:archiver error, connect internal only until freed ORA-16020: less destinations available than specified by LOG_ARCHIVE_MIN_SUCCEED_DEST Fix: 1) Check if there are any errors for the archive destination(s). Make sure that the number of VALID archive destinations is greater than or equal to the value specified by LOG_ARCHIVE_MIN_SUCCEED_DEST initialization parameter. set pages 1000 col dest_name for a30 col destination for a60 SELECT dest_id, dest_name, binding, status, destination, error FROM v$archive_dest; SHOW PARAMETER log_archive_min_succeed_dest 2) If space is full in one or more of the archive destinations (or if destination is not available), take any of the following steps: 2.a) Manually move the archives to another location and delete them from archive destination.   (OR) 2.b) Change the archive des...
Read more

Verifying Daemon “Avahi-Daemon” Not Configured And Running …FAILED (PRVG-1360)

  Execute "/tmp/CVU_19.0.0.0.0_grid/runfixup.sh" as root user on nodes "shsebs7idb1" to perform the fix up operations manually Press ENTER key to continue after execution of "/tmp/CVU_19.0.0.0.0_grid/runfixup.sh" has completed on nodes "shsebs7idb1" Fix: Daemon "avahi-daemon" not configured and running   Node Name                             Status   ------------------------------------  ------------------------   shsebs7idb1                           failed ERROR: shsebs7idb1: PRVG-9023 : Manual fix up command "/tmp/CVU_19.0.0.0.0_grid/runfixup.sh" was not issued by root user on node "shsebs7idb1" Result: "Daemon "avahi-daemon" not configured and running" could not be fixed on nodes "shsebs7idb1" SOLUTION: To fix this error, stop and disable avahi-daemon service. 1. Check avahi-daemon status: [root@shsebs7idb1 ~]# ls -...
Read more

Linux OL7/RHEL7: PRVE-0421 : No entry exists in /etc/fstab for mounting /dev/shm

  This is due to below bugs: OL7 -  Unpublished Bug 21441387- 12.2/MAIN/OL7.1: CVU REPORTS /DEV/SHM NOT MOUNTED WHEN IT IS MOUNTED  21441387.8 RHEL 7 - Unpublished Bug 25907259 : RHEL7.2: CVU REPORTS /DEV/SHM NOT MOUNTED WHEN IT IS MOUNTED  25907259.8 As a work-around in OL7: In OL7, the /dev/shm is not mounted in /etc/fstab on a fresh system, it is done in: [root@testc140 u02]# ll /usr/lib/dracut/modules.d/99base/init.sh -rwxr-xr-x. 1 root root 11379 Mar 26 10:14 vi /usr/lib/dracut/modules.d/99base/init.sh and comment the below lines 62 if ! ismounted /dev/shm; then 63 mkdir -m 0755 /dev/shm 64 mount -t tmpfs -o mode=1777,nosuid,nodev,strictatime tmpfs /dev/shm >/dev/null 65 fi so, this validation is not really valid for OL7.    We can also safely ignore this message. For reference metalink id: -  Doc ID 2065603.1
Read more

CPU Patch Analysis

 Oracle CPU patch analysis Identifying the Latest Critical Patch Update for Oracle E-Business Suite Release 12.2 (Doc ID 2484000.1) Database Quick-Link If you only care about the database patches, this MOS note steps you through really easily. Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patch sets and Base Releases (Doc ID 2118136.2) https://www.oracle.com/security-alerts/ Critical Patch Updates Critical Patch Updates provide security patches for supported Oracle on-premises products. They are available to customers with valid support contracts. Starting in April 2022, Critical Patch Updates are released on the third Tuesday of January, April, July, and October (They were previously published on the Tuesday closest to the 17th day of January, April, July, and October). The next four dates are: 16 April 2024 16 July 2024 15 October 2024 21 January 2025 A pre-release announcement will be published on the Thursday preceding eac...
Read more

How to write to a CSV file using Oracle SQL*Plus

  SPOOL command  The  SPOOL  command is  unavailable  in the browser-based SQL*Plus version,  iSQL*Plus . To generate files while using iSQL*Plus, change the necessary preference settings to directly output to a file. This is accomplished using the  SPOOL  statement. While  SPOOL  is  active , SQL*PLus will store the output of any query to the specified file. Therefore, the next command to enter is  spool : spool filepath Skipping ahead slightly,  after  your query is inserted, you also need to halt  spool  so the file output is closed by using the  spool off  command: spool off Insert the query  The last step after the settings are modified and  spool  is running is to insert your query. For our simple example, we’re outputting all books from our  books  table. SELECT   title,   author FROM   authors; Don’t forget the semi...
Read more

SKIP DNS RESLOV.CONF CHECK DURING RAC CONFIGURATION

 shsebs7idb1: PRVG-2064 : There are no configured name servers in the file              '/etc/resolv.conf' on the nodes "shsebs7idb1" Solution: [root@shsebs7idb2 ~]# ls -ltr /etc/resolv.conf -rw-r--r--. 1 root root 78 Jul  3 11:02 /etc/resolv.conf [root@shsebs7idb2 ~]# mv /etc/resolv.conf /etc/resolv.conf.sid Rename resolv.conf to resolv.conf.sid for example and re run the installation again . It  worked for me
Read more

How to fix Oracle SQL Developer connection issue "Got minus one from a read call"

Image
 During connection from SQL Developer to EBS database got below error message: Status : Failure - Test failed: IO Error: Got minus one from a read call Following solution can be done : 1. Go to directory $ORACLE_HOME / network / admin 2. Modify sqlnet.ora file with following parameter : tcp.validnode_checking = no 3. If you don 't want to disable this, you can put the machine names as follows: tcp.invited_nodes=(machine1, machine2) 3. Bounce the listener. Due to security enhancements it is not good to put tcp.validnode_checking = no or comment out will through error when we start listener like below: Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=shsebs7idb1.appsolworld.com)(PORT=1522))) TNS-00584: Valid node checking configuration error TNS-12560: TNS:protocol adapter error Better plan is to add localhost or IP address of client (Local Computer) in the sqlnet.ora file That will solve the problem NAMES.DIRECTORY_PATH=(TNSNAMES, ONAMES, HOSTNAME) SQLNET.EXPI...
Read more