Oracle Concurrent Manager is a important piece in Oracle E-Business Suite product. it helps in batch processing of many things.
I am here presenting the some details about it
It consists of several part. Explaining each of these in details .It gives u a glimpse of How an E-Business Suite Concurrent Manager Process Works
Types of Oracle Concurrent Manager
Internal Concurrent Manager (ICM)
The Internal Concurrent Manager (ICM) can be considered the “brain” of concurrent processing. It is responsible for the following functionality:
-Starts all other processes like Conflict resolution manager,standard manager
-Executes “control requests” submitted by the administrator.
-Activate/Deactivate/Abort Concurrent Manager
-Terminate Concurrent Request
-Monitors processes, restarting any that failed.
-Sets the target number of processes for each service based on the current work shift.
Starting the ICM
-adcmctl.sh script
-TNS Apps Listener must be started before starting ICM
Shutting down the ICM
-Shutting down the ICM will stop all other services like Conflict resolution manager,standard manager
– Normal shutdown signals processes to exit after completing their current tasks.
– Abort will terminate service processes.
– ICM will not exit until all other processes have exited.
-Use adcmctl.sh to shutdown ICM.
Service Managers(FNDSM)
Service Managers are spawned on the middle-tier nodes of a GSM enabled system in order to act as an agent of the ICM. When the ICM sees that it needs an service managers to perform some function, such as start a concurrent manager process, on a middle-tier node, it will make remote procedure control calls to the Apps listener on that node to start the Service manager. Once the Service Manager has been started and initialized, the ICM communicates directly to the Service manager through Remote Procedural call(RPC), giving it information to manage the services on that node.
-The Service manager is spawned from the APPS TNS Listener
– The APPS TNS Listener must be started on every middle-tier node in the system, and started by the user that starts ICM (e.g. applmgr)
-TNS Listener spawns Service Manager to run as agent of ICM for the local node
-The Service Manager is started by ICM on demand when needed. If no management actions are needed on a node a Service Manager will not be started by ICM until necessary. When ICM exits its Service Managers exit as well.
-The Service Manager environment is set by APPSORA.env as defined in listener.ora
-The listener.ora and tnsnames.ora files must be configured properly for the listener to be able to spawn the Service Manager and for the ICM to be able to check the status of the Service Manager.
Internal Monitors(FNDIMON)
Internal Monitors are used specifically in Parallel Concurrent Processing to allow for Internal concurrent manager fail over to other available middle tier nodes.
-Place an Internal Monitor on any node where the ICM can start in case of a failure.
-Internal Monitors are seeded on every registered node by default.
-If the ICM goes down, the Internal Monitor will attempt to start a new ICM on the local node.
-If multiple ICMs are started, only the first will stay active. The others will gracefully exit.
Oracle Concurrent Managers(FNDLIBR,INVLIBR)
Concurrent Managers provide asynchronous job processing by monitoring the FND_CONCURRENT_REQUESTS table on a continuous cycle. The job of a concurrent manager is to execute concurrent requests that are in Pending / Normal phase / status and that it is qualified to run according to its specialization rules.
Concurrent Manager Processes
– Act independently
– Select only requests that: (a) match the manager specialization rules, (b) are Pending/Normal, (c) have a requested start time
Description about important Oracle Concurrent Manager oracle tables
FND_CONCURRENT_QUEUES
TARGET_NODE
-Used to indicate where additional processes should be started
-Used by managers to determine if they should shut themselves down for migration
-Managers compare parameter value passed on startup to this value
-Used by UI to indicate where processes exist (not entirely accurate in migrating case)
-Assigned by ICM based on primary, secondary settings
NODE_NAME
-Indicates primary node for PCP – directed load
-Where processes should be started unless node is not online or has been determined to be unavailable
-If no node is specified ICM will assign target node by default to NODE_NAME2
-Indicates secondary node for PCP – directed load fail-over
-Only assigned as TARGET_NODE if primary node is unavailable
FND_CONCURRENT_PROCESSES
NODE_NAME
-Indicates where the manager process is running
-Also indicates where the manager’s files exist
-Populated using value from uname() (physical machine name)
-Used by ICM when terminating process
-Used when viewing log file in User Interface(UI)
-Used by Purge Program to delete process logfile
-Could be used for workload statistic calculations
-Will likely be used by RPMs to attempt to find a local OPP
-Similarly used in 11i.X to locate Reports Server
FND_CONCURRENT_REQUESTS
LOGFILE_NODE_NAME, OUTFILE_NODE_NAME
-Indicate where files exist
-Used in User Interface(UI) for File Viewing
-Used by Purge Program to delete files
-Value populated by mgr process, based on its own node
FND_NODES
NODE_NAME
-Indicates node name where DBC file is located.
-Adgendbc.sh script creates DBC file.
SERVER_ID
-Will be used to authenticate connections from the node.
-Updated by adgendbc.sh which calls AdminAppServer API.
Concurrent Manager tables
FND_NODES: Contains all the nodes level information
FND_CONCURRENT_PROCESSES : Contains all the concurrent manager process information
FND_CONCURRENT_REQUESTS : Contains a complete history of all concurrent requests (both past history and those scheduled to run in the future).
FND_CONCURRENT_QUEUES : Contains the information for all the concurrent manager created in the system
FND_CONCURRENT_PROGRAMS : Contains the information for all the concurrent program available in the system
FND_EXECUTABLES
FND_CP_SERVICES
FND_CONCURRENT_QUEUE_SIZE
FND_CONCURRENT_QUEUE_CONTENT
FND_CONCURRENT_PROGRAM_SERIAL
FND_CONCURRENT_TIME_PERIODS
FND_CONCURRENT_PROCESSORS
FNDSVCRG
The FNDSVCRG executable is triggered from the control scripts before and after a script starts or stops the service. FNDSVCRG will connect to the database to verify the configuration of the Seeded GSM Service. If the service in question is not enabled to be managed under GSM, the FNDSVCRG executable will do nothing and exit. The script would then continue to perform its normal start/stop actions. If the service is enabled for GSM management, the FNDSVCRG executable will update service-related information in the database including the environment context, the current service log file location, and the current state of the service
Concurrent Request Phase status Description
PENDING/Normal -Request is waiting for the next available manager.
PENDING/Standby-Program to run request is incompatible with other program currently running.
PENDING/Scheduled-Request is scheduled to start at a future time or date.
PENDING/Waiting-A child request is waiting for its Parent request to mark it ready to run. For example, a request in a request set that runs sequentially must wait for a prior request to complete.
RUNNING/Normal-Request is running normally.
RUNNING/Paused-Parent request pauses for all its child requests to finish running. For example, a request set pauses for all requests in the set to complete.
RUNNING/Resuming -All requests submitted by the same parent request have completed running. The Parent request resumes running.
RUNNING/Terminating-Request is terminated by choosing the Cancel Request button in Requests window.
COMPLETED/Normal-Request completed successfully.
COMPLETED/Error-Request failed to complete successfully.
COMPLETED/Warning-Request completed with warnings. For example, a request is generated successfully but fails to print.
COMPLETED/Cancelled-Pending or Inactive request is cancelled by choosing the Cancel Request button in the Requests window.
COMPLETED/Terminated-Request is terminated by choosing the Cancel Request button in the Requests window.
INACTIVE/Disabled-Program to run request is not enabled. Contact your system administrator.
INACTIVE/On Hold-Pending request is placed on hold by choosing the Hold Request button in the Requests window.
INACTIVE/No Manager-No manager is defined to run the request. Check with your system administrator. A status of No Manager is also given when all managers are locked by run-alone requests.
how to start concurrent manager in oracle apps r12
Start Concurrent Manager in R12
Connect to Application Tier user usually its applmgr
cd $ADMIN_SCRIPTS_HOME ./adcmctl.sh start apps/<apps-pass>
how to stop concurrent manager in oracle apps r12
Stop Concurrent Manager in R12
Connect to Application Tier user usually its applmgr
cd $ADMIN_SCRIPTS_HOME ./adcmctl.sh stop apps/<apps-pass>
how to check status concurrent manager in oracle apps r12
To check Status of Concurrent Manager
Connect to Application Tier user usually its applmgr
cd $ADMIN_SRCIPTS_HOME ./adcmctl.sh status apps/<apps-pass>
Concurrent Manager Log file location in R12
Concurrent Manager,ICM and concurrent request all generate the log files
A) Concurrent Request Log File – documents the execution of a particular request ( l.req )
B) Manager Log File – documents the performance of a concurrent manager process. ( W.mgr )
C) Internal Manager Log File – documents the performance of the ICM.(std.mgr). This log file displays the parameters used with the ’adcmctl’ command.
if $APPLCSF is set
Log files are in the Folder $APPLCSF/$APPLLOG.
Log files can also be viewed from within the applications from the View Concurrent Requests Form
R12.2 APPLCSF =$NE_BASE/inst/<CONTEXT_NAME>/logs/appl/conc/log
R12.1 APPLCSF= $INST_TOP//logs/appl/conc/log
If $APPLCSF is not set
Log files are in the Folder $PRODUCT_TOP/$APPLLOG.
Similarly for output files,
if $APPLCSF is set
R12.2 APPLCSF= $NE_BASE/inst/<CONTEXT_NAME>/logs/appl/conc/
R12.1 APPLCSF= $INST_TOP//logs/appl/conc/
Concurrent Manager troubleshooting
How do you check the status of the Oracle concurrent managers from the OS
–Linux command:
$ ps -ef | grep LIB
-Note that the Internal Concurrent Manager can be spotted in this listing because its command is “FNDLIBR FND CPMGR …” whereas the others show more like “FNDLIBR FND Concurrent_Processor …”
-The Unix userid shown in the first column of this listing is crucial: these concurrent manager processes should be owned by the same Unix userid which owns the Applications code ($APPL_TOP and its subdirectories); this user is usually referred to as “applmgr”
Where do all the files generated by the oracle concurrent managers go
-The ICM log file goes in the $FND_TOP/log directory and usually matches std.mgr.
-The workers’ log files go in $FND_TOP/log and match W.mgr
-The concurrent request log/out files go under the product top directory associated with the product running the request: for instance, log/out files for AR reports go under $AR_TOP.
-The log files for concurrent requests go in the $APPLLOG subdirectory under the appropriate product top directory and match l.req
-The out files for concurrent requests go into the $APPLOUT subdirectory
-If APPLCSF is set, it should point to the complete path to a directory which has $APPLLOG and $APPLOUT subdirectories. This $APPLCSF directory will be used instead of the various product top directories to write
all the log/out files to.
The most common concurrent manager problems are caused by file protection problems at the Unix/linux level.
-Are you starting the concurrent managers as applmgr?
-Can applmgr do the following to create a file in the
$FND_TOP/$APPLLOG directory?
$FND_TOP/$APPLOUT directory?
Unix: $ touch $FND_TOP/$APPLLOG/a
-If this fails, who owns the directory?
Unix: $ ls -ld $FND_TOP/$APPLLOG
-Is this directory a symbolic link? if so, what are the protections on the directory it points to?
-Are you running out of disk space on this partition? i-nodes?
Unix: $ df -k
Unix (on some systems) to check i-nodes: $ df -i
-Is APPLCSF set?
-If so, can applmgr do this?
Unix: $ touch $APPLCSF/$APPLLOG/a
-Check $APPLOUT (usually “out”) directories just like log directories.
If a PL/SQL Concurrent Program can’t write to an external file, you will receive an error message similar to:
MSG-00102: Error Message :ORA-20100: File o0000071.tmp creation for FND_FILE failed. You will find more information on the cause of the error in request log. ORA-06512: at "APPS.FND_FILE", line 378 ORA-06512: at "APPS.FND_FILE", line 473 ORA-06512: at "APPS.AP_XYZ", line 192 REP-1419: 'beforereport': PL/SQL program aborted.
NOTE: Applications also produces temporary PL/SQL output files used in concurrent processing. These files are written to a location on the database server node specified by the APPLPTMP environment setting. The APPLPTMP directory must be the same directory as specified by the utl_file_dir parameter in your database initialization file.
.
Rapid Install sets both APPLPTMP and the utl_file_dir parameter to the same default directory. As the temporary files placed in this directory may contain context sensitive information, it should be a secure directory on the database server node with read and write access for the database server owner. In a multi-node system, the directory defined by APPLPTMP does not need to exist on the application tier servers. During an upgrade with AutoUpgrade, you must provide the utl_file_dir parameter value for the APPLPTMP environment setting.
To isolate where the problem is, verify the following:
1) Make sure that the name of the file is valid (the file name should not include characters like “^”)
2) Make sure that APPLPTMP is set to a valid directory and that BOTH the applmgr user and the database user have read and write permissions on that directory (normally, it can be set to the same directory as APPLTMP)
3) Make sure that the file does not exit on the directory pointed by APPLPTMP
4) Make sure the directory pointed by APPLPTMP is the first entry on the utl_file_dir. Also, verify that all the entries on the utl_file_dir are valid and that the applmgr has read/write permissions.
If using an spfile, verify the proper syntax to set utl_file_dir:
Ex.
ALTER SYSTEM SET UTL_FILE_DIR='directory1','directory2' scope=spfile;
5) If still having problems, check if you can write a file directly using FND_FILE, which is the package used by the Application. From SQLPLUS, connected as the apps user, run:
SQL> exec FND_FILE.PUT_LINE(FND_FILE.LOG, 'THIS IS A TEST');
This should dump a file on APPLPTMP.
If this test works, it would indicate that FND_FILE is OK and the problem is possibly with the Application.
You may want to leave only one entry on utl_file_dir for this test.
6) If still having problems, check if you can write a file using UTL_FILE, which is used by FND_FILE.
Run the PL/SQL below, changing to the first entry on utl_file_dir (you may want to leave just one entry on utl_file_dir for this test).
set serveroutput on
DECLARE file_location VARCHAR2(256) := '';
file_name VARCHAR2(256) := 'utlfile1.lst';
file_text VARCHAR2(256) := 'THIS IS A TEST';
file_id UTL_FILE.file_type;
BEGIN
file_id := UTL_FILE.fopen(file_Location,file_name, 'W');
UTL_FILE.put_line(file_id, file_text);
UTL_FILE.fclose(file_id);
EXCEPTION
WHEN UTL_FILE.INVALID_PATH
THEN
dbms_output.put_line('Invalid path ' || SQLERRM);
WHEN OTHERS
THEN
dbms_output.put_line('Others '|| SQLCODE || ' ' || SQLERRM);
END;
/
This program should dump a file on the requested directory. If the test fails, the problem is probably on the Database side.
CONCURRENT PROCESSING SERVER SCRIPTS
afcmstat.sql Displays all the defined managers, their maximum capacity, pids, and their status.
afimchk.sql Displays the status of ICM and PMON method in effect, the ICM’s log file, and determines if the concurrent manger monitor is running.
afcmcreq.sql Displays the concurrent manager and the name of its log file that processed a request.
afrqwait.sql Displays the requests that are pending, held, and scheduled.
afrqstat.sql Displays of summary of concurrent request execution time and status since a particular date.
afqpmrid.sql Displays the operating system process id of the FNDLIBR process based on a concurrent request id. The process id can then be used with the ORADEBUG utility.
afimlock.sql Displays the process id, terminal, and process id that may be causing locks that the ICM and CRM are waiting to get. You should run this script if there are long delays when submitting jobs, or if you suspect the ICM is in a gridlock with another oracle process.
How to do concurrent Manager Tuning
Tuning the Internal Concurrent Manager (ICM)
The ICM performance is affected by the three important Oracle parameters PMON cycle, queue size, and sleep time.
PMON cycle — This is the number of sleep cycles that the ICM waits between the time it checks for concurrent managers failures, which defaults to 20. You should change the PMON cycle to a number lower than 20 if your concurrent managers are having problems with abnormal terminations.
Queue Size — The queue size is the number of PMON cycles that the ICM waits between checking for disabled or new concurrent managers. The default for queue size of 1 PMON cycle should be used.
Sleep Time — The sleep time parameter indicates the seconds that the ICM should wait between checking for requests that are waiting to run. The default sleep time is 60, but you can lower this number if you see you have a lot of request waiting (Pending/Normal). However, reducing this number to a very low value many cause excessive cpu utilization.
Adjusting the Individual Concurrent Manager Cache Size
Concurrent manager performance can also be enhanced by increasing the manager cache size to be at lease twice the number of target processes. The cache size specifies the number of requests that will be cached each time the concurrent manager reads from the FND_CONCURRENT_REQUESTS table. Increasing the cache size will boost the throughput of the managers by attempting to avoid sleep time.
Purging Concurrent Requests
It is seen that When the records in FND_CONCURRENT_PROCESSES and FND_CONCURRENT_REQUESTS exceed 50K, you can start to experience serious performance problems within your Oracle Applications. Inorder to avoid these issues, we should regularly purge data in these tables usig specific request called “Purge Concurrent Requests And/Or Manager Data” .It should be scheduled to run on a regular basis. This request can be configured to purge the request data from the FND tables as well as the log files and output files on accumulate on disk.
Analyzing Oracle Apps Dictionary Tables for High Performance
Concurrent Manager tables may go fragmented over time, So it is recommended to rebuild them in regular maintaince
Also It is also very important to run the request Gather Table Statistics
Some of the Important Tables are
FND_CONCURRENT_PROCESSES
FND_CONCURRENT_PROGRAMS
FND_CONCURRENT_REQUESTS,
FND_CONCURRENT_QUEUES.
I hope you like this post on Oracle Concurrent Manager .
Also Read
Concurrent Manager Queries :This article contains awesome top 30 Concurrent Manager Queries for Concurrent Manager troubleshooting ,resolution ,run time, details
ORA-01427 :Check out this for the solution on ORA-01427: single-row subquery returns more than one row error ,how to resolve it when it happens with Concurrent Manager
request set in oracle apps : Request set gives the capability to submit the same set of requests regularly using a single transaction.
Concurrent Manager Interview questions ::Check out 24 Concurrent Manager Interview questions to help you in EBS interview. This consists of all sort of question on standard manager,service manager
Parallel Concurrent Processing :What is PCP, How to setup it, how to define internal monitor
Oracle Concurrent Manager :How an E-Business Suite Concurrent Manager Process Works,Oracle Concurrent Manager,What is internal monitor,What is service manager and troubleshooting
https://docs.oracle.com/cd/E18727_01/doc.121/e12893/T174296T174302.htm
Recommended Courses
The following are some of the recommended courses you can buy if you want to get a step further
Given below are the links to some of the courses
Oracle DBA 11g/12c – Database Administration for Junior DBA : This course is good for the people who are starting as Junior DBA or aspire to be Oracle DBA. This will provide a good understanding of backup & recovery and General administration tasks
Oracle Database: Oracle 12C R2 RAC Administration : This course covers the installation, administration of Oracle RAC. A good course for Oracle DBA who want to upgrade his skills for Oracle RAC
Oracle Data Guard: Database Administration for Oracle 12C R2 : This course covers the installation, administration of Oracle Dataguard. A good course for Oracle DBA who want to upgrade his skills for Oracle Dataguard
