500 Internal Server Error in R12

Troubleshooting 500 Internal Server Error in Oracle E-Business suite

R12: Troubleshooting 500 Internal Server Error in Oracle E-Business suite [ID 813523.1]
Error Block:
Symptom:1
Solution:1
Symptom:2
Solution:2
Alternative Solution

Applies to:

Oracle Applications Technology Stack - Version: 12.0 to 12.0.6 - Release: 12.0 to 12.0
Generic UNIX
IBM IA64 AIX
Generic Linux
HP-UX PA-RISC (32-bit)
Oracle Solaris on x86 (32-bit)
Oracle Solaris on SPARC (64-bit)

Purpose

The troubleshooting guide is provided to assist in solving various types of '500 Internal Server Error' issues that occur in the browser window while accessing Oracle E-Busines suite R12. Though the error is very generic, the underlying error noticed in logfiles may differ. Hence, the guide includes those different types of error examples scenarios that relate problem symptoms and various solutions available for resolution.

Oracle Application Server 10.1.3 (OracleAS 10.1.3) is installed as part of Oracle Applications Release 12, and consists of the following components:

* Oracle Process Manager and Notification Server (OPMN)
* Oracle HTTP Server (Apache)
* Oracle Containers for J2EE (OC4J)
* Configured Application Modules
OACORE - Core Application Modules
OAFM - Oracle Transport Agent, MapViewer
FORMS - Forms (using Servlet Mode)

Instructions for the Reader

A Troubleshooting Guide is provided to assist in debugging a specific issue. When possible, diagnostic tools are included in the document to assist in troubleshooting.
Troubleshooting Details

500 Internal Server Error

The HTTP Response status codes beginning with the digit "5" indicates cases in which the server is aware that it has encountered an error or is otherwise incapable of performing the request. Any client (e.g. the Web browser ) goes through the following cycle when it communicates with Web server:

* Obtain an IP address from the IP name of web site .
* This lookup (conversion of IP name to IP address) is provided by domain name servers (DNS).
* Open an IP socket connection to that IP address.
* Write an HTTP data stream through that socket.
* Receive an HTTP data stream back from your Web server in response. This data stream contains status codes whose values are determined by the HTTP protocol. Parse this data stream for status codes and other useful information. Now this error occurs in the final step when the client receives an HTTP status code that it recognises as '500'.

The error '500 Internal Server Error' is a very generic error message. This error can only be resolved by fixes to the Web server end and generally not a client-side problem. Hence it is important to locate and analyse the logs which should give further information about the error.

Error Block:

Check each of the error blocks for the complete error details. If it is matching the error details what is observed at your instance then refer the suggested solution.
Symptom:1

Browser Error : If You receive following error while accessing E-Business R12 instance

500 Internal ServerError
java.lang.NoClassDefFoundError
at oracle.apps.fnd.sso.AppsLoginRedirect.AppsSetting(AppsLoginRedirect.java:120)
at oracle.apps.fnd.sso.AppsLoginRedirect.init(AppsLoginRedirect.java:161)

Caused by: java.lang.Throwable: java.lang.RuntimeException:
oracle.apps.jtf.base.resources.FrameworkException: Cache not initialized
at oracle.apps.jtf.cache.CacheManager.registerCacheComponent(CacheManager.java:4691)
at oracle.apps.jtf.cache.CacheManager.registerCacheComponent(CacheManager.java:1731)
at oracle.apps.jtf.cache.CacheAdmin.registerComponentCacheNoDBSave(CacheAdmin.java:309)
at oracle.apps.fnd.cache.Cache.registerCache(Cache.java:159)
at oracle.apps.fnd.cache.Cache.initCache(Cache.java:123)
at oracle.apps.fnd.cache.Cache.(Cache.java:89)
at oracle.apps.fnd.cache.AppsCache.(AppsCache.java:86)
at oracle.apps.fnd.cache.AolCaches.getCache(AolCaches.java:155)
at oracle.apps.fnd.profiles.Profiles.(Profiles.java:247)
at java.lang.J9VMInternals.initializeImpl(Native Method)


Application.log:

Error initializing servlet
java.lang.NoClassDefFoundError
at oracle.apps.fnd.sso.AppsLoginRedirect.AppsSetting(AppsLoginRedirect.java:120)
at oracle.apps.fnd.sso.AppsLoginRedirect.init(AppsLoginRedirect.java:161)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].server.http.HttpApplication.loadServlet(HttpApplication.java:2231)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].server.http.HttpApplication.findServlet(HttpApplication.java:4617)
The above error has been observed both intermittently or sometimes consistently while accessing E-Business R12 instance via browser. In both cases the issue is with the port value for the java object cache. Either a port conflict at startup or multiple instances with JOC in distributed mode attempting to use the same port resulting in intermittent errors.

Solution:1

1. Choose a correct value or another value for $CONTEXT_FILE variable: s_java_object_cache_port

Make sure your System Profile "JTF_DIST_CACHE_PORT" has the same port specified as the value mentioned for variable: s_java_object_cache_port in context file. You can check the sytem profile value by running following sql

select fnd_profile.value('JTF_DIST_CACHE_PORT') from dual;
2. Please check the GUEST password are set properly in following places:
- DBC file under $INST_TOP/appl/fnd/12.0.0/secure
- System Profile option: 'Guest User Password'
select fnd_web_sec.validate_login('GUEST','') from dual ; should return 'Y'
- In context file by running command : grep -i guest $CONTEXT_FILE

For any issues with Guest user password mismatch, please refer the following note
Note 443353.1 -- How To Successfully Change The Guest Password In E-Business Suite 11.5.10 and R12

Symptom:2


Browser Error : The following error occurs in AIX platform

500 Internal Server Error
java.lang.NoClassDefFoundError: oracle.apps.fnd.profiles.Profiles (initialization failure)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:123)
at oracle.apps.fnd.sso.AppsLoginRedirect.AppsSetting(AppsLoginRedirect.java:120)
at oracle.apps.fnd.sso.AppsLoginRedirect.init(AppsLoginRedirect.java:161)
at com.evermind[Oracle Containers for J2EE 10g (10.1.3.0.0) ].server.http.HttpApplication.loadServlet(HttpApplication.java:2231)

Application.log
Exception in static block of jtf.cache.CacheManager. Stack trace is:
oracle.apps.jtf.base.resources.FrameworkException: IAS Cache initialization failed. The Distributed
Caching System failed to initialize on port: 12345. The list of hosts in the distributed cac
hing system is: . The port 12345 should be free on each host running the JVMs. The default port 12345 can be overridden
using -Doracle.apps.jtf.cache.IASCacheProvidercacheProvider.port=
The Java object Controller issue are generally specific to AIX platform. This failure was a result of the Applications JDK missing required java.security updates specific to the applications server and a port specific code issue within cache.jar.

Solution:2

1. Apply both the following patches:

Patch <<5261515>> "AS10.1.3: PATCH OVER AIX5L JDK15"
Patch <<5946958>> "JOC DISTRIBUTED MODE FAILURE WHEN REGISTERING GROUP MEMBERS"
Unpublished Bug <<5968938>> has been filed to include 5946958 in OracleAS 10.1.3.4

For more details, you can refer following note
Note 422766.1 -- Cache Initialization Failure on AIX platform

2. Sometimes those kind of error is also caused by firewall between concurrent manager node vs web node or database vs web node. Hence ensure to open the required firewall port.

Alternative Solution

Incase the above solutions doesnot solve the issue, then following steps can also be carried out after taking proper backup of the system
1. Remove any *.lock files in $ORA_CONFIG_HOME/10.1.3/j2ee//persistence/<group_default_group_/

2. Set the parameter LONG_RUNNING_JVM=false present in $ORA_CONFIG_HOME/10.1.3/j2ee/oacore/config/oc4j.properties and test the result.

3. Ensure that all the middle tier services are running fine with following command

$ adopmnctl.sh status -l
You are running adopmnctl.sh version 120.4.12000000.3
Checking status of OPMN managed processes...

Processes in Instance: test9_celaixclu9.celaixclu9.us.oracle.com
---------------------------------+--------------------+---------+---------
ias-component | process-type | pid | status
---------------------------------+--------------------+---------+---------
OC4JGroup:OC4J | OC4J:oafm | 548916 | Alive
OC4JGroup:OC4J | OC4J:forms | 307358 | Alive
OC4JGroup:OC4J | OC4J:oacore | 581638 | Alive
HTTP_Server | HTTP_Server | 598042 | Alive



4. Ensure that the instance meets Operating Systems Requirements, Software Requirements that is specified in Oracle Applications E-Business R12 Installation document.

5. If the instance has been created by cloning another instance or any issues with listener configuration is suspected then create a clean autoconfig envrionment and rerun autoconfig. Refer the following note

Note 391406.1 -- How to get a clean Autoconfig Environment

6. Clear the browser cache and E-Business suite instance cache.

7. Compile all JSP files after sourcing the environment. Run the command

cd $FND_TOP/patch/115/bin
ojspCompile.pl --compile --flush

8. If feasible, completely reboot the instance. Stop all middle tier services and stop database services as well OS services. Then restart all those services.

While you log a service request be sure to provide following information :

1. Access the following URL and complete all tests. Upload the result output to the associated metalink SR

http://host.domain:port/OA_HTML/jsp/fnd/aoljtest.jsp

(Replace the host, domain, port value with your instance details )

2. Upload the debug log files instance after reproducing the issue. You may refer following note for the same
Note 422419.1 -- R12 - How To Enable and Collect Debug for HTTP, OC4J and OPMN