Pages

Thursday, July 11, 2013

ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)



Problem Statement:-

 I have just finished up with the SOA Suite 11.1.1.7 environment, and started the soa_server1 using ‘startManagedWebLogic.sh’ script and server bought up without any problem, but after shutting down when I tried starting through node managed it was keep failing and was giving the below error.

ERROR: transport error 202: bind failed: Address already in use
ERROR: JDWP Transport dt_socket failed to initialize, TRANSPORT_INIT(510)
JDWP exit error AGENT_ERROR_TRANSPORT_INIT(197): No transports initialized [../../../src/share/back/debugInit.c:690]
FATAL ERROR in native method: JDWP No transports initialized, jvmtiError=AGENT_ERROR_TRANSPORT_INIT(197)
[ERROR] aborted
JRockit aborted: Unknown error (50)
<Jul 11, 2013 11:24:52 AM> <FINEST> <NodeManager> <Waiting for the process to die: 24746>
<Jul 11, 2013 11:24:52 AM> <INFO> <NodeManager> <Server failed during startup so will not be restarted>
<Jul 11, 2013 11:24:52 AM> <FINEST> <NodeManager> <runMonitor returned, setting finished=true and notifying waiters>

Solution:-

The error message was pretty clear that, while starting weblogic server is not able to bind with some port, now there were three challenges here-

1) What is the port which weblogic failed to bind?
2) Which PID is using that unidentified port?
3) Where to change that port value?

Port identification:-

Weblogic servers does not have much ports, as per my understanding as of now only few ports been used e.g. http listen port, ssl port, alsb debug port and java debug port. 

I was sure about http listen port and ssl configuration ports number for other servers on same machine, so there was no point to look into that area, but debug port is something I believe could cause the issue.

Looking at server start up parameter in .out file, I was able to figure out port 8453 was getting used, and I believe that was used for debugging purpose because error message says Java Debugging wire protocol (JDWP) failed to bind the port. 

So, port number been identified, next steps to check whether this port in is listening mode or not, its pretty straight forward to check the same, run the netstat command to check the same. Here is the command

“netstat –an | grep 8453’

You will see below kind of message-
tcp        0      0 0.0.0.0:8453                0.0.0.0:*                   LISTEN

So, its was confirmed this port was in use by some PID, next challenge was to findout the PID which was consuming this port. 

PID Identification:-

Goggling through internet I came across two commands which helps to check the PID based on port number in Linux environment, In my case I was using RED Hat Linux Enterprise (RHEL 5.8) version. 


Run the below command to find out used PID for these port.



Command 1: 

“lsof | grep 8453”


Below kind of message you will get in respons-


java      10387    fusion   10u     IPv4           14242572        0t0        TCP *:8453 (LISTEN)

In above message 10387 is the PID



Command2:

“fuser 8453/tcp” 


You will see below message-


8453/tcp:            10387


So 10387 is the port id.



Changing the port:-


So, now you know what is the port number, which PID is consuming this port, so next step is either kill that process which is consuming the port or change the port number to some other number for starting process/server. Its’ purely depend on your requirement and criticality of that PID which is already consuming the port 8453.


Run this command to know more about PID. 


“ps -ax | grep 10387”


You will see details about process id which will help you to decide either kill this process or change the port number to some other number for starting server which you are trying to start. 


In my case I have found that port was configured in setDomainEnv.sh file, since all the managed server in the domain which I have created were installed on same physical server, this conflict was coming. I have changed the setDomainEnv.sh file to consider different port number for all managed servers.









Wednesday, July 10, 2013

SOA Suite 11.1.1.7 Installation problem related to ‘soa-infra’ application



I have installed the SOA Suite 11.1.1.7 on RHEL 5.8 and started the soa_server1 successfully, but later realized that ‘soa-infra’ component is in failed status as result at EM console I am not able to see the ‘SOA’ module at all. 

I spent plenty of time to go through the logs file of soa_server1 managed server but was not able to figure out much why it happened since it was new installation and I have followed all the installation steps as suggested in installation guide and in logs file there were many different error message were given related to soa-infra application. Here are few of them listed –

<Jul 9, 2013 5:24:59 PM EST> <Warning> <J2EE> <BEA-160140> <Unresolved optional package references (in META-INF/MANIFEST.MF): [Extension-Name: oracle.soa.apps, Specification-Version: 11.1.1, Implementation-Version: 11.1.1, referenced from: /u01/app/oracle/domains/DEV1_eQSOADomain/servers/soa_server1/tmp/_WL_user/soa-infra/ztorh]. Make sure the referenced optional package has been deployed as a library.>

<Jul 9, 2013 5:25:06 PM EST> <Warning> <EJB> <BEA-015036> <In the EJB PeopleQueryMDB(Application: soa-infra, EJBComponent: oracle.bpm.bpm-services.ejb.jar), the value for the activation config property 'connectionFactoryJNDIName' will be ignored since it is overridden with the value 'jms/bpm/PeopleQueryConnectionFactory';>

<Jul 9, 2013 5:25:10 PM EST> <Warning> <Munger> <BEA-2156203> <A version attribute was not found in element web-app in the deployment descriptor in /u01/app/oracle/domains/DEV1_eQSOADomain/servers/soa_server1/tmp/_WL_user/soa-infra/ztorh/agadminservice.war/WEB-INF/web.xml. A version attribute is required, but this version of the Weblogic Server will assume that the JEE5 is used. Future versions of the Weblogic Server will reject descriptors that do not specify the JEE version.>

####<Jul 9, 2013 5:26:02 PM EST> <Error> <Deployer> <TCS-SOA> <soa_server1> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <11d1def534ea1be0:1510ae5:13fc251426a:-8000-0000000000000003> <1373354762472> <BEA-149231> <Unable to set the activation state to true for the application 'soa-infra'.
weblogic.application.ModuleException:
                at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:1520)
                at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:484)
                at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:425)
                at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
                at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
                at weblogic.application.internal.flow.ScopedModuleDriver.start(ScopedModuleDriver.java:201)
                at weblogic.application.internal.flow.ModuleListenerInvoker.start(ModuleListenerInvoker.java:249)
                at weblogic.application.internal.flow.ModuleStateDriver$3.next(ModuleStateDriver.java:427)
                at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
                at weblogic.application.internal.flow.ModuleStateDriver.start(ModuleStateDriver.java:119)
                at weblogic.application.internal.flow.StartModulesFlow.activate(StartModulesFlow.java:28)
                at weblogic.application.internal.BaseDeployment$2.next(BaseDeployment.java:672)
                at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:52)
                at weblogic.application.internal.BaseDeployment.activate(BaseDeployment.java:212)
                at weblogic.application.internal.EarDeployment.activate(EarDeployment.java:59)
                at weblogic.application.internal.DeploymentStateChecker.activate(DeploymentStateChecker.java:161)
                at weblogic.deploy.internal.targetserver.AppContainerInvoker.activate(AppContainerInvoker.java:79)
                at weblogic.deploy.internal.targetserver.BasicDeployment.activate(BasicDeployment.java:184)
                at weblogic.deploy.internal.targetserver.BasicDeployment.activateFromServerLifecycle(BasicDeployment.java:361)
                at weblogic.management.deploy.internal.DeploymentAdapter$1.doActivate(DeploymentAdapter.java:52)
                at weblogic.management.deploy.internal.DeploymentAdapter.activate(DeploymentAdapter.java:200)
                at weblogic.management.deploy.internal.AppTransition$2.transitionApp(AppTransition.java:31)
                at weblogic.management.deploy.internal.ConfiguredDeployments.transitionApps(ConfiguredDeployments.java:240)
                at weblogic.management.deploy.internal.ConfiguredDeployments.activate(ConfiguredDeployments.java:170)
                at weblogic.management.deploy.internal.ConfiguredDeployments.deploy(ConfiguredDeployments.java:124)
                at weblogic.management.deploy.internal.DeploymentServerService.resume(DeploymentServerService.java:181)
                at weblogic.management.deploy.internal.DeploymentServerService.start(DeploymentServerService.java:97)
                at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
                at weblogic.work.ExecuteThread.execute(ExecuteThread.java:256)
                at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused By: java.lang.NoClassDefFoundError: weblogic/sca/api/ScaReferenceProcessor


Was  completely stuck with the problem, looking at error message I thought something might have wrong with the binaries what I have download.

I have discussed this issue with Oracle guys to confirm the reason of failure and the solution which came out after discussion was quite surprised for me but its worked and I was able to start the ‘soa-infa’ component after that.


Solution:

Oracle suggested that if you do Weblogic 10.3.6 installation using silent mode there are few jars files get missed during installation, this is known bug and Oracle support is working on that. In my case I have done the installation using silent mode only.

So to get those jar files the only way you need to run the Weblogic installation using ‘Graphical’ mode and get those jar, copy it to your previous installation location and restart the server. I did all these steps, and I was wonder issue got resolved after that. So missing jar is listed here –

Location: $MiddlewareHome/modules folder

com.oracle.weblogic.sca.engine_1.2.0.0.jar
com.oracle.ws.http_client_1.3.0.0.jar
com.oracle.ws.orawsdl_1.3.0.0.jar
ws.databinding.plugins_1.2.0.0.jar
ws.databinding_1.2.0.0.jar

Location: $MiddlewareHome/modules/features folder

weblogic.server.modules.sca_10.3.4.0.jar
weblogic.server.modules.sca_10.3.4.0.xml

Just get all above jar files from the weblogic installation which is done through graphical installation and copy it to your previous installation and restart the soa server.

Note: Don’t much focus on the version of jar, since this issue I believe is with old Weblogic version as well e.g. 10.0.3.3, 10.0.3.4 etc.

Regards..
Manish