TCP Channel initialization failed error when starting Websphere Application Server - websphere

I'm encountering an error when I tried to run my websphere application server.
TCP Channel TCPInboundChannel_ipcc.Default_IPC_Connector_Name initialization failed. The socket bind failed for host localhost and port 9633. The port may already be in use.
Below is the stack trace of the error:
[9/13/10 15:35:12:942 SGT] 00000000 JMXSoapAdapte A ADMC0013I: The SOAP connector is available at port 8880
[9/13/10 15:35:12:967 SGT] 00000000 TCPPort E TCPC0003E: TCP Channel TCPInboundChannel_ipcc.Default_IPC_Connector_Name initialization failed. The socket bind failed for host localhost and port 9633. The port may already be in use.
[9/13/10 15:35:12:992 SGT] 00000000 IPCConnectorS E ADMC0072E: Failed to initialize or start channel chain IPCCInboundChain.Default_IPC_Connector_Name/LC_IPCCInboundChain.Default_IPC_Connector_Name due to exception com.ibm.wsspi.channel.framework.exception.ChainGroupException:
Error initializing chain IPCCInboundChain.Default_IPC_Connector_Name in group IPCCInboundGroup.Default_IPC_Connector_Name, exception=com.ibm.wsspi.channel.framework.exception.RetryableChannelException: Cannot assign requested address: bind
at com.ibm.ws.channel.framework.impl.ChannelFrameworkImpl.initChainGroup(ChannelFrameworkImpl.java:3667)
at com.ibm.ws.runtime.component.ChannelFrameworkServiceImpl.initChainGroup(ChannelFrameworkServiceImpl.java:772)
at com.ibm.ws.management.connector.ipc.IPCConnectorServer.start(IPCConnectorServer.java:330)
at com.ibm.ws.management.component.JMXConnectors.startIPCConnector(JMXConnectors.java:970)
at com.ibm.ws.management.component.JMXConnectors.stateChanged(JMXConnectors.java:2211)
at com.ibm.ws.security.core.distSecurityComponentImpl.fireEvent(distSecurityComponentImpl.java:2181)
at com.ibm.ws.security.core.distSecurityComponentImpl.fireStartedEvent(distSecurityComponentImpl.java:2143)
at com.ibm.ws.security.core.distSecurityComponentImpl.startSecurity(distSecurityComponentImpl.java:642)
at com.ibm.ws.security.core.SecurityComponentImpl.startSecurity(SecurityComponentImpl.java:109)
at com.ibm.ws.security.core.ServerSecurityComponentImpl.start(ServerSecurityComponentImpl.java:338)
at com.ibm.ws.runtime.component.ContainerHelper.startComponents(ContainerHelper.java:538)
at com.ibm.ws.runtime.component.ContainerImpl.startComponents(ContainerImpl.java:627)
at com.ibm.ws.runtime.component.ContainerImpl.start(ContainerImpl.java:618)
at com.ibm.ws.runtime.component.ApplicationServerImpl.start(ApplicationServerImpl.java:227)
at com.ibm.ws.runtime.component.ContainerHelper.startComponents(ContainerHelper.java:538)
at com.ibm.ws.runtime.component.ContainerImpl.startComponents(ContainerImpl.java:627)
at com.ibm.ws.runtime.component.ContainerImpl.start(ContainerImpl.java:618)
at com.ibm.ws.runtime.component.ServerImpl.start(ServerImpl.java:503)
at com.ibm.ws.runtime.WsServerImpl.bootServerContainer(WsServerImpl.java:298)
at com.ibm.ws.runtime.WsServerImpl.start(WsServerImpl.java:214)
at com.ibm.ws.runtime.WsServerImpl.main(WsServerImpl.java:666)
at com.ibm.ws.runtime.WsServer.main(WsServer.java:59)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:600)
at com.ibm.wsspi.bootstrap.WSLauncher.launchMain(WSLauncher.java:213)
at com.ibm.wsspi.bootstrap.WSLauncher.main(WSLauncher.java:93)
at com.ibm.wsspi.bootstrap.WSLauncher.run(WSLauncher.java:74)
at org.eclipse.core.internal.runtime.PlatformActivator$1.run(PlatformActivator.java:78)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:92)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:68)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:400)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:48)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
at java.lang.reflect.Method.invoke(Method.java:600)
at org.eclipse.core.launcher.Main.invokeFramework(Main.java:340)
at org.eclipse.core.launcher.Main.basicRun(Main.java:282)
at org.eclipse.core.launcher.Main.run(Main.java:981)
at com.ibm.wsspi.bootstrap.WSPreLauncher.launchEclipse(WSPreLauncher.java:340)
at com.ibm.wsspi.bootstrap.WSPreLauncher.main(WSPreLauncher.java:110)
It logs "Server server1 open for e-business" at the end but then I cannot access my application and even the admin console. I checked port 9633 if it is in use through
netstat but it isn't. I also tried to change the port to 21633 but it still gives me the same error. The weird thing about this is I have no problems in running my server (and running my application on that server) last week but when I tried to do so yesterday, this error suddenly shows. I already restarted my WAS and even shut down my pc several times but to no avail. Websphere is not running as a Windows service and I'm trying to run it from eclipse. However, even if I tried running it as a service, I still get the same error. I haven't changed any configuration at all so I'm wondering what might be causing this.
My environment: Windows 7, Websphere Application Server 7.0.0.9
Anyone familiar with this problem? Any help is appreciated! Thanks!

Make sure your host name on the IPC_CONNECTOR_ADDRESS port definition for the server says localhost and not the server name.

I received this error, repeatedly, for (seemingly) every port that WAS binds, even though nobody was listening on them prior to WAS startup.
This thread led me to a solution: apparently there was some issue around the IPv6 stack. Since I don't care about IPv6 for my use, I was able to tell the JVM to prefer IPv4 by setting the following in the setupCmdLine.{bat,sh} script in my profile's bin directory.
set IBM_JAVA_OPTIONS=-Djava.net.preferIPv4Stack=true
(Change syntax as appropriate for your environment, of course.)
This was on Windows 8/WAS 8.5 (developer's edition).

You mention that you cannot access your application and the admin console. The admin console is typically bound to WC_adminhost and WC_adminhost_secured ports. You can find those under the WAS_HOME/profiles/profile_name/properties/portdef.props file. If you've customized any ports, make sure that you haven't created a conflict between any of them.
Is there any reason that you cannot access your application through any of the other standard ports? IPC is only important for high-speed communication between local instances. Most web application access can be done through the WC_defaulthost and WC_defaulthost_secure ports.

This is due to incorrect IP resolution for "localhost". Please ping localhost and ensure the response is from 127.0.0.1 (loopback) or the actual host IP itself. If not, fix it and it will resolve your issue

I've got the same issue using websphere v6.1 on Windows 7, using RAD 8.0.4
[18/09/13 09:15:01:036 EST] 00000022 TCPPort E TCPC0003E: TCP Channel TCP_1 initialization failed. The socket bind failed for host * and port 9060. The port may already be in use.
[18/09/13 09:15:02:038 EST] 00000020 TCPPort E TCPC0003E: TCP Channel TCP_2 initialization failed. The socket bind failed for host * and port 80. The port may already be in use.
[18/09/13 09:15:03:111 EST] 00000022 TCPPort E TCPC0003E: TCP Channel TCP_1 initialization failed. The socket bind failed for host * and port 9060. The port may already be in use.
[18/09/13 09:15:04:112 EST] 00000020 TCPPort E TCPC0003E: TCP Channel TCP_2 initialization failed. The socket bind failed for host * and port 80. The port may already be in use.
Here's what I did:
Verified no ports are listening on 80 or 9060 by opening a command terminal and running 'netstat -a'. If that's the case, terminate the application using it, or if you can't, try doing a restart. Otherwise, proceed to step 2.
Follow the instructions here: "http://www-01.ibm.com/support/docview.wss?uid=swg21628479".
Just a note with this is the location of "setupCmdLine.bat" file. The way my websphere is installed, it has multiple 'bin/setupCmdLine.bat' files. Folders containing this file includes, "IBM/SDP/runtimes/base_v61/bin"; "IBM/SDP/runtimes/base_v61/profiles/was61profile1"; "IBM/WebSphere/AppServer/bin"; "IBM/WebSphere/AppServer/profiles/AppSrv1/bin". And this made it tonnes confusing, because if you apply the solution to the wrong file, it won't take affect.
To find the right folder to update "setupCmdLine.bat", I went into RAD -> Servers Tab -> Right clicked on the Websphere server -> Properties -> WebSphere Application Server v6.1 menu item. And on the right side, there's the "Profile directory".
For me this pointed to '.../IBM/SDP/runtimes/base_v61/profiles/was61profile1'

Related

Jmeter RTE Plugin Connection timed out: connect

I'm trying to use Jmeter RTE Plugin to make mainframe tests.
Using a pconn or wc3270, it`s connects good but trying to record or using an existent JMX file, i'm getting a connection exception.
There is any other configuration that I need to do?
com.blazemeter.jmeter.rte.core.exceptions.RteIOException: Communication error wit server: 192.168.2.2
at com.blazemeter.jmeter.rte.core.listener.ExceptionHandler.throwAnyPendingError(ExceptionHandler.java:41)
at com.blazemeter.jmeter.rte.protocols.tn3270.Tn3270Client.connect(Tn3270Client.java:159)
at com.blazemeter.jmeter.rte.sampler.RTESampler.buildClient(RTESampler.java:617)
at com.blazemeter.jmeter.rte.sampler.RTESampler.sample(RTESampler.java:516)
at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:638)
at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:558)
at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:489)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:256)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.ConnectException: Connection timed out: connect
at java.base/java.net.PlainSocketImpl.waitForConnect(Native Method)
at java.base/java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:107)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:412)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:255)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:237)
at java.base/java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.base/java.net.Socket.connect(Socket.java:608)
at com.bytezone.dm3270.streams.TerminalServer.run(TerminalServer.java:48)
... 1 more
I already tried; reinstall jmeter, java, get a jmeter working installation and script but nothing fixed that issue.
Configs of wc3270 and rte plugin
Unfortunately we're not able to help without seeing your wc3270 and RTE Config (or RTE Recorder) configuration details.
The exception means that JMeter cannot connect to the host 192.168.2.2 on the given port so I would recommend double checking that you can reach that host using telnet or equivalent.
You can also enable debug logging and see whether jmeter.log file contains any clues.
And finally you can check out Testing Mainframe Environments Using Open Source Tools and cross check it with your setup, maybe you're missing something obvious.
Coming back on this thread to share the solution and add an information.
My jmeter test it`s connecting through VPN connection and the problem it was Java trying to use IPv4 over IPv6 for IPv4 connections.
resolution: setx _JAVA_OPTIONS -Djava.net.preferIPv4Stack=true

Jenkins - slave mac - unable to connect

I have a Jenkins build server (v2.77) installed on a windows server 2016 and I'd like to add a mac as a slave node. I set the node to run via "Java Web Start". On the mac side I downloaded the slave-agent.jnlp file from jenkins, but it doesn't connects to the server.
This is the exception:
java.io.IOException: Failed to connect to 37.210.56.16:49693
at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:243)
at hudson.remoting.Engine.connect(Engine.java:660)
at hudson.remoting.Engine.innerRun(Engine.java:524)
at hudson.remoting.Engine.run(Engine.java:447)
Caused by: java.net.ConnectException: Operation timed out
at java.base/sun.nio.ch.Net.connect0(Native Method)
...
I tried to turn off the windows firewall, to change the TCP port for JNLP agents option in jenkins to fixed/random, but they didn't connected.
Any ideas?
Thanks for help
Finally I found the solution. I rent the server from the google, and in the google cloud console I had to define for this VM instance a firewall rule to accept a specific port number for this JNLP agent connection. Now it's working well.

Mulesoft - SFTP Component to go through SOCKS Proxy

I was hoping someone could help.
I am trying to sftp to a remote server that is behind a firewall. I have a proxy setup and can sftp via the unix terminal succesfully. However when using mulesofts SFTP connector I get a timeout. From research it looks like this is because it is run on the JVM.
Mule comes with wrapper.conf to update any jvm settings so I have set the following settings:
wrapper.java.additional.15=-DsocksProxyHost:hostname
wrapper.java.additional.16=-DsocksProxyPort=1080
When I do this the Mule service no longer runs and get the following error:
WrapperManager Error: Unexpected exception opening backend socket: java.net.SocketException: Can't connect to SOCKS proxy:Invalid argument or cannot assign requested address
Hopefully someone can assist.
Thank you.
Perhaps typo in the
-DsocksProxyHost:hostname
Should be = rather than :.
The port is unnecessary, 1080 is the default. Documentation is here, see section 2.4 for SOCKS
http://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html

No command response from neo4j-shell, despite establishing connection

Created new EC2 instance of neo4j via CloudFormation template found here (ubuntu host).
https://github.com/neo4j-contrib/ec2neo
Got the web interface to work fine, and DB is up and running.
Trying to connect with neo4j-shell from my local dev machine, and I am able to establish a connection to the remote EC2 server.
$ neo4j-shell -host ec2-xx-xx-xx-xx.compute-1.amazonaws.com
Welcome to the Neo4j Shell! Enter 'help' for a list of commands
NOTE: Remote Neo4j graph database service 'shell' at port 1337
neo4j-sh (?)$
netstat confirms that a connection has been ESTABLISHED
tcp6 0 0 xx.xx.xx.xx:1337 my.local.ip.add:13785 ESTABLISHED
At this point, I type help, or any neo4j command, and I get no response back from the server. The console just hangs. As soon as I stop the neo4j service on the server, I get the following exception on the client console.
java.rmi.UnmarshalException: Error unmarshaling return header; nested exception is:
java.io.EOFException
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:229)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:162)
at java.rmi.server.RemoteObjectInvocationHandler.invokeRemoteMethod(RemoteObjectInvocationHandler.java:194)
at java.rmi.server.RemoteObjectInvocationHandler.invoke(RemoteObjectInvocationHandler.java:148)
at com.sun.proxy.$Proxy1.interpretLine(Unknown Source)
at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:149)
at org.neo4j.shell.impl.AbstractClient.evaluate(AbstractClient.java:133)
at org.neo4j.shell.impl.AbstractClient.grabPrompt(AbstractClient.java:101)
at org.neo4j.shell.StartClient.grabPromptOrJustExecuteCommand(StartClient.java:383)
at org.neo4j.shell.StartClient.startRemote(StartClient.java:330)
at org.neo4j.shell.StartClient.start(StartClient.java:196)
at org.neo4j.shell.StartClient.main(StartClient.java:135)
Caused by: java.io.EOFException
at java.io.DataInputStream.readByte(DataInputStream.java:267)
at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:215)
... 11 more
I have made the following change to neo4j-wrapper.conf :
wrapper.java.additional=-Djava.rmi.server.hostname=ec2-xx-xx-xx-xx.compute-1.amazonaws.com
All iptables are "disabled", to eliminate variables. I am able to run neo4j-shell on the server itself, to 127.0.0.1
What am I missing in my network config or neo4j server config?
Try to ssh into the instance and run it there. remote connections have been a pain for a long time because of the underlying Java RMI port handling.
You can also try out cycli which supports http and auth.

distributed load testing on aws with jmeter

I have been trying to setup aws ec2 machines for load testing of my web server using jmeter but I am stuck. I have a jmeter client on my local machine and I want to set up multiple jmeter-server nodes on ec2 to do the load testing and I am, thus far, just trying to get one server node up and running. But it hasn't worked out for me yet.
I have the same jmeter running on my local machine and the server and the java version was a little different but I don't think that is the problem. Most of the people have had problems with getting the correct ip for connecting between the client and the server nodes but I, after a lot of searching, have gotten through all those problems. I am stuck at when the server node attempts to return the result and tries to connect to the client, my local machine. The server tries to connect to the external ip address of my local machine. But it throws a connection refused error, which apparently was caused by connection timeout. I guess it's some firewall issue but I tried turning off the firewall on my local machine but it still throws the same error. I am not sure how can I get past this and it's taking way too much time then it should.
Could somebody please suggest me something to solve this? Thanks!
My local machine is a Mac OS X 10.7.5 and my server nodes are on ubuntu.
This is the error that it throws:
2013/01/29 12:23:37 ERROR - jmeter.samplers.RemoteListenerWrapper: testStarted(host) java.rmi.ConnectException: Connection refused to host: xxx.xxx.xxx.10; nested exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:619)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:216)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:128)
at org.apache.jmeter.samplers.RemoteSampleListenerImpl_Stub.testStarted(Unknown Source)
at org.apache.jmeter.samplers.RemoteListenerWrapper.testStarted(RemoteListenerWrapper.java:83)
at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfStart(StandardJMeterEngine.java:226)
at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:349)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
at java.net.Socket.connect(Socket.java:546)
at java.net.Socket.connect(Socket.java:495)
at java.net.Socket.<init>(Socket.java:392)
at java.net.Socket.<init>(Socket.java:206)
at sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:40)
at sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:146)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:613)
... 8 more
Well, I finally solved the problem. I ended up using ssh reverse tunnels. I am not sure if there is a better way to do this though. So, in case anyone has a similar problem, this is how I did it:
Create a reverse ssh tunnel from the server to the client. So, at client side:
ssh -Nf -R [client.rmi.localport]:localhost:[client.rmi.localport on serverside] user#server
start server and have a client.rmi.localport as well; the port at which the tunnel was created;
start the client as: ./bin/jmeter-server -Djava.rmi.server.hostname=127.0.0.1.
And that's it! You have your distributed testing ready.
Solution that worked for me on Linux/OSX:
1.On the client edit bin/jmeter.properties and add:
remote_hosts=127.0.0.1:55501
client.rmi.localport=55512
mode=Batch
num_sample_threshold=250
2.On the server edit bin/jmeter.properties and add:
server_port=55501
server.rmi.localhostname=127.0.0.1
server.rmi.localport=55511
3.Now connect to the server using this ssh tunel:
ssh -L 55501:127.0.0.1:55501 -L 55511:127.0.0.1:55511 -R 55512:127.0.0.1:55512 user#hostname
4.Edit jmeter-server script to start jmeter.sh
${DIRNAME}/jmeter.sh ${RMI_HOST_DEF} -Dserver_port=${SERVER_PORT:-1099} -s -j jmeter-server.log "$#"
5.Now run on the server:
bin/jmeter-server -Djava.rmi.server.hostname=127.0.0.1
6.And on the client run jmeter with gui or add -n if gui is not needed:
bin/jmeter.sh -Djava.rmi.server.hostname=127.0.0.1
or, with test plan:
bin/jmeter.sh -Djava.rmi.server.hostname=127.0.0.1 -t /path/to/test-plan.jmx
Looks like you have to move your jmeter-master instance (jmeter client) to EC2 instance too.
As per JMeter Distributed Testing Step-by-step:
2. check all the clients are on the same subnet;
For distributed testing to work, the systems must be on the same subnet, otherwise RMI will not be able to connect.
Looks like to be your case: jmeter-slaves are in one subnet (EC2) and jmeter-master in another (your local workstation).
I wrote a free, open source script to help do exactly this. I went through the same issues listed by the OP and, even though I did get things working in the end, it was never great and I wanted something to automate away the hassle.

Resources