Cloudera Manager failed to start Activity Monitor - hadoop

I downloaded CDH 4.5 quick start vm from here. Each service looks good except for below error was seen after I opened Activities tab to view mapreduce activities:
The Activity Monitor server (activitymonitor (localhost)) is unavailable or not responding to connections.
The problem remains after I tried to restart Activity Monitor service, then I found following error message in the log. Can anybody help take a look?
11:24:35.862 PM WARN org.mortbay.log
failed SelectChannelConnector#localhost.localdomain:9999: java.net.BindException: Address already in use
11:24:35.864 PM WARN org.mortbay.log
failed Server#59cc2f42: java.net.BindException: Address already in use
11:24:35.869 PM ERROR com.cloudera.cmon.firehose.Main
Failed to start Firehose
com.cloudera.enterprise.EnterpriseServiceException: java.net.BindException: Address already in use
at com.cloudera.cmon.firehose.AgentMessageService.startService(AgentMessageService.java:144)
at com.cloudera.enterprise.EnterpriseService.start(EnterpriseService.java:71)
at com.cloudera.enterprise.EnterpriseService.start(EnterpriseService.java:68)
at com.cloudera.cmon.firehose.Main.main(Main.java:371)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:126)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
at org.mortbay.jetty.nio.SelectChannelConnector.open(SelectChannelConnector.java:216)
at org.mortbay.jetty.nio.SelectChannelConnector.doStart(SelectChannelConnector.java:315)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at org.mortbay.jetty.Server.doStart(Server.java:235)
at org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
at com.cloudera.cmon.firehose.AgentMessageService.startService(AgentMessageService.java:142)
... 3 more

"java.net.BindException: Address already in use" clearly states that port 9999 is already occupied by some other service. You have to check the PID of the service and stop it:
lsof -P | grep LISTEN | grep 9999
use the PID to kill or if you know the service then stop gracefully

Related

Camunda Optimize cannot be started. Port already in used

As I'm trying to exploit the Camunda Enterprise version, now I'm stuck at the Optimize server. I'm not able to start the Optimize server followed by instruction. Here is the log file
16:59:37.322 [main] DEBUG o.e.j.u.component.AbstractLifeCycle - starting ServerConnector#48974e45{HTTP/1.1,[http/1.1]}{0.0.0.0:8095}
16:59:37.325 [main] WARN o.e.j.u.component.AbstractLifeCycle - FAILED ServerConnector#48974e45{HTTP/1.1,[http/1.1]}{0.0.0.0:8095}: java.io.IOException: Failed to bind to /0.0.0.0:8095
java.io.IOException: Failed to bind to /0.0.0.0:8095
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:307)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:231)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
at org.eclipse.jetty.server.Server.doStart(Server.java:385)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:72)
at org.camunda.optimize.jetty.EmbeddedCamundaOptimize.startOptimize(EmbeddedCamundaOptimize.java:169)
at org.camunda.optimize.Main.main(Main.java:17)
Caused by: java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
... 8 common frames omitted
I know it is obvious but I tried to changing different port. It's still showing the same error, no matter which port I choose. Please help to take a look.
The port is already in use, the message is obvious. All you can do is to kill the process that is listening to this thread.
I use Windows, but you can search for Linux way to do, the principle is the same.
Windows:
Find a process that uses the port 8095 using netstat -aon | find "8095". The result looks like:
TCP 0.0.0.0:8003 0.0.0.0:0 LISTENING 23332
TCP [::]:8003 [::]:0 LISTENING 23332
Find in the Task Manager in the Details tab what runs under the PID (process ID) number 23332.
End such a task. It will be probably java.exe since there is a pending process listening to the port. Restart the IDE and everything should work well. If it is ex. a database or anything else, you have to use a different port than 8095 either at your side or the application that already uses it.
Linux: https://unix.stackexchange.com/questions/140482/kill-any-service-running-at-a-specific-port

failed to launch apache.spark.master

Whenever i run start-master.sh command on my local machine i am getting following error please someone help me to fix this issue
Terminal Error
Error which i get in terminal
starting org.apache.spark.deploy.master.Master, logging to /usr/local/spark-2.0.1-bin-hadoop2.6/logs/spark-andani-org.apache.spark.deploy.master.Master-1-andani.sakha.com.out
failed to launch org.apache.spark.deploy.master.Master:
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:748)
Log Error
If i check the spark log file following is the error
Exception in thread "main" java.net.BindException: Cannot assign requested address: Service 'sparkMaster' failed after 16 retries! Consider explicitly setting the appropriate port for the service 'sparkMaster' (for example spark.ui.port for SparkUI) to an available port or increasing spark.port.maxRetries.
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at io.netty.channel.socket.nio.NioServerSocketChannel.doBind(NioServerSocketChannel.java:125)
at io.netty.channel.AbstractChannel$AbstractUnsafe.bind(AbstractChannel.java:485)
at io.netty.channel.DefaultChannelPipeline$HeadContext.bind(DefaultChannelPipeline.java:1089)
at io.netty.channel.AbstractChannelHandlerContext.invokeBind(AbstractChannelHandlerContext.java:430)
at io.netty.channel.AbstractChannelHandlerContext.bind(AbstractChannelHandlerContext.java:415)
at io.netty.channel.DefaultChannelPipeline.bind(DefaultChannelPipeline.java:903)
at io.netty.channel.AbstractChannel.bind(AbstractChannel.java:198)
at io.netty.bootstrap.AbstractBootstrap$2.run(AbstractBootstrap.java:348)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at java.lang.Thread.run(Thread.java:748)
The error is due to your sparkMaster was not able to contact to your internal IP
Once check your /etc/hosts file weather it pointing to proper host name or your previous IP address might have changed.
Reconfigure it and run the command once again.

How install alluxio1.2 on openstack

I try to install alluxio1.2 on a VM centos on openstack with spark and hdfs but the installation doesn't works. Spark and hdfs are already install and work
ERROR logger.type (AlluxioMaster.java:main) - Uncaught exception while running Alluxio master, stopping it and exiting.
java.lang.RuntimeException: java.net.BindException: Address already in use
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at alluxio.web.UIWebServer.startWebServer(UIWebServer.java:164)
at alluxio.master.AlluxioMaster.startServingWebServer(AlluxioMaster.java:467)
at alluxio.master.AlluxioMaster.startServing(AlluxioMaster.java:452)
at alluxio.master.AlluxioMaster.startServing(AlluxioMaster.java:447)
at alluxio.master.AlluxioMaster.start(AlluxioMaster.java:389)
at alluxio.master.AlluxioMaster.main(AlluxioMaster.java:86)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.nio.SelectChannelConnector.open(SelectChannelConnector.java:187)
at alluxio.web.UIWebServer.startWebServer(UIWebServer.java:154)
... 5 more
Are there a special installation to install alluxio on one openstack machine ?
It looks like the Alluxio master's web UI cannot start because the address is already in use. This happens if the port is taken by another process. Alluxio web UI uses the port 19999 for the web UI by default. If you expect another process to be using that port, you can change the Alluxio master web UI port by changing the configuration parameter (http://www.alluxio.org/docs/master/en/Configuration-Settings.html#master-configuration), alluxio.master.web.port, to another port number.

MongoDB, SparkJava using Maven Connection Error

Error Log :
Listening on 0.0.0.0:4567
2014-11-27 00:37:47,570 WARN - log - FAILED SocketConnector#0.0.0.0:4567: java.net.BindException: Address already in use: JVM_Bind
2014-11-27 00:37:47,572 WARN - log - FAILED org.eclipse.jetty.server.Server#5595a5a9: java.net.BindException: Address already in use: JVM_Bind
you didn't terminate spark properly and it's still running and bound to the default port 4567.
Just close the previous process and restart. If you can't find it or it's running in the background you can use NETSTAT to find the process and kill it.
on windows netstat -b to find the process

Unable to start node manager on hadoop 0.23.0

while installing Hadoop 0.23.0 on the cluster, node manager is unable to start and getting the following error.
Caused by: org.jboss.netty.channel.ChannelException: Failed to bind to: 0.0.0.0/0.0.0.0:8080
at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:303)
at org.apache.hadoop.mapred.ShuffleHandler.start(ShuffleHandler.java:255)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices.start(AuxServices.java:123)
at org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
... 4 more
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:126)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
It means 8080 is already in use.
so
sudo netstat -nvvpa |grep 8080 and see the o/p.
if it listens to any java, then if possible stop the process. and then try to start the nodemanager again.
This made my problem solved. thankyou.

Resources