Nimbus and Supervisor is on the same machine. I'm running my jar on storm with 'bin/storm XX.jar XX.XX' and stdout shows that this topology is running well.
But while I use 'bin/storm list', no infomation of my topology is show.
Related
I run Apache Storm in a cluster and I was looking for ways to stop and/or restart Nimbus, Supervisor and UI. Would writting a servise help? What should I write in this service file and where should I place it? Thank you in advance
Yes, writing a service is the recommended way to run Storm. The commands you want to run are storm nimbus to start Nimbus (minimum 1 per cluster), storm supervisor to run the supervisor (1 per worker machine), storm ui (1 per cluster) and storm logviewer (1 per worker machine). There are other commands you can also run, but you can find these by simply running storm, it will print a list.
Regarding how to write the service, take a look at the upstart cookbook http://upstart.ubuntu.com/cookbook/.
There's an example script here you can probably use to get started https://unix.stackexchange.com/a/84289
you can make them as service and start them up as the node starts and same can be used to stop them.
/etc/rc.d/SERVICE start or stop or restart
We can easily stop them using the command "ps -aux | grep nimbus" or supervisor etc. Then we have to find the process id and kill it with the “kill” command.
I have a hadoop cluster 2.7.4 version. Due to some reason, I have to restart my cluster. I need job IDs of those jobs that were executed on cluster before cluster reboot. Command mapred -list provide currently running of waiting jobs details only
You can see a list of all jobs on the Yarn Resource Manager Web UI.
In your browser go to http://ResourceManagerIPAdress:8088/
This is how the history looks on the Yarn cluster I am currently testing on (and I restarted the services several times):
See more info here
I'm new to Apache Storm and I'm currently working on a project which uses storm. While trying to understand the basics of storm, I came across nimbus and supervisors. I started setting up a remote cluster. I edited the storm.yaml file and setup the nimbus and zookeeper to localhost. I tried running my nimbus, zookeeper in my local machine. I started nimbus using "storm nimbus", but nimbus is not started whereas my zookeeper is actively running. enter image description hereI have the screenshot of what I get.
Can somebody help me in sorting out.
go to the directory where you have installed your storm go to bin folder and run:
storm nimbus
reference image:
You have to run the start command from the bin directory of the storm installation directory.
Navigate to the bin folder inside storm installation folder and then run "storm nimbus".
The storm release is 1.0.1. My nimbus and supervisor are running. When I submit a ExclamationTopology that in storm-starter, the worker is not starting and there isn't a worker.log.
I find the error here from supervisor.log.
Thanks a lot for your suggestion.
I'm able to submit a topology job in the multi-tenant cluster. The job is running. However, the logviewer page is not available. Is there any way to solve this issue?
you need to start the logviewer before you click on topology port to see logviewer.
To start logviewer run:
$ storm logviewer same as you run $ storm list
I faced the same issue for logviewer's home page, but directly navigating to a particular log file that exists in the logs folder works. Try this:
MachineIP:8000/log?file=worker-6700.log