I have zookeeper servers, and I'm trying to install storm using those zk servers.
My storm.yaml file looks like:
storm.zookeeper.servers:
- "ZKSERVER-0"
- "ZKSERVER-1"
- "ZKSERVER-2"
storm.local.dir: "/opt/apache-storm-2.2.0/data"
nimbus.host: "localhost"
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
I tested ping with those ZKSERVERs, and it worked fine.
However, when I start nimbus with ./storm nimbus command, it doesn't show any error, but it doesn't end either.
root#69e55d266f5a:/opt/apache-storm-2.2.0/bin:> ./storm nimbus
Running: /usr/jdk64/jdk1.8.0_112/bin/java -server -Ddaemon.name=nimbus -Dstorm.options= -Dstorm.home=/opt/apache-storm-2.2.0 -Dstorm.log.dir=/opt/apache-storm-2.2.0/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib:/usr/lib64 -Dstorm.conf.file= -cp /opt/apache-storm-2.2.0/*:/opt/apache-storm-2.2.0/lib/*:/opt/apache-storm-2.2.0/extlib/*:/opt/apache-storm-2.2.0/extlib-daemon/*:/opt/apache-storm-2.2.0/conf -Xmx1024m -Djava.deserialization.disabled=true -Dlogfile.name=nimbus.log -Dlog4j.configurationFile=/opt/apache-storm-2.2.0/log4j2/cluster.xml org.apache.storm.daemon.nimbus.Nimbus
The terminal just shows the above logs, and that doesn't change until I run control+C.
What could be a problem here?
Can you share the log of the nimbus?
Generally, the nimbus will be in a running state until you stop it, or it faces an error. If you want to be sure about your nimbus status, you can check the log of your nimbus (./logs/nimbus.log).
on running ./storm nimbus command. The process has started as it is showing in your example. This is the usual behavior.
If you want to run the storm in the background, try to run it with the nohup command
nohup ./storm nimbus > storms.log &
Related
I am getting the below message.
Could not find leader nimbus from seed hosts [master]. Did you specify
a valid list of nimbus hosts for config nimbus.seeds?
Delete storm under zookeeper.try to use hostname or ip in nimbus.seeds
storm.zookeeper.servers :
- "master"
- "salve1"
- "salve2"
storm.zookeeper.port : 2181
nimbus.seeds : ["master"]
nimbus.thrift.port : 6690
storm.local.dir : "/root/storm"
supervisor.slots.ports :
- 6700
- 6701
- 6702
- 6703
Why this can happen?
Make sure Zookeeper is running, and is accessible on the machine you're running the command from. You can check this with curl. On the machine you're starting your storm command from, try running curl master:2181. You should get an empty reply.
e.g.
$ curl localhost:2181
curl: (52) Empty reply from server
Do the same for the two other hosts you run Zookeeper on.
Then make sure curl master:6690 also returns an empty reply, since that is the Thrift port you've configured.
If you are getting connection refused on either command, you need to fix your network setup, so the machines can talk to each other.
I started storm nimbus and supervisor with following command,
sudo storm nimbus &
sudo storm supervisor &
Is there a way to stop the cluster without using kill -9 processes
Strange issue:
The zookeeper works normally on my cluster if I start it using ./zkServer.sh on each machine, respectively.
However, when I try to start it remotely from the master node:
ssh 192.168.xxx.xxx "/opt/apache/zookeeper-3.4.5/bin/zkServer.sh start"
it looks fine:
JMX enabled by default
Using config: /opt/apache/zookeeper-3.4.5/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
But actually, zookeeper is not running on that machine, which can be confirmed by jps.
The most strange thing is:
If I directly start zookeeper on that node using ./zkServer.sh start then I can successfully stop it remotely from the master node using
ssh 192.168.xxx.xxx "/opt/apache/zookeeper-3.4.5/bin/zkServer.sh stop"
Why could this happen? Any help would be appreciated.
Though after a lot of search I was able to get it, below is the command that successfully starts it.
ssh -i "somekey.pem" username#hostname 'bash -i -c "~/zookeeper-3.4.6/bin/zkServer.sh start"'
I'm getting the following error trying so start storm nimbus for a local dev Windows 7 workstation:
Error: Could not find or load main class Files\Java\jdk1.8.0_92\bin;C:\Program
storm.yaml:
storm.zookeeper.servers:
- "127.0.0.1"
nimbus.seeds: ["127.0.0.1"]
storm.local.dir: "C:\\Users\\userX\\Apps\\ApacheStorm\\apache-storm-1.0.1\\data"
supervisor.slots.ports:
- 6700
- 6701
- 6702
- 6703
I successfully started ZooKeeper. But when I try to run: storm nimbus from the Storm bin folder I get the error.
Any ideas?
Use PROGRA~1 instead of Program files in System variables' path.
For example:
C:\Program Files\Java\jdk1.6.0_45 ==> C:\PROGRA~1\Java\jdk1.6.0_45
I set up Apache Storm 0.9.3 in fully distributed mode (3 nodes) that leverages a fully distributed Apache Zookeeper cluster (3.4.6) consisting of 3 nodes. I did the following testing and found out that Storm Nimbus failed to start if the first zookeeper server in the configuration storm.yaml is down or temporary unreachable.
Test #1:
bring up all three zookeeper nodes
start nimbus, supervisor, ui on the storm master node, and launch supervisor on other two nodes
in this case, everything goes well
Test #2:
shut down one of the three zookeeper nodes (zookeeper is still functional)
start nimbus, supervisor, ui on the storm master node, and launch supervisor on other two nodes
in this case, if the failed zookeeper node happened to be the first one in storm.zookeeper.servers, if fails to start neither nimbus nor supervisor on the master node.
I am wondering if any of you guys encounter this problem? Is it something going unexpected or something going wrong with my configuration? Or something else?
My configuration is listed below
storm.zookeeper.servers :
- "zookeeper1.hostname.local"
- "zookeeper2.hostname.local"
- "zookeeper3.hostname.local"
nimbus.host : storm-master.hostname.local
nimbus.thrift.port : 6627
storm.zookeeper.port : 2181
supervisor.slots.ports :
- 6700
- 6701
- 6702
- 6703
ui.port : 8744
storm.local.dir : /opt/apache-storm-0.9.3/storm-local