EDIT: I was able to get this to work. I created a tutorial to show how:
http://www.dreamsyssoft.com/blog/blog.php?/archives/5-How-to-use-HBase-Hadoop-Clustered.html
I can get HBase working just fine when I set the hbase-site.xml property:
<name>hbase.rootdir</name>
<value>file:///app/hbase/hbase/</value>
This works just fine, it stores the data in the directory as expected, however I want it to connect to my running hadoop instance now instead of using a local file. I set it to
<value>hdfs://localhost:9000/</value>
Instead of the local file and it will not work. Is there some extra configuration I need to do on the hadoop side to support this? Hadoop is running and using port 9000.
Here's my core-site.xml from hadoop:
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
Also here's the error when trying to do a "list" in hbase shell:
hbase(main):001:0> list
TABLE
12/06/28 15:26:29 ERROR zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries
12/06/28 15:26:29 WARN zookeeper.ZKUtil: hconnection Unable to set watcher on znode /hbase/master
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/master
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1021)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:154)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.watchAndCheckExists(ZKUtil.java:226)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperNodeTracker.start(ZooKeeperNodeTracker.java:76)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:580)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:569)
at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:186)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:98)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.jruby.javasupport.JavaConstructor.newInstanceDirect(JavaConstructor.java:275)
at org.jruby.java.invokers.ConstructorInvoker.call(ConstructorInvoker.java:91)
at org.jruby.java.invokers.ConstructorInvoker.call(ConstructorInvoker.java:178)
I believe Hbase cannot find the zookeeper quorum, you have to set the hbase.zookeeper.quorum property in hbase-site.xml. Also check if classpath is set properly or not, check this doc out
http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/package-summary.html#classpath
I used the configuration suggested in http://hbase.apache.org/book.html#zookeeper
stop Ipatables and start hbase with root mode.
there may be compatibility issue please check http://rajkrrsingh.blogspot.in/2013/11/hbase-compatibility-with-hadoop-2x.html
Related
When running PredictionIO on a local single node Hadoop (HortonWorks) I get the following error:
Storage Backend Connections
Verifying Meta Data Backend
Verifying Model Data Backend
Verifying Event Data Backend
ERROR client.HConnectionManager$HConnectionImplementation - The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
ERROR storage.Storage$ - Error initializing storage client for source HBASE
ERROR storage.Storage$ - org.apache.hadoop.hbase.MasterNotRunningException: The node /hbase is not in ZooKeeper. It should have been written by the master. Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
java.util.NoSuchElementException: None.get
at scala.None$.get(Option.scala:313)
at scala.None$.get(Option.scala:311)
at io.prediction.data.storage.Storage$.sourcesToClientMeta(Storage.scala:90)
at io.prediction.data.storage.Storage$.getDataObject(Storage.scala:193)
at io.prediction.data.storage.Storage$.getDataObject(Storage.scala:178)
at io.prediction.data.storage.Storage$.getLEvents(Storage.scala:274)
at io.prediction.data.storage.Storage$.verifyAllDataObjects(Storage.scala:246)
at io.prediction.tools.Console$.status(Console.scala:1259)
at io.prediction.tools.Console$$anonfun$main$1.apply(Console.scala:646)
at io.prediction.tools.Console$$anonfun$main$1.apply(Console.scala:601)
at scala.Option.map(Option.scala:145)
at io.prediction.tools.Console$.main(Console.scala:601)
at io.prediction.tools.Console.main(Console.scala)
Unable to connect to all storage backend(s) successfully. Please refer to error message(s) above. Aborting.
How should I configure HBase + Zookeepr on a local Hadoop installation?
At Hbase folder /conf/hbase-site.xml, config or add:
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>YOUR ZOOKEEPER FOLDER</value>
</property>
For example:
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/myname/zookeeper-3.4.6</value>
</property>
I have a single node hadoop and have installed hbase also on my ubuntu 12.04. Now i want to install titan over hbase. I have setup hadoop-1.0.3 and hbase-0.94.18 and titan/hbase-0.4.2
I have added a user mnit.My /usr/local/ folder contains hadoop2 , hbase2, titan2 .First i start my hadoop using command bin/start-all.sh and then i start hbase using command bin/start-hbase.sh . after it when i do jps i found the following :
mnit#aman:/usr/local$ jps
9921 DataNode
11386 HRegionServer
11041 HQuorumPeer
11537 Jps
11115 HMaster
10153 SecondaryNameNode
10252 JobTracker
9691 NameNode
10483 TaskTracker
now i start gremlin.sh in titan2 using command bin/gremlin.sh .
i applied the following commands
mnit#aman:/usr/local/titan2$ bin/gremlin.sh
gremlin> conf = new BaseConfiguration();
==>org.apache.commons.configuration.BaseConfiguration#19288c2
gremlin> conf.setProperty("storage.backend","hbase");
==>null
gremlin> conf.setProperty("storage.hostname","127.0.0.1");
==>null
gremlin> g = TitanFactory.open(conf);
WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper - Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
When i searched over this problem i found that there is a file named pom.xml but the titan that i have downloaded does not contain pom.xml. please tell me if this is a problem due to pom.xml. or i am doing something wrong or there is some other issue.
Thanks in advance
zk is managed by hbase in my system. i have added the following line in bin/hbase-env.sh
export HBASE_MANAGES_ZK=true
the content of my hbase-site.xml is as follows :
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://localhost:54310/user/hbase</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2222</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>localhost</value>
</property>
<property>
<name>hbase.zookeeper.property.datadir</name>
<value>/app/hadoop/tmp/zookeeper</value>
</property>
</configuration>
Your Titan and HBase configurations appear to be inconsistent. Your hbase-site.xml overrides the default ZK port (2181) to 2222, but it seems you haven't told Titan to use this non-default ZK port by setting storage.port in your Titan config file. Naturally, they can't talk to each other in that state. This doesn't have anything to do with pom.xml.
By the way, please don't simultaneously crosspost to SO and the aureliusgraphs Google Group. They're both good venues with slightly different purposes, but you seem to have just copy-pasted between this SO question and your subjectless thread on the aureliusgraphs list.
I am running Hadoop-1.2.1 and HBase-0.94.11 in a pseudo-distributed mode.
Due to power failure Hadoop and HBase set up went down.Next time when I restarted my machine and the pseudo-distribution set up, HBase stopped working with the following errors on HBase shell:
13/11/27 13:53:27 ERROR zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries
13/11/27 13:53:27 WARN zookeeper.ZKUtil: hconnection Unable to set watcher on znode (/hbase/hbaseid)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:172)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:450)
at org.apache.hadoop.hbase.zookeeper.ClusterId.readClusterIdZNode(ClusterId.java:61)
at org.apache.hadoop.hbase.zookeeper.ClusterId.getId(ClusterId.java:50)
at org.apache.hadoop.hbase.zookeeper.ClusterId.hasId(ClusterId.java:44)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.ensureZookeeperTrackers(HConnectionManager.java:720)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:789)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:129)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
Following are the processes :
hduser#user-ubuntu:~$ jps
16914 NameNode
19955 Jps
29460 Main
17728 TaskTracker
19776 HMaster
17490 JobTracker
17392 SecondaryNameNode
Are you sure your Zookeeper process is running (your jps listing doesn't show an entry for QuorumPeerMain)? The jps stack may not show all java processes running - try using a ps axww | grep QuorumPeerMain.
If your zookeeper refuses to start, check its logs to see if there are some stack trace clues
It's straightforward the zookeeper quorum process is not running - if it has been, there'd have been another java process:
hduser#user-ubuntu:~$ jps
16914 NameNode
19955 Jps
29460 Main
17728 TaskTracker
19776 HMaster
17490 JobTracker
17392 SecondaryNameNode
xxxxx HQuorumPeer
Zookeeper is required for HBase cluster - as it manages it.
Possible solutions:
By default HBase manages zookeeper itself i.e. starting and stopping the zookeeper quorum (the cluster of zookeeper nodes) - to verify the settings look into the file conf/hbase-evn.sh (in your hbase directory) there must be a line:
export HBASE_MANAGES_ZK=true
Basically tells HBase whether it should manage its own instance of Zookeeper or not. In case it is set to false, edit to true.
Also verify the HBase conf at conf/hbase-site.xml,
The minimum conf that should work for pseudo-distributed mode is:
<configuration>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://localhost:9000/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/<yourusername>/zookeeper</value>
</property>
</configuration>
Now stop the HBase, if it's been running:
$ ./bin/stop-hbase.sh
make the neccessary changes and start it again:
$ ./bin/start-hbase.sh
Answers you may find helpful:1 2
I am not able to start Hbase, whenever i start i get only Hmaster and Hregionserver in jps. Hquorompeer keeps missing.I checked logs and i am getting below error:
java.lang.RuntimeException: Unable to run quorum server
at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:454)
at org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:409)
at org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:151)
at org.apache.hadoop.hbase.zookeeper.HQuorumPeer.runZKServer(HQuorumPeer.java:80)
at org.apache.hadoop.hbase.zookeeper.HQuorumPeer.main(HQuorumPeer.java:70)
Caused by: java.io.IOException: Failed to process transaction type: 1 error: KeeperErrorCode = NoNode for /hbase
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:153)
at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
at org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:417)
... 4 more
Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /hbase
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.processTransaction(FileTxnSnapLog.java:211)
at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:151)
The reason that you are encountering this error could be the data directory where Zookeeper stores snapshots and logs is corrupted.
In order to avoid the HQuorumpeer daemon to die out, you need to provide a path to a new directory where zookeeper can store its snapshots. To do this you need to add the following property in Hbase.site.xml
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>location of the newly created folder</value>
<description>Property from ZooKeeper's config zoo.cfg.
The directory where the snapshot is stored.
</description>
</property>
the default path of "hbase.zookeeper.property.dataDir" is /tmp/hbase-*/zookeeper(/tmp/hbase-hadoop/zookeeper), remove it and try to start zookeeper again
Removing all files from ZooKeeper directory solved the issue. In my case
rm /var/lib/zookeeper/version-2/*
I am running an hdfs instance in pseudo-distributed mode, and tried to make another hbase instance connected to it on the same server. Logs in hadoop are fine, but I constantly got the connection failure in hbase' log
==================================================================================
2012-05-01 10:49:07,212 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181
2012-05-01 10:49:07,213 WARN org.apache.zookeeper.ClientCnxn: Session 0x13708dc552d0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
2012-05-01 10:49:08,882 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181
2012-05-01 10:49:08,882 WARN org.apache.zookeeper.ClientCnxn: Session 0x13708dc552d0001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
==================================================================================
Configuration of core-site.xml#hadoop
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
Configuration of hbase-site.xml#hbase
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://localhost:9000/hbase</value>
</property>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
I also tried to replace localhost with the actual ip of the server, but got the same error.
First, you need to make sure your hbase master node is running, you can use jps to check.
If it is not running, you can run it by start-hbase.sh command or hbase master start.
And then check its status by other commands, like netstat -an | grep 9000
Second, if the previous method does not work, check your firewall configuration such as iptables and SELinux.
Use sudo iptables -L to check your iptables configuration. You can disable the iptables by sudo service iptables stop command under redhat based linux systems.
Use getenforce to check if SElinux is in enforcing mode.
Third, check the system configuration, for example, ssh etc.
You need to replace the core hadoop jar in the $HBASE_HOME/bin/lib/hadoop-{{version}}core.jar with the one in the $HADOOP_HOME/hadoop-{{version}}core.jar
I was running into the same problem when I tried to install hbase 0.92 from hbase 0.90-xxx which was working fine, i replaced the hbase-env.sh and hbase-site.xml from the old hbase to the new but forgot to copy the hadoop core jar.
I'm always suspicious when I see localhost in a config. Also when you use localhost, then it becomes very difficult (to impossible) to access any services from the psuedo distributed system from a host other than the one you are running on.
You did say you tried the IP address, but you might want to make sure its the IP address that the node really thinks its at.
Check the zookeeper logs from when it starts up and see what IP address it "thinks" its at. There should be a line like:
2012-01-31 09:32:46,083 - INFO [main:Environment#97] - Server environment:host.name=ip-10-8-127-58.ec2.internal
Then use the value of host.name as the value for all places in ALL Hadodop, HBase, Hive, Zookeeper, etc config files that need a hostname (assuming they are all on the same machine as you said you are in psuedo-distributed mode)
Also you did not show your hbase.zookeeper.quorum setting in hbase-site.xml. That is where hbase gets its knowledge of the zookeeper's address
<property>
<name>hbase.zookeeper.quorum</name>
<value>ip-10-8-127-58.ec2.internal</value>
</property>
I think Hbase cannot find the zookeeper quorum, you have to set the hbase.zookeeper.quorum property in hbase-site.xml. Also check if classpath is set properly or not, check this doc out http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/mapreduce/package-summary.html#classpath