I have installed Whirr and created an EC2 cluster. The cluster is created correctly and I can ssh to the nodes and check that Hadoop is working correctly. However, whenever I try to use the hadoop-proxy.sh, I get the following message:
bind: Cannot assign requested address
And if I try to see the HDFS in a different shell (I have previously configured the HADOOP_CONF_DIR variable), I get the following error:
13/11/29 05:15:09 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively
Bad connection to FS. command aborted. exception: Server IPC version 7 cannot communicate with client version 4
I have tried with different properties files when setting up the cluster, using CDH, without using it... But I am still getting the same error. This is the properties file that I am currently using to launch the cluster with Whirr:
whirr.cluster-name=otrotest
whirr.instance-templates=1 hadoop-namenode+yarn-resourcemanager+mapreduce-historyserver,2 hadoop-datanode+yarn-nodemanager
whirr.provider=aws-ec2
whirr.identity=MY_ID
whirr.credential=MY_SECRET_KEY
whirr.private-key-file=/home/hduser/.ssh/whirr_id_rsa
whirr.public-key-file=/home/hduser/.ssh/whirr_id_rsa.pub
whirr.env.MAPREDUCE_VERSION=2
whirr.env.repo=cdh4
whirr.hadoop.install-function=install_cdh_hadoop
whirr.hadoop.configure-function=configure_cdh_hadoop
whirr.mr_jobhistory.start-function=start_cdh_mr_jobhistory
whirr.yarn.configure-function=configure_cdh_yarn
whirr.yarn.start-function=start_cdh_yarn
whirr.hardware-id=t1.micro
whirr.image-id=us-west-2/ami-6aad335a
whirr.location-id=us-west-2
whirr.java.install-function=install_openjdk
whirr.java.install-function=install_oab_java
I am new to Whirr and I guess I am missing something... But I don't know how to solve this. Any help would be much appreciated. Thanks in advance.
Related
I'm using Nifi 1.6.0.
I'm trying to write to HDFS and to Hive (cloudera) with nifi.
On "PutHDFS" I'm configure the "Hadoop Confiugration Resources" with hdfs-site.xml, core-site.xml files, set the directories and when I'm trying to Start it I got the following error:
"Failed to properly initialize processor, If still shcedule to run,
NIFI will attempt to initalize and run the Processor again after the
'Administrative Yield Duration' has elapsed. Failure is due to
java.lang.reflect.InvocationTargetException:
java.lang.reflect.InvicationTargetException"
On "PutHiveStreaming" I'm configure the "Hive Metastore URI" with
thrift://..., the database and the table name and on "Hadoop
Confiugration Resources" I'm put the Hive-site.xml location and when
I'm trying to Start it I got the following error:
"Hive streaming connect/write error, flow file will be penalized and routed to retry.
org.apache.nifi.util.hive.HiveWritter$ConnectFailure: Failed connectiong to EndPoint {metaStoreUri='thrift://myserver:9083', database='mydbname', table='mytablename', partitionVals=[]}:".
How can I solve the errors?
Thanks.
For #1, if you got your *-site.xml files from the cluster, it's possible that they are using internal IPs to refer to components like the DataNodes and you won't be able to reach them directly using that. Try setting dfs.client.use.datanode.hostname to true in your hdfs-site.xml on the client.
For #2, I'm not sure PutHiveStreaming will work against Cloudera, IIRC they use Hive 1.1.x and PutHiveStreaming is based on 1.2.x, so there may be some Thrift incompatibilities. If that doesn't seem to be the issue, make sure the client can connect to the metastore port (looks like 9083).
We are trying to install Phoenix 4.4.0 on HBase 1.0.0-cdh5.4.4 (CDH5.5.5 four nodes cluster) via this installation document: Phoenix installation
Based on that we copied our phoenix-server-4.4.0-HBase-1.0.jar to hbase libs on each region server and master server, so that, on each /opt/cloudera/parcels/CDH-5.4.4-1.cdh5.4.4.p0.4/lib/hbase/lib folder in the master and three region servers.
After that we reboot the HBase service via Cloudera Manager.
Everything seems to be ok, but when we are trying to access to phoenix shell via ./sqlline.py localhost command, we get a Zookeeper error in that way:
15/09/09 14:20:51 WARN client.ZooKeeperRegistry: Can't retrieve clusterId from Zookeeper
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
So we are not sure that the installation is properly done. Is necessary any further configuration?
We are not even sure wether we are using the sqlline command properly.
Any help will be appreciated.
After reinstalling the 4 nodes cluster on AWS, phoenix is now working properly.
It's a pitty that we don't know exactly what was really happening, but we think that after several changes in our config, we broke something that made phoenix impossible to work.
One thing to take into consideration is that sqllline command has to be executed with an ip that is in the zookeeper quorum, and this is something we were doing wrong, since we were trying to run it from the namenode, and it wasn't in the zookeeper quorum.Once we run sqlline.py from a datanode, everything is working fine.
Btw, the installation guide that we finally followed is Phoenix Installation
i am trying to learn Hadoop and i'v reached HBase section in Hadoop Definitive Guide.
i tried to start HBase and got error. Could someone give me step-by-step guide?
opel#ubuntu:~$ zkServer.sh start
JMX enabled by default
Using config: /home/opel/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
opel#ubuntu:~$ start-hbase.sh
starting master, logging to /home/opel/hbase-0.94.20/logs/hbase-opel-master-ubuntu.out
opel#ubuntu:~$ hbase shell
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 0.94.20, r09c60d770f2869ca315910ba0f9a5ee9797b1edc, Fri May 23 22:00:41 PDT 2014
hbase(main):001:0> status
14/06/02 22:40:44 ERROR client.HConnectionManager$HConnectionImplementation: Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
14/06/02 22:40:45 ERROR client.HConnectionManager$HConnectionImplementation: Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
14/06/02 22:40:47 ERROR client.HConnectionManager$HConnectionImplementation: Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
14/06/02 22:40:49 ERROR client.HConnectionManager$HConnectionImplementation: Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
14/06/02 22:40:51 ERROR client.HConnectionManager$HConnectionImplementation: Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
14/06/02 22:40:55 ERROR client.HConnectionManager$HConnectionImplementation: Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
14/06/02 22:40:59 ERROR client.HConnectionManager$HConnectionImplementation: Check the value configured in 'zookeeper.znode.parent'. There could be a mismatch with the one configured in the master.
ERROR: org.apache.hadoop.hbase.MasterNotRunningException: Retried 7 times
Here is some help for this command:
Show cluster status. Can be 'summary', 'simple', or 'detailed'. The
default is 'summary'. Examples:
hbase> status
hbase> status 'simple'
hbase> status 'summary'
hbase> status 'detailed'
is there anything wrong?
I had the same problem. For me the solution was to add the following property to the hbase-site.xml (for me it can be found under /usr/lib/hbase/conf directory):
<configuration>
<property>
<name>zookeeper.znode.parent</name>
<value>/hbase-unsecure</value>
</property>
</configuration>
But this is only for the standalone mode. I still have no idea how to solve this problem when using external ZooKeeper.
there wont be any problem with the configurations if you are using the Cloudera Manager VM.
The problem is HMaster is not up. To resolve it, go to Cloudera Manager and restart the HBase services. it will resolve the issue.
When I had this problem I could able to fix this by not using zookeeper.
If you're running HBase in standalone mode then you don't need zookeeper. I could able to skip the zookeeper part my making the hbase.cluster.distributed property false.
<property>
<name>hbase.cluster.distributed</name>
<value>false</value>
</property>
Now, I could able to play with hbase without zookeeper.
In cloudera management page, Goto services -> hbase1 and start the sevice problem will be resolved. No need to make the hbase unsecure property.
This problem tooks me a whole night, and this is how i resolved it:
After starting hadoop, go to : http://localhost:50070/dfshealth.html#tab-datanode
You will see a list of available datanode in a table, you just need to add it in your hbase-site.xml as follow for me:
<configuration>
<property>
<name>zookeeper.znode.parent</name>
<value>127.0.0.1:50010</value>
</property>
</configuration>
Best thing check your HBase logs. It will give you the clear idea about the error. In my case i was running Kafka + zookeeper and HBase on the same server. So, whenever i was trying to run hbase shell i was kept getting same error on the console. When I checked logs and found
port is already in use
so i just changed the value for
hbase.zookeeper.property.clientPort
in hbase-site.xml file and everything start running.
open zookeeper/bin and run the command - ./zkServer.sh start
After successful execution, execute command - /zkCli.sh
then execute command get /hbase-unsecure
if it returns as null then, create -s /testmaster "127.0.0.1:2222"
Also, edit hbase-site.xml by adding
<property>
<name>zookeeper.znode.parent</name>
<value>/testmaster</value>
</property>
PS - keep the value of hbase.cluster.distributed property as false.
Hope this solves your error.
I have installed hadoop 2.2.0 & hbase-0.94.18 on ubuntu 12.04. When I try to run the command
create 't1','c1'
in hbase shell, I get the following error-
ERROR client.HConnectionManager$HConnectionImplementation:
Check the value configured in 'zookeeper.znode.parent'.
There could be a mismatch with the one configured in the master.
What's wrong?
A few things in no particular order:
To start with, let the error display continue. It will try 7 times and then exit. Before it exits, it will show the name of exception occurring. Try to look it up. It probably says MasterNotRunningException.
Verify that master is indeed running by doing $sudo jps. You should see an entry for HMaster. If not, start the hbase-master service.
Assuming you're going for pseudo-distributed mode, you may also want to check your /etc/hosts to make sure that entries point to 127.0.0.1 and not 127.0.1.1.
For cloudera's installs, here is a guide on how to setup HBase in pseudo-distributed mode. It also includes instructions to install hbase-master and zookeeper correctly.
Maybe you should check the file hbase-site.xml about zookeeper.znode.parent whether it's right. its default value is /hbase
Mine was set by default to /hbase-unsecure (hbase-site.xml)
When I set hbase.rootdir configuration in hbase-site.xml to local filesystem like file://hbase_root_dir_path, hbase worked OK.But when I change it to hdfs://localhost:9000/hbase, hbase was also OK at the beginning. After a short time(usually a few seconds), however, it didn't work.I found the HMaster stopped with jps command.Of course I could not open the localhost:60010 web page.I read the log, and found sth wrong like the following:
INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level KeeperException when processing sessionid:0x13e35b26eb80001 type:delete cxid:0x13 zxid:0xc txntype:-1 reqpath:n/a Error Path:/hbase/backup-masters/localhost,35320,1366700487007 Error:KeeperErrorCode = NoNode for /hbase/backup-masters/localhost,35320,1366700487007
INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2182. Will not attempt to authenticate using SASL (unknown error)
ERROR org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler: Failed open of region=person,,1365998702159.a5af90c23325829096517fb3b15bca17., starting to roll back the global memstore size.
java.lang.IllegalStateException: Could not instantiate a region instance.
WARN org.apache.zookeeper.ClientCnxn: Session 0x13e35b26eb80002 for server null, unexpected error, closing socket connection and attempting reconnect
I use the pseudo-distributed mode of hbase in Ubuntu 12.04 LTS.
In my /etc/hosts, I have already changed the the IP of hostname to 127.0.0.1.And my hadoop safemode status if OFF.My hadoop version is 1.0.4 and my hbase version is 0.94.6.1(both are the latest stable release), the HBase Reference guide says hbase-0.94.x works fine with hadoop-1.0.x.
I think sth about the HDFS results the problem, because it really works with the local filesystem.By the way, there is a hbase-x.x.x-security release, what's the difference between it and hbase-x.x.x release and do I need to use the security release?
Dit you set your Zookeeper quorum? It seems Zookeeper is trying to connect to your localhost.
Try setting the addresses of the machines you wan't to use using the hbase.zookeeper.quorum property in hbase-site.xml. Also, if you're not managing your own Zookeeper instance make sure that in hbase-env.sh this line isn't commented export HBASE_MANAGES_ZK=true.