Am using Hadoop-2.2.0 on two node cluster, hadoop is configured correctly and working fine , Now am trying to install sqoop 2 (sqoop-1.99.3-bin-hadoop200) on it and when am trying to access sqoop 2 web UI (like localhost:12000) am getting the following .
Apache Sqoop ROOT
And when try to access cloudera.com:12000/sqoop/version am getting following
HTTP Status 404 -
And when I use this in sqoop client
[stratapps#cloudera2 ~]$ sqoop.sh client
Sqoop home directory: /usr/local/sqoop2
Sqoop Shell: Type 'help' or '\h' for help.
sqoop:000> set server --host cloudera.com --port 12000 --webapp sqoop
Server is set successfully
sqoop:000> show version --all
client version:
Sqoop 1.99.3 revision 2404393160301df16a94716a3034e31b03e27b0b
Compiled by mengweid on Fri Oct 18 14:15:53 EDT 2013
Exception has occurred during processing command
Exception: com.sun.jersey.api.client.UniformInterfaceException Message: GET http://cloudera.com:12000/sqoop/version returned a response status of 404 Not Found
My catelina.proparties file for common.loader looks like
common.loader=
${catalina.base}/lib,
${catalina.base}/lib/*.jar,
${catalina.home}/lib,
${catalina.home}/lib/*.jar,
${catalina.home}/../lib/*.jar,
/usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/*.jar,
/usr/local/hadoop/hadoop-2.2.0/share/hadoop/common/lib/*.jar,
/usr/local/hadoop/hadoop-2.2.0/share/hadoop/hdfs/*.jar,
/usr/local/hadoop/hadoop-2.2.0/share/hadoop/hdfs/lib/*.jar,
/usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/*.jar,
/usr/local/hadoop/hadoop-2.2.0/share/hadoop/mapreduce/lib/*.jar,
/usr/local/hadoop/hadoop-2.2.0/share/hadoop/tools/*.jar,
/usr/local/hadoop/hadoop-2.2.0/share/hadoop/tools/lib/*.jar,
/usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/*.jar,
/usr/local/hadoop/hadoop-2.2.0/share/hadoop/yarn/lib/*.jar
my Sqoop.proprties file org.apache.sqoop.submission.engine.mapreduce.configuration.directory looks like
# Hadoop configuration directory
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/hadoop/hadoop-2.2.0/etc/hadoop
Please share your input I goggled a lot on this but dint find any solution yet.
Thankyou,
Malleshwar
Why are you setting hostname of sqoop server to cloudera.com ? Use you machine's host address, where sqoop server is running . In case you are using a single machine, then change cloudera.com to localhost.
Related
I have linux Box with CDAP installed and I configured the Hive import and Export plugins in CDAP.
In the same machine, I have Hadoop with HIVE installed. Am able to start all of the Hadoop services and verified using jps command and create and query the hive tables.
The actual problem is when am trying to connect the hive from cdap. It is unable to connect to hive and it is throwing the below error message.
Connection string: jdbc:hive2://localhost:10000/defaultdb;auth=deligateToken;
Output Directory: /tmp/hive - this directory is already exists
Error:
I tried changing the connection string to
Option1 : Connection string: jdbc:hive2://localhost:10000/defaultdb;auth=deligateToken; - COnnection refused error
Option 2: Connection string: jdbc:hive2:// - unable to instantiate error.
Option 3: Connection string: jdbc:hive2://localhost:10001/defaultdb;auth=deligateToken; - still it is not working
I am trying to set up hadoop locally on my windows 7 computer by the instructions on the following link:
https://dimensionless.in/know-how-to-install-and-run-hadoop-on-windows-for-beginners/
I followed each single step and hadoop looks to be properly installed as I checked it by running this command: hadoop version in the windows command prompt and it returned the installed hadoop version hadoop 3.1.0 successfully.
However it failed to start all the nodes e.g. namenode, data node, yarn. I realised it must be to do with the port used as I use local port 9000 in the core site configuration: hdfs://localhost:9000 , and I checked if the port is open by running command: telnet localhost 9000, and it returned the message that failed to open the port.
Can anyone provide any guidance on the above issue which looks to be the port issue which then failed the hadoop service from starting up?
Thank you.
I am new to this field. I was checking CDH 5.8 quick-start VM to try some basic hive/impala example.
But I hit an issue, while I am opening HUE it's giving below error. I searched solution for but didnt get anything which can resolve my issue.
Configuration files located in /etc/hue/conf.empty
Potential misconfiguration detected. Fix and restart Hue.
Hive The application won't work without a running HiveServer2.
I checked the and it's up & running. Tried restarting the service & CDH, didnt help.
Hive Server2 is running [ OK ]
When navigated to Hive tried some command it gave me below error.
Could not connect to quickstart.cloudera:10000 (code THRIFTTRANSPORT): TTransportException('Could not connect to quickstart.cloudera:10000',)
FOR Impala I am getting
AnalysisException: This Impala daemon is not ready to accept user requests. Status: Waiting for catalog update from the StateStore.
Tried starting hive --service metastore but got error
[cloudera#quickstart conf.empty]$ hive --service metastore
2017-03-03 05:37:14,502 WARN [main] mapreduce.TableMapReduceUtil: The hbase-prefix-tree module jar containing PrefixTreeCodec is not present. Continuing without it.
Starting Hive Metastore Server
org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address 0.0.0.0/0.0.0.0:9083.
Not sure what is wrong or if I need to change some config. Can you anyone guide me towards the solution ?
You HiveServer2 requires Metastore up and running. Seems your Metastore Server cannot start because the port 9083 is already used by some service. Check it:
netstat -tulpn | grep 9083
If something is using this port you need to either change the port of you metastore in hive configuration or stop the application which already uses this port.
I try to run the sqoop demo from Datastax Enterprise 4.8, I set up an Analytics cluster of 4 nodes, then with another node set up MySql, and populate the data as in the demo example, I followed all the steps of the demo, and everything seems working fine until the point where I actually run the sqoop data migration command. All DBs are created correctly, and cluster is running fine (I can see it with nodetool status and with OpsCenter), but when I run the sqoop command, I got an exception:
host# /bin/dse sqoop --options-file /usr/share/dse/demos/sqoop/import.options
/usr/share/dse/bin/dse.in.sh: line 4: /bin/dse-client-tool: No such file or directory
Unable to start sqoop: jobtracker not found
The import.options file:
*cql-import
--table
npa_nxx
--cassandra-keyspace
npa_nxx
--cassandra-table
npa_nxx_data
--cassandra-column-mapping
npa:npa,nxx:nxx,latitude:lat,longitude:lon,state:state,city:city
--connect
jdbc:mysql://10.xxx.xxx.xxx/npa_nxx_demo
--username
root
--password
xxxxx
--cassandra-host
10.xxx.xxx.xxx,10.xxx.xxx.xxx*
anyone has ideas why is this error? I reinstalled the DSE, and still got the same... Thanks.
I found the reason, need to do a softlink of the dse-client-tool in /bin dir:
# ln -s /usr/shares/dse/bin/dse-client-tool /bin/dse-client-tool
then it works, not sure why the link not created during the installation...
Start DSE as an analytics node.
Edit /etc/default/dse, set HADOOP_ENABLED=1 in the cassandra.yaml to start the DSE service.
bin/dse cassandra -t
I am new to Hive and hopefully this is going to be an easy thing to solve
for someone with more experience, but I am having trouble doing it on my
own.
On my EC2 app server I am running the following command with no error:
beeline -u jdbc:hive2://master
This is working on Hive 13 which was installed through a bootstrap action
using the latest AMI version. 'master' is pointing to my EMR cluster
Then I downloaded the source for Hive 14 and built it. I have replaced my
/home/hadoop/hive directory with the package that was built.
However, if I try to execute the same command, I get an error:
scan complete in 6ms
Connecting to jdbc:hive2://master
Error: Could not open client transport with JDBC Uri: jdbc:hive2://master:
Cannot open without port. (state=08S01,code=0)
Beeline version 0.14.0 by Apache Hive
0: jdbc:hive2://master (closed)>
Running it with the port provided works correctly:
beeline -u jdbc:hive2://master:10000
I would like to be able to able to run the command without providing the
default port number.
Can anyone direct me with an instruction.
Thanks,
Hive Beeline Connection in Two Modes:
1.Embedded Mode:
If both Hive Client and Hive server are same then connect beeline by using below url:
!connect jdbc:hive2://
2.Remote Mode:
If server in one machine but client in one machine you can connect beeline using below url:
!connect jdbc:hive2://<host>:<port>