I know this is a repeat question but this link on stackoverflow did not solved my problem. When i try to run Hive I get following error :
Pawandeep-Singh-2:apache-hive-1.1.1-bin PawandeepSingh1$ bin/hive
readlink: illegal option -- f
usage: readlink [-n] [file ...]
Missing Hive Execution Jar: /usr/local/Cellar/hive/apache-hive-1.1.1-bin/lib/hive-exec-*.jar
I am using Mac OSX Yosemite. Can anyone tell me what I am doing wrong here or if you could provide the link to the tutorial from where you installed Hive.
Thank you..
Download the jar manually from this link..
click here
put the jar into /usr/local/Cellar/hive/apache-hive-1.1.1-bin/lib folder.
Check if this solves your problem....
Thanks finally my friend solved the problem and now hive is running. Actually in Mac you have to use .profile file to set the path for hive installation instead of .bash_profile or .bashrc files.
Related
I'm learning how to use Postgres with Python (sqlalchemy). I'm stuck when installing psycopg2. I have the following error :
pg_config executable not found error
I know that's a very common error and there is already a lot of answers about this but I couldn't find anything to help me solve my issue.
I especially don't understand this answers :
add the path to Postgres to your .profile file by appending the
following:
PATH="/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH"
What does it mean ? what should I do exactly ?
I also tried to enter which -a pg_configin my terminal but nothing happen.
Thanks a lot !
The problem was solved by typing
PATH="/Applications/Postgres.app/Contents/Versions/latest/bin:$PATH"
into the terminal window prior to installing psycopg2.
I'm beginning with Spark so not really sure where my problem is and looking for a helpful hint here. I'm trying to run Spark (pyspark) on a windows 7 machine as an admin but it does not seem to be working (I still get the WindowsError 5). See image below:
I've downloaded the file (release 1.2.0 with pre-built for Hadoop 2.4 or later), unzipped it using tar via command line and set IPYTHON=1 before calling bin\pyspark. When I call it, pyspark runs but I get the error below as per image.
When I try calling certain SparkContext objects, I get name 'sc' is not defined.
I've got python 2.7.8 installed, Spyder IDE and am in a corporate network environment.
Does any one have a clue what could be going on here? I've looked up a few questions such as Why am i getting WindowsError: [Error 5] Access is denied? but could not find a clue.
Briefly:
I had what should be the same problem. For me, it was that the *.cmd files in the $spark/bin directory weren't marked as executable; please try to confirm by:
right clicking on pyspark2.cmd and:
properties / security tab then examine 'Read & execute'
I found the workaround on another site, that recommended downloading hadoop-winutils-2.6.0.zip (sorry don't have a link). Here is an example of the cmd to use (after moving to proper directory):
t:\hadoop-winutils-2.6.0\bin\winutils.exe chmod 777 *
I did need to run the chmod 777 cmd to make the /tmp/hive writeable too.
good luck!
(... new here - sorry for the poor formatting)
(update: Matt thanks for fixing formatting issues!)
root cause: the tar program i used on windows via tar -zxf <file.tgz> did not apply
the proper attributes to the extracted files. in this case the 'executable' files
weren't properly set. yeah, maybe i should update my version of cygwin.
I am following Joseph Adler instructions on how to install ( page 555 here - http:// it-e
books. info/book/1014/ ) Hadoop on my lubuntu.
I wrote in terminal:
wget http://archive.cloudera.com/cdh/3/hadoop-0.20.2-cdh3u4.tar.gz
tar xvfz hadoop-0.20.2-cdh3u4.tar.gz
and everything went fine, .tar.gz file was downloaded and then it was untarred.
But when I wrote
hadoop version
in the terminal, there appeared a message saying that there is no command hadoop.
Does anybody has an idea on what should I do to use (already) installed but (still) somehow invisible Hadoop?
Thanks for help!
In Linux invoking a command without prefixing its path requires the location where the command resides should be present the environment variable PATH.
Here, For executing the command you got to specify either absolute or relative path of the command. Following can be used, replace with the extracted location.
<EXTRACT_LOC_PATH>/hadoop-0.20.2-cdh3u4/bin/hadoop version
If your present working directory is /hadoop-0.20.2-cdh3u4/bin/ then ./hadoop version would be sufficient.
Whenever you are getting COMMAND NOT FOUND ERROR the problem will be there in .bashrc file only. You might not have properly set the JAVA_HOME, HADOOP_HOME and PATH Variable. So check it out whether you have given proper path for all these 3 variables or not.
I've checked answers on stackoverflow, no solutions work for my case.
Command:
bin/hadoop namenode -format
Error Message:
/bin/java: No such file or directory1.7.0_09/
/bin/java: No such file or directory1.7.0_09/
/bin/java: cannot execute: No such file or directory
Relevant change in hadoop_env.sh
# The java implementation to use. Required.
export JAVA_HOME=/usr/local/jdk1.7.0_09/
I use soft-link by
ln -s "c:\Program Files\java\jdk1.7.0_09" /usr/local/jdk1.7.0_09
Java HOME:
C:\Program Files\Java\jdk1.7.0_09
Path :
C:\cygwin64\bin;C:\cygwin64\usr\sbin
If any one has clues, please feel free to point it out. Thanks.
#xhudik #s.singh Finally! There is a problem when modifying hadoop_env.sh in Windows. I've fixed the problem with dos2unix command to eliminate dos style character.
If dos2unix command can't be found in cygwin, re-download cygwin and update it.
Please follow the link here:
https://superuser.com/questions/612435/cygwin-dos2unix-command-not-found
The command is
dos2unix hadoop_env.sh
Then everything is all set. Hope my experience would help others.
Thanks for s.singh and xhudik's help.
there is no java. Are you sure that your java binaries (./java , ./javac...) are in the specified directories? Maybe ln is a problem. Java also doesn't like " " in directory name (c:\program files) ...
You need to correctly place java distribution and then define JAVA_HOME variable. You can test it by:
$JAVA_HOME/bin/java -version
Set your java home like this:
JAVA_HOME=C:/Program Files/java/jdk1.7.0_09 in hadoop_env.sh
also you need to set Java Path in environment variable for java.
If still getting issue, then please let us know.
For learning and best practice on hadoop, try using cloudera version or Hortonworks version of hadoop . You can download their windows version. Please check link:
hortonworks.
cloudera
Or you can use IBM Smart Cloud enterprise. IBM is giving free access for students and learning.
I installed Hadoop (1.0.2) for a single node on Windows 7 with Cygwin, and it is working. However, I cannot get PIG (0.10.0) to see the Hadoop.
1) "Error: JAVA_HOME is not set."
I added this line to pig (under bin): export JAVA_HOME=/cygdrive/c/PROGRA~1/Java/jdk1.7.0_05
2) which: no hadoop in (/usr/local/b.....)
cygpath: cannot create short name of C:\pig-0.10.0\logs
Cannot locate pig.jar. do 'ant jar', and try again
I tried adding below lines to pig and it is still not finding hadoop. What should i do?
export PIG_HOME="/cygdrive/c/pig-0.10.0"
export PATH=$PATH:$PIG_HOME/bin
export PIG_CLASSPATH=/cygdrive/hadoop/hadoop-1.0.2/conf
You might need to add your Hadoop install to your path as well. e.g.
export HADOOP_INSTALL=/Users/yourname/dev/hadoop-0.20.203.0
export PATH=$PATH:$HADOOP_INSTALL/bin
I had same issue with pig-0.11. Seems this is cygwin specific issue.
Copying pig-0.11.1-withouthadoop to pig-withouthadoop.jar under PIG_HOME fixed the issue for me
I was trying to set up PIG on my gateway machine which has Windows 7 installed on it.
This issue is very specific to Cygwin.
After breaking my head for a couple of hours I found the solution :
Solution is very simple.
Just rename the jar file under ”pig-0.10.1-withouthadoop.jar” to “pig-withouthadoop.jar”.
Its documented here
Also, you can add path : (hadoop directory)\hadoop-v.v.v\bin to environment variables manually in Windows 7. This will solve this problem
which: no hadoop in (/usr/local/b.....)
You must visit this for installing pig 12 on hadoop 2.2.0 without any errors as it re compiles the pig library for hadoop version specified.
http://javatute.com/javatute/faces/post/hadoop/2014/installing-pig-11-for-hadoop-2-on-ubuntu-12-lts.xhtml
After following the steps, you will get the running pig without any errors on grunt.
Just enjoy doing.
% pig [return]
I had a similar problem with Pig 0.12.0 (and Hadoop 1.0.3) installed on Fedora 19.
When trying any Pig command, e.g.
pig -help
I was getting the error:
Cannot locate pig-withouthadoop.jar. do 'ant jar-withouthadoop.jar', and try again
Hadoop and Pig installation /bin folders were properly included in my PATH.
Simply copying pig-0.12.0-withouthadoop.jar to PIG_HOME folder fixed the issue for me.