I have installed Apache sqoop on Mac using homebrew with all it's dependencies. But it seems sqoop is not doing anything at all.
Even if I just type 'sqoop help' it outputs -
Warning: /usr/local/Cellar/sqoop/1.4.6_1/libexec/bin/../../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
And no help is printed.
Other commands like snoop-import is there too but does not seem to be doing anything at all when executed.
Does sqoop not work on Mac or is there anything wrong on my setup?
Thanks.
After I installed it from this location -
http://ftp.jaist.ac.jp/pub/apache/sqoop/1.4.7/
its working fine. Though I had to set some environment variables properly. Thanks.
export HADOOP_PREFIX=/usr/local/Cellar/hadoop/2.7.7
export HADOOP_HOME=/usr/local/Cellar/hadoop/2.7.7
export HADOOP_HDFS_HOME=$HADOOP_PREFIX
export HADOOP_MAPRED_HOME=$HADOOP_HDFS_HOME
export HADOOP_YARN_HOME=/usr/local/Cellar/hadoop/2.7.7
export HCAT_HOME=/usr/local/opt/hive/libexec/hcatalog
export HADOOP_CONF_DIR=/usr/local/Cellar/hadoop/2.7.7/etc/hadoop
export ZOOKEEPER_HOME=/usr/local/Cellar/zookeeper/3.4.12
export HBASE_HOME=/usr/local/Cellar/hbase
export SQOOP_HOME=/usr/local/Cellar/sqoop
export HADOOP_CLASSPATH=`hadoop classpath`
export HIVE_HOME=/usr/local/Cellar/hive/2.1.0/
Also, not all version of Hadoop is compatible with all version of HIVE/HBASE/SQOOP. You need to install right versions.
Hope this helps.
Related
I am using a Mac Pro machine and want to install maven. So here is what I am doing. First I download the appropriate .zip. I unzip it and in terminal I type
Theodosioss-MacBook-Pro:~ theo$ export M2_HOME=/Users/theo/apache-
maven-3.5.4
and
Theodosioss-MacBook-Pro:~ theo$ export
PATH=$PATH:/Users/theodosiostziomakas/apache-maven-3.5.4
Then I check if maven is installed but I get this.
mvn --version
-bash: mvn: command not found
How to properly fix this?
First you need to add maven path to your PATH variable correctly,as others have said
export PATH=/YOUR_LOCAL_LOCATION/apache-maven-3.5.4/bin:$PATH
then you need to call source /etc/profile(pay attention to check the user type,root or no root) to make it into effect and then open a new terminal for test
Try
export PATH=$PATH:/Users/theodosiostziomakas/apache-maven-3.5.4/bin
mvn executable is inside bin folder.
You have not indicated proper dir, try as below
export PATH=/YOUR_LOCAL_LOCATION/apache-maven-3.5.4/bin:$PATH
The following works for me.
Changes are done to .bash - .profile did not work on my mac
Update the config in ~/.profile
export M2_HOME=<PATH>/apache-maven-X.Y>Z
export M2=$M2_HOME/bin
export PATH=$M2:$PATH
I downloaded the binary tarball of hadoop from here: http://hadoop.apache.org/releases.html (ver 2.8.4). I unpacked the tar.gz file and then changed the etc/hadoop-env.sh from
export JAVA_HOME={$JAVA_HOME}
to my java jdk locaction:
export JAVA_HOME=C:\Program Files\Java\jdk1.8.0_131
I also added these two lines:
export HADOOP_HOME=D:/hadoop/hadoop-2.8.4
export PATH=$PATH:$HADOOP_HOME/bin
But when i try to run
$ hadoop version
from cmd i get an error message that says
Error: HADOOP_HOME is not set correctly
What did I do wrong and how should I change the hadoop_home path for it to work?
Other than {$JAVA_HOME} has the dollar sign in the wrong spot (needs to be outside the brackets), Windows doesn't run the shell script to locate your variables
You need to set environment variables in Windows from the Control Panel. And you also need to remove all spaces from the file path of "Program Files"
Its not clear if you're using Cygwin or using Windows Linux subsystem, but it's different from the native CMD
Set the path HADOOP_HOME Environment variable as below:
export HADOOP_HOME=D:\hadoop\hadoop-2.8.4
export PATH=$PATH:$HADOOP_HOME\bin
$ hadoop version
It will work
I come across such error when I try to use hadoop-3.3.1, the latest version. I have searched a lot about "HADOOP_HOME not correctly set" and there are no useful results.
But after I downgrade to hadoop-3.2.2, this error disappears.
I think you can try the non-latest version again.
I'm trying to set my $GOPATH variable to run some example code on my machine:
$ smitego-example go run main.go
main.go:5:2: cannot find package "github.com/#GITHUB_USERNAME#/smitego" in any of:
/usr/local/go/src/pkg/github.com/#GITHUB_USERNAME#/smitego (from $GOROOT)
($GOPATH not set)
$ smitego-example export $GOPATH=$HOME
-bash: export: `=/Users/#OSX_USERNAME#': not a valid identifier
Contents of github.com/#GITHUB_USERNAME#/smitego/smitego.go:
package smitego
How can I set my GOPATH so it works always and forever?
Update, as of Go 1.8: If you're installing Go 1.8 (released: Feb 2017) or later, GOPATH is automatically determined by the Go toolchain for you.
It defaults to $HOME/go on macOS (nee OS X) - e.g. /Users/matt/go/. This makes getting started with Go even easier, and you can go get <package> right after installing Go.
For the shell: (the manual method)
~/.bash_profile should contain export GOPATH=$HOME/go and also export PATH=$GOPATH/bin:$PATH. The use of the $ is important: make sure to note where I've used it (and where I have not).
For Sublime Text:
Sublime Text menu > Preferences > Package Settings > GoSublime > Settings: User
{
"shell": ["/bin/bash"],
"env": {"GOPATH": "/Users/#USERNAME#/go/"},
}
Make sure your GOPATH is not set to the full path of the package; just the root of your go folder where src, pkg, and bin reside. If you're not using GoSublime, I'd suggest installing that first.
The accepted answer didn't work for me. I investigated and found the cause: I am using zsh, not bash.
I need to add the following two lines to ~/.zshrc:
export GOPATH=/Users/username/go
export PATH=$GOPATH/bin:$PATH
You don't put the $ prefix on a variable when you're assigning it, only when you're reading it.
export GOPATH=$HOME
To make this permanent, put the command in your .bash_profile.
That will work for Terminal shells. If you need to set environment variables that will affect GUI applications, see Environment variables in Mac OS X
Download and install Go tools
https://golang.org/doc/install
Setup Go workspace
mkdir $HOME/go && cd $HOME/go
mkdir bin pkg src
Setup Go environment
sudo vi ~/.bash_profile
export GOPATH=$HOME/go
PATH=$PATH:$GOPATH/bin
Test by creating, building and running a Go project
mkdir -p $GOPATH/src/github.com/todsul/hello
touch $GOPATH/src/github.com/todsul/hello/hello.go
go install
hello
The http://www.golang-book.com/guides/machine_setup#osx
only has instructions for setting the path on ~/.bashrc, not ~/.bash_profile which thanks to this thread was able to get my example file to build.
export GOPATH=$HOME
export PATH=$PATH:$GOPATH/bin
Other Mac users need to add the above to their ~/.bash_profile.
After installing go with brew or with package this solved my problem:
export GOROOT="/usr/local/go"
export GOPATH="$HOME/Documents/goWorkSpace"
export PATH="$HOME/Documents/goWorkSpace/bin:$PATH"
on macOS High Sierra Version 10.3.3, Go[go version go1.10.1 darwin/amd64] Installed here :
Added following on :~/.bashrc
export GOPATH=/usr/local/go
export PATH=$PATH:$GOPATH/bin
and then Go Works
People working with the latest macs and above Catalina version,
you guys need to update the .zshrc file instead of .bash.
Add the following two lines to ~/.zshrc:
export GOPATH=/Users/username/go
export PATH=$GOPATH/bin:$PATH
it should work.!!
This got change a while back, please refer to the link below to understand why .zshrc and not .bash_profile
https://eshop.macsales.com/blog/56921-moving-from-bash-to-zsh-terminal-changes-in-macos-catalina/
I am on Windows 7 and installed Cygwin and PostgresSql-8.4 on it. I have an open-source application written in C that requires to be build and for that, I am using Cygwin.
My problem is with setting the path for PostgreSql in Cygwin. As per the instruction that came with open-source, the build requires me to export path to postgreSql header and library path as follows:
export ENV_PG_INC_PATH=/usr/include/pgsql
export ENV_PG_LIB_PATH=/usr/lib/pgsql
I tried to export the same path in Windows using Cygwin as follows:
export ENV_PG_INC_PATH=$ENV_PG_INC_PATH:"/cygdrive/C/Program Files (x86)/PostgreSQL/8.4/include"
export ENV_PG_LIB_PATH=$ENV_PG_LIB_PATH:"/cygdrive/C/Program Files (x86)/PostgreSQL/8.4/lib"
But this doesn't seems to be working as when I try to access the dll's or any exe's inside these folders, it throws the error as follows:
-bash: _int.dll: command not found
I don't know what is it that I am doing wrong as I am new to Cygwin. Any help would be appreciated.
Thanks in advance.
You have to escape all spaces in file paths:
export ENV_PG_LIB_PATH=$ENV_PG_LIB_PATH:"/cygdrive/C/Program\ Files\ (x86)/PostgreSQL/8.4/lib"
I installed Hadoop (1.0.2) for a single node on Windows 7 with Cygwin, and it is working. However, I cannot get PIG (0.10.0) to see the Hadoop.
1) "Error: JAVA_HOME is not set."
I added this line to pig (under bin): export JAVA_HOME=/cygdrive/c/PROGRA~1/Java/jdk1.7.0_05
2) which: no hadoop in (/usr/local/b.....)
cygpath: cannot create short name of C:\pig-0.10.0\logs
Cannot locate pig.jar. do 'ant jar', and try again
I tried adding below lines to pig and it is still not finding hadoop. What should i do?
export PIG_HOME="/cygdrive/c/pig-0.10.0"
export PATH=$PATH:$PIG_HOME/bin
export PIG_CLASSPATH=/cygdrive/hadoop/hadoop-1.0.2/conf
You might need to add your Hadoop install to your path as well. e.g.
export HADOOP_INSTALL=/Users/yourname/dev/hadoop-0.20.203.0
export PATH=$PATH:$HADOOP_INSTALL/bin
I had same issue with pig-0.11. Seems this is cygwin specific issue.
Copying pig-0.11.1-withouthadoop to pig-withouthadoop.jar under PIG_HOME fixed the issue for me
I was trying to set up PIG on my gateway machine which has Windows 7 installed on it.
This issue is very specific to Cygwin.
After breaking my head for a couple of hours I found the solution :
Solution is very simple.
Just rename the jar file under ”pig-0.10.1-withouthadoop.jar” to “pig-withouthadoop.jar”.
Its documented here
Also, you can add path : (hadoop directory)\hadoop-v.v.v\bin to environment variables manually in Windows 7. This will solve this problem
which: no hadoop in (/usr/local/b.....)
You must visit this for installing pig 12 on hadoop 2.2.0 without any errors as it re compiles the pig library for hadoop version specified.
http://javatute.com/javatute/faces/post/hadoop/2014/installing-pig-11-for-hadoop-2-on-ubuntu-12-lts.xhtml
After following the steps, you will get the running pig without any errors on grunt.
Just enjoy doing.
% pig [return]
I had a similar problem with Pig 0.12.0 (and Hadoop 1.0.3) installed on Fedora 19.
When trying any Pig command, e.g.
pig -help
I was getting the error:
Cannot locate pig-withouthadoop.jar. do 'ant jar-withouthadoop.jar', and try again
Hadoop and Pig installation /bin folders were properly included in my PATH.
Simply copying pig-0.12.0-withouthadoop.jar to PIG_HOME folder fixed the issue for me.