My Hadoop version is 2.5.2.
I want to mount Hadoop with fuse. I have tried two methods, but both failed. One method is in:
HADOOP_HOME: ant compile-c++-libhdfs -Dlibhdfs=1`.
But it failed for it can't find build.xml. The other method is in hadoop-hdfs/target/native/main/native/fuse-dfs, I execute make but still don't have fuse-dfs.
How should I build fuse-dfs so I can mount HDFS by fuse?
I'm assuming you already have Java and Maven installed and can build the other Hadoop components. After that you run:
$ mvn package -Pnative -Drequire.fuse=true -DskipTests -Dmaven.javadoc.skip=true
This instruction is in later READMEs for fuse-dfs (hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/doc/README)
BUILDING
fuse-dfs executable can be built by setting require.fuse option to true using Maven. For example:
in HADOOP_PREFIX: mvn package -Pnative -Drequire.fuse=true -DskipTests -Dmaven.javadoc.skip=true
The executable fuse_dfs will be located at HADOOP_PREFIX/hadoop-hdfs-project/hadoop-hdfs-native-client/target/main/native/fuse-dfs/
but the 2.5.2 source refers to the older instructions.
My build environment is a CentOS 6 server. I added the additional packages mentioned in https://wiki.apache.org/hadoop/HowToContribute under Native libraries:
For RHEL (and hence also CentOS):
yum -y install lzo-devel zlib-devel gcc gcc-c++ autoconf automake libtool openssl-devel fuse-devel cmake
References:
https://wiki.apache.org/hadoop/HowToContribute - build instructions
Related
Hello i just uploaded the same question yesterday, but my explanation was poor.
Apache Hadoop Common FAILURE while installing Hadoop
< it's a my previous post.
i'm want to install Hadoop in a raspberry pi base on the raspbian OS.
http://data.andyburgin.co.uk/post/157450047463/running-hue-on-a-raspberry-pi-hadoop-cluster
< i refer to this site.
**Compile Hadoop
We now need to compile the Hadoop binaries, download and unpack the Hadoop 2.6.4 source, tweak pom.xml so it bypasses the documentation generation as this fails on the Pi, apply the HADOOP-9320 patch and build the binary.
cd
apt-get install oracle-java8-jdk
wget http://apache.mirror.anlx.net/hadoop/common/hadoop-2.6.4/hadoop-2.6.4-src.tar.gz
tar -zxvf hadoop-2.6.4-src.tar.gz
vi hadoop-2.6.4-src/pom.xml
Disable the problem step in by adding the following to <properties>…</properties>
<additionalparam>-Xdoclint:none</additionalparam>
Next apply the HADOOP-9320 patch
cd hadoop-2.6.4-src/hadoop-common-project/hadoop-common/src
wget https://issues.apache.org/jira/secure/attachment/12570212/HADOOP-9320.patch
patch < HADOOP-9320.patch
cd ~/hadoop-2.6.4-src/
Next install a whole bunch of build tools and libraries:
apt-get install maven build-essential autoconf automake libtool cmake zlib1g-dev pkg-config libssl-dev libfuse-dev libsnappy-dev libsnappy-java libbz2-dev**
<< i've finished this line.
but the error occurred at the next stage > **sudo mvn package -Pdist,native -DskipTests -Dtar**
how do i fix this error? please help. thank you :)
I'm using Ubuntu and new to maven i started to install maven 3 with
wget http://archive.apache.org/dist/maven/maven-3/
3.1.0/binaries/apache-maven-3.1.0-bin.tar.gz
i need to install this version to be convenient with JDK6 after that i extracted this in /opt
then edited in .bashrc
export MVN_HOME="/tmp/apache-maven-3.1.0"
export PATH="$PATH:$MVN_HOME/bin"
i tested mvn -version but got
The program 'mvn' can be found in the following packages:
* maven
* maven2
Try: sudo apt-get install <selected package>
Have you checked which directories are already in $PATH and contain a mvn ?
Change your export statement to
export PATH="$MVN_HOME/bin:$PATH"
in order to make sure that your mvn comes first. Besides that you might want to scan for variants of mvn that are already on your system:
find / -name mvn
I'm new to storm and maven i installed
Apache Maven 3.1.0
and need to use it to maven a project
when i used mvn compile exec:java -Dexec.classPathScope=compile -Dexec.mainClass=MYClass.cr
I got
The program 'mvn' can be found in the following packages:
* maven
* maven2
Try: sudo apt-get install <selected package>
except
mvn -version
Apache Maven 3.1.0
Your problem is that you are running /usr/bin/mvn (verify this by typing which mvn). This is where apt-get will install maven. However, you have installed maven somewhere under /opt, so simply running mvn isn't finding the maven you installed.
First ensure that your .bashrc is updated to include the correct path, and that the new path is before /usr/bin
PATH=/opt/apache-maven-3.1.0/bin:$PATH
Then you need to ensure that .bashrc is loaded. You can log out and in again, start a new console, or simply source it (. ~/.bashrc).
I am not able to install maven in Ubuntu 12.04
I tried : sudo apt-get install maven
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package maven is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'maven' has no installation candidate
when i run : apt-cache search maven
libapache-pom-java - Maven metadata for all Apache Software projects
libcommons-parent-java - Maven metadata for Apache Commons project
maven-ant-helper - helper scripts for building Maven components with ant
maven-repo-helper - Helper tools for including Maven metadata in Debian packages
libjenkins-plugin-parent-java - Jenkins Plugin Parent Maven POM
I am getting a message like this. How to resolve this ?
First search for Maven package....
For that run this command in terminal..
apt-cache search maven
Then install...
sudo apt-get install maven
And at last for verification...
mvn -version
Hope this will help you.... ;)
Download maven from apache maven official site and extract.
Move the application directory to /usr/local
sudo cp -R apache-maven-X.X.X /usr/local
Make a soft link in /usr/bin for universal access of mvn
sudo ln -s /usr/local/apache-maven-X.X.X/bin/mvn /usr/bin/mvn
Verify mvn installation
mvn --version
Can I install gcc++ on CentOS 6.x without `yum install gcc-c++ ....' ?
Is there any .tar or .rpm package available for download?
Yum will install rpm from it's repository.
So I don't understand why you want to avoid yum, it will solve dependencies and install them as well.
However, here is official RPM repository mirror (one of many):
http://centos.arminco.com/5/os/i386/CentOS/
Here is list of all mirrors : http://www.centos.org/modules/tinycontent/index.php?id=30
You will need at least 3 RPMs:
gcc-4.4.6-3.el6.i686.rpm
gcc-c++-4.4.6-3.el6.i686.rpm
libgcc-4.4.6-3.el6.i686.rpm
For compilation of C/C++ you will also need libstdc++, glibc, etc
When you run
yum install gcc
Everything is done
As you did not specified architecture I assume i386, but URL is very similar for x86_64:
http://centos.arminco.com/6/os/x86_64/Packages/
If you want to install it as a local user (or as a superuser)
GNU GSRC provides an easy way to do so
Link: http://www.gnu.org/software/gsrc/
After installation via bzr, simply do these:
./bootstrap
./configure --prefix=~/local
make -C gnu/gcc
(or make -C gnu/gcc MAKE_ARGS_PARALLEL="-jN" to speed up for a N-core system)
make -C gnu/gcc install