Apache Hadoop Common FAILURE while building Hadoop - hadoop

Hello i just uploaded the same question yesterday, but my explanation was poor.
Apache Hadoop Common FAILURE while installing Hadoop
< it's a my previous post.
i'm want to install Hadoop in a raspberry pi base on the raspbian OS.
http://data.andyburgin.co.uk/post/157450047463/running-hue-on-a-raspberry-pi-hadoop-cluster
< i refer to this site.
**Compile Hadoop
We now need to compile the Hadoop binaries, download and unpack the Hadoop 2.6.4 source, tweak pom.xml so it bypasses the documentation generation as this fails on the Pi, apply the HADOOP-9320 patch and build the binary.
cd
apt-get install oracle-java8-jdk
wget http://apache.mirror.anlx.net/hadoop/common/hadoop-2.6.4/hadoop-2.6.4-src.tar.gz
tar -zxvf hadoop-2.6.4-src.tar.gz
vi hadoop-2.6.4-src/pom.xml
Disable the problem step in by adding the following to <properties>…</properties>
<additionalparam>-Xdoclint:none</additionalparam>
Next apply the HADOOP-9320 patch
cd hadoop-2.6.4-src/hadoop-common-project/hadoop-common/src
wget https://issues.apache.org/jira/secure/attachment/12570212/HADOOP-9320.patch
patch < HADOOP-9320.patch
cd ~/hadoop-2.6.4-src/
Next install a whole bunch of build tools and libraries:
apt-get install maven build-essential autoconf automake libtool cmake zlib1g-dev pkg-config libssl-dev libfuse-dev libsnappy-dev libsnappy-java libbz2-dev**
<< i've finished this line.
but the error occurred at the next stage > **sudo mvn package -Pdist,native -DskipTests -Dtar**
how do i fix this error? please help. thank you :)

Related

Not able to build fuse-dfs to mount HDFS

My Hadoop version is 2.5.2.
I want to mount Hadoop with fuse. I have tried two methods, but both failed. One method is in:
HADOOP_HOME: ant compile-c++-libhdfs -Dlibhdfs=1`.
But it failed for it can't find build.xml. The other method is in hadoop-hdfs/target/native/main/native/fuse-dfs, I execute make but still don't have fuse-dfs.
How should I build fuse-dfs so I can mount HDFS by fuse?
I'm assuming you already have Java and Maven installed and can build the other Hadoop components. After that you run:
$ mvn package -Pnative -Drequire.fuse=true -DskipTests -Dmaven.javadoc.skip=true
This instruction is in later READMEs for fuse-dfs (hadoop-hdfs-project/hadoop-hdfs/src/main/native/fuse-dfs/doc/README)
BUILDING
fuse-dfs executable can be built by setting require.fuse option to true using Maven. For example:
in HADOOP_PREFIX: mvn package -Pnative -Drequire.fuse=true -DskipTests -Dmaven.javadoc.skip=true
The executable fuse_dfs will be located at HADOOP_PREFIX/hadoop-hdfs-project/hadoop-hdfs-native-client/target/main/native/fuse-dfs/
but the 2.5.2 source refers to the older instructions.
My build environment is a CentOS 6 server. I added the additional packages mentioned in https://wiki.apache.org/hadoop/HowToContribute under Native libraries:
For RHEL (and hence also CentOS):
yum -y install lzo-devel zlib-devel gcc gcc-c++ autoconf automake libtool openssl-devel fuse-devel cmake
References:
https://wiki.apache.org/hadoop/HowToContribute - build instructions

How to solve the error when compile the hadoop2.7.3?

The software I used as following:
And I meet the error as following:
The system I used is Centos 7 and the JDK I used is 1.7.0_07. Anyone can help me to solve the problem?
You might be missing some of the libraries required to build hadoop. Do you have all this installed?
yum -y install wget gcc gcc-c++ autoconf automake libtool zlib-devel cmake openssl openssl-devel snappy snappy-devel bzip2 bzip2-devel protobuf protobuf-devel
If libraries are not the case, you can find this project useful. This is a bash script which builds hadoop from scratch. May be there you'll find a step you've missed:
https://github.com/hadoopfromscratch/hadoopfromscratch

Janus WebRTC installation issue

I am installing Janus WebRTC Gateway in a Ubuntu Machine (14.04 64 bit). I followed the instructions as in the following link:
However, I get the following error when trying to execute janus:
https://github.com/meetecho/janus-gateway (readme.md file)
[FATAL] [janus.c:main:3670] No Janus API transport is available...
enable at least one and restart Janus
Anyone has any idea what the issue might be? I will only use the REST API without WebStockets or RabbitMQ.
I successfully installed Janus on Ubuntu 14 according to the following steps:
sudo apt-get install libmicrohttpd-dev libjansson-dev libnice-dev libssl-dev libsrtp-dev libsofia-sip-ua-dev libglib2.0-dev libopus-dev libogg-dev libini-config-dev libcollection-dev libwebsockets-dev pkg-config gengetopt automake libtool doxygen graphviz git cmake
sudo apt-get install libavformat-dev
mkdir -p ~/build
cd ~/build
git clone git://github.com/meetecho/janus-gateway.git
cd janus-gateway
sh autogen.sh
./configure --disable-data-channels --disable-websockets --disable-rabbitmq --disable-docs --prefix=/opt/janus LDFLAGS="-L/usr/local/lib -Wl,-rpath=/usr/local/lib" CFLAGS="-I/usr/local/include"
make && sudo make install
sudo make configs
Running it by:
cd /opt/janus/bin/
./janus -F /opt/janus/etc/janus/
I had this issue before, I had write a script to install everything just run this
wget https://gist.githubusercontent.com/johnmelodyme/966f474a99b6dd0cf4e7ac19ba4258da/raw/0f1779499c62eeee3e2a577ef641e94e57b71154/janus.sh && sh janus.sh
Hope This Help Much, I believe there are certain dependencies needs to be installation but you missed it. In https://github.com/meetecho/janus-gateway it stated the Dependencies needed, have to be installed without error.
It is because libmicrohttpd version is lower than requirement, Download and install libmicrohttpd manually (dont use yum or apt-get).

Octave windows binaries including classdef

Are there any octave default branch binaries for windows available somewhere?
If not, is there a simple way of getting a version including classdef for Linux? simple, because usually I do not use Linux.
I don't think there are windows builds from default (aka development) available but you may ask on the help mailinglist. (Some users there build windows binaries with MXE from development sources)
If you want to build on GNU/Linux I would suggest Debian or Ubuntu in a VM. It is as easy as installing the dependencies, clone with hg, bootstrap, configure, make... You'll find instructions on the wiki http://wiki.octave.org/Octave_for_Debian_systems#Compiling_from_source
But always keep in mind that these are development sources which generally aren't suitable for productive use and may break sometime.
Here's how I ended up doing it (approximately and skipp'in all the failed attempts).
Install VirtualBox
Create new machine, mount ubuntu image, chose dynamic HDD size (or at least 10 GB)
Install Ubuntu. If strange mixed color screen appears on first boot:
[right] CTRL+F1
[right] CTRL+F7
Open Terminal (CTRL+ALT+T)
If sceen resolution cannot be made larger 640*480:
sudo apt-get install virtualbox-guest-dkms
sudo reboot
sudo apt-get install mercurial
hg clone http://www.octave.org/hg/octave -r default
sudo apt-get install gfortran debhelper automake dh-autoreconf texinfo texlive-latex-base texlive-generic-recommended epstool transfig pstoedit libreadline-dev libncurses5-dev gperf libhdf5-serial-dev libblas-dev liblapack-dev libfftw3-dev texi2html less libpcre3-dev flex libglpk-dev libsuitesparse-dev gawk ghostscript libcurl4-gnutls-dev libqhull-dev desktop-file-utils libfltk1.3-dev libgl2ps-dev libgraphicsmagick++1-dev libftgl-dev libfontconfig1-dev libqrupdate-dev libarpack2-dev dh-exec libqt4-dev libqscintilla2-dev default-jdk dpkg-dev gnuplot-x11 libbison-dev libxft-dev llvm-3.3 (takes a while)
./bootstrap
mkdir builddir
cd builddir
../configure --enable-jit --prefix=/opt/octave3.8 JAVA_HOME=/usr/lib/jvm/default-java LLVM_CONFIG=/usr/bin/llvm-config-3.2 CFLAGS="-O2 -march=native" CXXFLAGS="-O2 -march=native" (no TargetData.h found...)
make (takes half an hour or more)
make check (takes a while, gave a failed assertion)
make install
./run-octave

Linphone compiling problems on osx

I'm trying to compile linphone on mountain lion. I have downloaded the sources and followed all the instructions in the README.macos file. The first time, when I got to '$ port install ige-mac-integration' an error occurred. It said gtk2 had to be installed with x11, however, the instructions said to install it with quartz and no_x11. I tried installing gtk2 again, this time with x11, but when I get to the compiling process it tells me quartz is needed. I can't install both, because the gtk2 installation will complain about it, and it seems I need both to complete the entire process of installing and compiling linphone.
I have tried these steps with the downloadable sources, as well as the git sources, both gave me the same problems. Linphone does not have a forum, so I couldn't ask it on their site.
I just need to get the sources and start using them, so any recommendations on how to open the linphone source, edit it, and compile/run it would be very welcome. The ideal situation would be where I can just open an xcode project file.
This is the readme file:
**********************************
* Compiling linphone on macos X *
**********************************
You need:
- Xcode (download from apple or using appstore application)
- Macports: http://www.macports.org/
Download and install macports using its user friendly installer.
- Install build time dependencies
$ port install automake autoconf libtool intltool
- Install some linphone dependencies with macports
$ port install speex
$ port install libosip2 # WARNING: currently outdated in macport
$ port install libeXosip2 #WARNING: currently outdated in macport
$ port install ffmpeg-devel
$ port install libvpx
- Install srtp (optional) for call encryption
$ port install srtp
If that fails, get from source:
$ git clone git://git.linphone.org/srtp.git
$ cd srtp && autoconf && ./configure --prefix=/opt/local && make libsrtp.a
$ sudo make install
- Install zrtpcpp (optional), for unbreakable call encryption
$ port install cmake
$ git clone git://git.linphone.org/zrtpcpp.git
$ cd zrtpcpp && cmake -Denable_ccrtp=false . && make
$ sudo make install
- Install gtk. It is recommended to use the quartz backend for better integration.
$ port install gtk2 +quartz +no_x11
$ port install hicolor-icon-theme
- Compile and install the tunnelsu
If you got the source code from git, run ./autogen.sh first
Then or otherwise, do:
$ ./configure --prefix=/opt/local && make && sudo make install
- Compile linphone
If you got the source code from git, run ./autogen.sh first.
Then or otherwise, do:
$ ./configure --prefix=/opt/local --with-readline=/opt/local --disable-strict --disable-x11 --with-srtp=/opt/local --with-gsm=/opt/local --enable-zrtp && make
Install to /opt/local
$ sudo make install
Done.
If you want to generate a portable bundle, then install gtk-mac-bundler.
Use git:
$ git clone https://github.com/jralls/gtk-mac-bundler.git
$ cd gtk-mac-bundler && make install
$ export PATH=$PATH:~/.local/bin
#make this dummy charset.alias file for the bundler to be happy:
$ sudo touch touch /opt/local/lib/charset.alias
Then run, inside linphone source tree:
1. Run configure as told before but with "--enable-relativeprefix" appended.
$ make
$ make bundle
The resulting bundle is located in linphone build directory, together with a zipped version.
For a better appearance, you can install the gtk-quartz-engine (a gtk theme) that make gtk application more similar to other mac applications (but not perfect).
$ git clone https://github.com/jralls/gtk-quartz-engine.git
$ cd gtk-quartz-engine
$ autoreconf -i
$ ./configure --prefix=/opt/local && make
$ sudo make install
Generate a new bundle to have it included.
Below is the details to configure the dependencies and build linphone.
==============================================================
Step: 1:- Dependency configuration:
I just followed the instruction which is available in "README.macos" upto sqlite3 configuration. I had some problem with dependancies configuration.
Because of macport version, srtp configuration is failed. So I have downloaded "MacPorts-2.1.3-10.8-MountainLion.pkg" and installed manually.
Because of GTK version, libsoup is failed. So I have downloaded "GTK_2.18.5-X11.pkg" and installed manually.
After the manual installation, I have followed "README.macos" again for reconfiguration.
NOTE: Sometime terminal won't recognize "wget".
--> Solution: just execute below command.
echo 'alias wget="curl -O"' >> ~/.bash_profile
Step: 2:- Compile and Install:
I got some problem related to intltool while compiling.
Solution: just I set the path by executing below commands
export PATH=$PATH:/opt/local/bin
export MANPATH=$MANPATH:/opt/local/share/man
export INFOPATH=$INFOPATH:/opt/local/share/info
Now it is successfully compiled and installed.
Step: 3:- Creating bundle to run the app:
Just I followed below commands to make the build.
If you want to generate a portable bundle, then install gtk-mac-bundler.
Use git:
$ git clone https://github.com/jralls/gtk-mac-bundler.git
$ cd gtk-mac-bundler && make install
$ export PATH=$PATH:~/.local/bin
#make this dummy charset.alias file for the bundler to be happy:
$ sudo touch touch /opt/local/lib/charset.alias
Then run, inside linphone source tree:
1. Run configure as told before but with "--enable-relativeprefix" appended.
$ make
$ make bundle
It will create "linphone.app" file in current linphone directory. It will support only for "Mountain Lion".
Step: 4:- Support for lower version: (e.g.: Lion, Snow Leopard...)
We have to configure "libiconv hack" to supporting for lower version
I think it will help you.
I wrote an answer here. It's a bit long but I sincerely hope it is clear enough and will help you.
It contains the build settings and other libs you need to include.
Cool stuff I didn't need to run the make :)
Best,

Resources