I have an installed CDH cluster and used hadoop version, but it returns only with Hadoop version. Is there any way to get maybe all installed components version number on a graphical interface? Which command can get for example Spark version number?
Open CM (hostname:portnumber) -> Hosts tab -> Host Inspector to find what version of
CM and CDH is installed across all hosts in the cluster, as well as installed cdh components list with version details
Spark version can checked in using
spark-submit --version
Spark was developed separately from Hadoop-hdfs and Hadoop-mapreduce as a standalone tool which can be be used along with Hadoop, as such most of its interfaces are are different from hadoop.
Related
I am new to HDP installation using Ambari. I want to install Hadoop 2.9.0 using Ambari web installation. My Ambari version is 2.7.0.0 and I am using HDP 3.0 which has Hadoop 3.1.0. But I need to install Hadoop 2.9.0. Can someone please let me know if this can be done? And how can this be achieved?
I have not started the cluster installation yet and I'm done with Ambari installation.
Ambari uses pre-defined software stacks.
HDP does not offer any stack with Hadoop 2.9.0
You would therefore need to manually install that version of Hadoop yourself, although you can still manage the servers (but not the Hadoop configuration) using Ambari
In any case, there's little benefit to installing a lower version of the software, plus you won't get Hortonworks support if you do that
Is there any other alternative procedure of installing either from source as described in below link?
https://cwiki.apache.org/confluence/display/AMBARI/Installation+Guide+for+Ambari+2.5.2
or
installing HDP version of Ambari.
Do I have any other alternative source to easy install Ambari eg., like from binary files?
You can build and run Apache Ambari on OSX from source (via the doc link you provided).
There are no binary files that are pre-built and offered by Ambari for OSX.
Even if you get Ambari up and running on OSX you will NOT be able to provision an HDP stack on OSX. This is because the services defined by the HDP stack only support linux or windows. Yum, a package manager utility, is required to install the services (ie. Yarn, Spark, etc.) on linux. Yum is not available on osx.
You would have to write a custom stack for it to work on osx. That would be a good deal of work.
That being said there is nothing stopping you from installing services via source or pre-packaged bins from the various apache repos on OSX. This will allow you to run Hadoop processes locally on OSX but without the niceties provided by Ambari for provisioning, managing, and monitoring.
I'm using CDH 4.7.0 and will be installing Flume to feed HDFS data. I also downloaded Flume v1.4.0 from Apache (the same version that CDH comes with. There seem to be 2 flume-ng-core files between the one that comes with CDH and the one from Apache. There versions are 1.4.0 and 1.4.0-cdh4.7.0. Should I be using 1.4.0-cdh4.7.0 or can I safely use 1.4.0?
Flume 1.4.0 and Flume version 1.4.0-cdh4.7.0 are same but 1.4.0-cdh4.7.0 is compiled and tested with CDH4.7.0 therefore 1.4.0-cdh4.7.0 is risk free to use with CDH4.7.0.
Hence I recommend to use the cdh4.7.0 version of flume along with your CDH4.7.0 version.
I like to configure Hadoop Setup in windows environment is there any step to step guide available for installing Hadoop 2.5.1 in with configuring both hadoop master and slave in windows platform.
also like to know what are all the limitations in Hadoop windows Setup?
Refer the below links to build Hadoop on windows :
http://harishshan.blogspot.in/2014/10/install-hadoop-251-on-windows-7-64bit.html
http://www.srccodes.com/p/article/38/build-install-configure-run-apache-hadoop-2.2.0-microsoft-windows-os
I am using Cloudera Manager with CDH4.2.2 for my 3+1 cluster. On starting the installation with cloudera manager, it automatically downloads and installs JDK1.6. I want to use JDK1.7 with CDH for my convinience. Is it possible or is there any version of CDH which while installating Hadoop in the cluster automatically downloads and installs and successfully runs Hadoop with JDK1.7?
If yes, may I know which version of CDH is it and where do i get to download it from?
I want to work with JDK1.7 instead of 1.6 because i want to install Apache Giraph on CDH but it seems Giraph does not fit fine with JDK1.6 and needs the JDK1.7.
With Regards,
JDK 1.7 is supported for all CDH applications as of CDH 4.4 and Cloudera Manager 4.7.
That being said, no version of Cloudera Manager 4.x installs JDK 1.7 during the installation (latest version is 4.8.2). The only version of Cloudera Manager that installs JDK 1.7 automatically is 5.0.0.
To summarize: If you want an automated installation of JDK 1.7 via Cloudera Manager, you need to upgrade to CDH 5, and CM 5.0.0. Alternatively, you could upgrade to CDH4.4, and then perform a manual installation of JDK 1.7.