I installed Hortonworks Sandbox via Virtualbox. And when i started ambari every service is stopped like you can see in this screenshot screenshot
I tried to start manually each of the services but nothing happens when i click the start button. And plus, i have many erros in my notifications section.
Also this is what my ambari agent logs looks like log1 log2
Any idea on how i can resolve this?
#emeric
You must first start HDFS. You may get warnings that require executing this command:
hdfs dfsadmin -safemode leave
Once HDFS is started, you can begin to start YARN, and other services. If you do not have enough resources in your environment you will not be able to start everything. So only start what you need.
Related
I just installed Hortonwroks Sandbox via virtualbox. And when i started Ambari every services was red like you can see in this screenshot . Have i missed something? i'm a beginner in hadoop
Actually, when we start HDP Sandbox all Services services go into the starting stage except Strome, Atlas, Hbase (This can be checked by Gear Icon on the top right side from there you can check the reason behind of failed Services).
Try to Manually Start services in the following manner
Zookeeper
HDFS
YARN
MapReduce
Hive
I set-up a new Hadoop Cluster with Hortonworks Data Platform 2.5. In the "old" cluster (installed HDP 2.4) I was able to see the information about running Spark jobs via the History Server UI by clicking the link show incomplete applications:
Within the new installation this link opens the page, but it always sais No incomplete applications found! (when there's still an application running).
I just saw, that the YARN ResourceManager UI shows two different kind of links in the "Tracking UI" column, dependent on the status of the Spark application:
application running: Application Master
this link opens http://master_url:8088/proxy/application_1480327991583_0010/
application finished: History
this link opens http://master_url:18080/history/application_1480327991583_0009/jobs/
Via the YARN RM link I can see the running Spark app infos, but why can't I access them via Spark History Server UI? Was there somethings changed from HDP 2.4 to 2.5?
I solved it, it was a network problem: Some of the cluster hosts (Spark slaves) couldn't reach each other due to a incorrect switch configuration. Found it out, as I tried to ping each host from each other.
Since all hosts can ping each other hosts the problem is gone and I can see active and finished jobs in my Spark History server UI again!
I didn't noticed the problem, because the ambari-agents worked on each host, and the ambari-server was also reachable from each cluster host! However, since ALL hosts can reach each other the problem is solved!
I have installed CDH 5.5.1 with Hue, Hadoop, Spark, Hive, Oozie, Yarn and ZooKeeper.
When I run a Spark job or MapReduce job, Hue displays a issue in the job history. The problem is that when I restart the CDH services (Not the physical nodes), it removes all the job histories that were before the restart.
On Hadoop there are several files that I suspect have information about the task and might be the ones that hold the job information. Their hadoop paths are:
/tmp/logs/user/logs/
/user/history/done/2016/
I have looked for it in the Cloudera Manager configuration page, Hue configuration page and some configuration files with no success. I don't know how to prevent this removal. Am I missing something?
If you really just need to see job history on a Hadoop cluster, the YARN History Server should have a history of all YARN jobs run on the cluster.
Hue has a JIRA ticket for the issue you describe, titled "Job browser should talk to the YARN history server to display old jobs": https://issues.cloudera.org/browse/HUE-2558. Basically, Hue needs to talk to the YARN History Server (not just the Resource Manager) to get the information you're looking for.
The good news is that the task appears to have been completed and included with the release of Hue 4.0, which occurred on 5/11/2017. The bad news is that Cloudera has not yet done a release with that version of Hue rolled in.
I have established a single node hadoop cluster on Cent OS 7. It is successfully installed and all the daemons are up.
My challenge is when I go to web interface like ***IP_Address:50070*** it is not showing up anything. Please suggest how do I resolve it.
Things I tried:
Reconfigured properties, formatted HDFS and restarted all the daemons.
Please suggest. Thanks!
I could access most functionality of hadoop admin site, like below:
But, when I tried to visit the history of each application, I am no luck any more:
Any body know what happens to my environment? Where should I check?
BTW, when I try to run "netstat -a" on my VM, I found no records for port 8088 or 19888, which is very unreasonable to me, because 8088 lead to hadoop main-page and works well.
In this web interface, you can see your jobs in real time if they are running or the history :
Once a M/R finish, the ressource manager does'nt matter of it. This is the job of the historyServer.
Your historyServer (optionnal part of hadoop YARN) seems not to be launched.
It's this service which listen on 19888.
You can launch it with the command : /etc/init.d/hadoop-mapreduce-historyserver start