I loved DCOS demos on Azure. No I wonder - having a private OpenStack based clud how to install Mesos with that UI manually? Is it possible or it is a part of DCOS they do not provide as OpenSource product?
The DCOS Dashboard is pretty cool :-). Currently it is just available via the DCOS beta on AWS and Azure. There will be on prem packages later on as well, potentially even a community edition. Feel free to contact/follow Mesosphere for updates.
Until then you can use the standard Mesos, Marathon, and Chronos UIs as Alex pointed out.
You can use Mesos and Marathon WebUI, by default they are available on ports 5050 and 8080 respectively.
Related
We have an existing Apache Mesos cluster and want to try DCOS in its shiny new Open Source form. However, it would be painful to do a destructive re-install of DCOS. So is it possible to just 'overlay' DCOS on an existing Mesos installation? Would any of the steps change in the DCOS installation guide or could the installer detect the existing Mesos and install DCOS components over it?
I don't think you can simply overlay DC/OS on top of your Mesos cluster. There are multiple reasons for that; one of those is that configuration is for Mesos and marathon is done differently in DC/OS as it is done for Mesos clusters.
At work we use Amazon linux Ec2 instances for production purposes. Also, for our internal dev setup we use openstack Cent OS instances.
I want to make a common CLI or expose REST APIs to start and stop instances on both these cloudstacks. (I already have machine images). I understand I can use any of the common SDKs (I plan to use GO) and build this.
Recently, I came across this. I am just wondering if such a thing is already available. Or does the above repo mean something else? There have been also some other articles which mention EC2 support for openstack. I am not sure if it means the same as I what I want to achieve.
There already is some compatibility with ec2 command line clients, for Nova, what you have linked to expands on that to include some network functions (VPC etc.), and openstack heat is compatible with some aws cloudformation templates.
have you looked at euca2ools? - this client was developed by Eucalyptus cloud and is compatible with AWS and nova EC2
I was looking at Rancher(an orchestration engine for docker). I think there isn't build in support of hadoop setup.
Take a look at the latest version of Rancher, it has a catalog function that includes Hadoop deployment out of the box.
this is in the latest 0.49 release of Rancher for sure.
One source of information would be "Docker Releases Orchestration Tool Kit", which mentions docker machine, docker swarm, and more importantly, built on top of the swarm API, mesosphere.
Mesosphere’s technology is the only way for an organization to run a Docker Swarm workload in a highly elastic way on the same cluster as other types of workloads.
For example, you can run Cassandra, Kafka, Storm, Hadoop and Docker Swarm workloads alongside each other on a single Mesosphere cluster, all sharing the same resources.
I have an query, If we can use Mesos Cluster by directly installing master and slave nodes. Then why do we need DCOS , is it that DCOS provides additional support along with mesos cluster. Please elaborate on this part.
Depends on your needs :-):
Here what in my opinion the Community Edition (the Enterprise Edition includes more proprietary features such as security) of DCOS adds to self setup of Mesos:
Easy setup, including Marathon and MesosDNS.
Command Line Interface with one Click install from the Universe. I personally especially like the simple installs of these services as it is really simple to install for example HDFS or cassandra in your cluster. Note: As with the above you can probably with some effort configure such setup yourself as both projects are on Github.
Very nice UI
So overall I would summarize DCOS provides a very easy and tested best-practice setup of Mesos and its ecosystem.
Hope this helps!
I would like to test out Hadoop & HBase in Amazon EC2, but I am not sure how complicate it is. Is there a stable community AMI that has Hadoop & HBase installed? I am thinking of something like bioconductor AMI
Thank you.
I highly recommend using Amazon's Elastic MapReduce service, especially if you already have an AWS/EC2 account. The reasons are:
EMR comes with a working Hadoop/HBase cluster "out of the box" - you don't need to tune anything to get Hadoop/HBase working. It Just Works(TM).
Amazon EC2's networking is quite different from what you are likely used to. It has, AFAIK, a 1-to-1 NAT where the node sees its own private IP address, but it connects to the outside world on a public IP. When you are manually building a cluster, this causes problems - even using software like Apache Whirr or BigTop specifically for EC2.
An AMI alone is not likely to help you get a Hadoop or HBase cluster up and running - if you want to run a Hadoop/HBase cluster, you will likely have to spend time tweaking the networking settings etc.
To my knowledge there isn't, but you should be able to easily deploy on EC2 using Apache Whirr which is a very good alternative.
Here is a good tutorial to do this with Whirr, as the tutorial says you should be able to do this in minutes !
The key is creating a recipe like this:
whirr.cluster-name=hbase
whirr.instance-templates=1 zk+nn+jt+hbase-master,5 dn+tt+hbase-regionserver
whirr.provider=ec2
whirr.identity=${env:AWS_ACCESS_KEY_ID}
whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
whirr.hardware-id=c1.xlarge
whirr.image-id=us-east-1/ami-da0cf8b3
whirr.location-id=us-east-1
You will then be able to launch your cluster with:
bin/whirr launch-cluster --config hbase-ec2.properties