I need to boot a bunch of ec2 instances (could be virtualbox instances, doesn't matter) as a swarm. docker-machine seems to only work for doing one at a time, and that would take hours to boot 50 machines. I would like to boot the swarm master, and then in parallel boot the 49 swarm workers. This seems like a common use case, but docker-machine has no parameter I can find for the number of instances to boot.
How can I do this? Do I need to just script it myself?
Posted on the docker forums, but typically nobody answers there: https://forums.docker.com/t/how-to-spin-up-multiple-docker-machines-at-once-i-need-20-or-50-now/8749
What you're looking for is configuration management software. Chef, Ansible, Terraform and their competitors were created to deploy and manage clusters like this.
Docker-machine is great for running containers on my MacBook, but not so great for deploying cloud infrastructure. Look into deploying your swarm into Auto-Scaling Groups with the configuration management tool of your choice.
Related
Currently we have Java applications deployed in an Application server (Websphere to be exact). To fix common performance and memory related problems we encounter, we do tweakings like:
Adjust the thread pool setting - to prevent waiting threads.
Adjust the application server's garbage collection behavior.
Now, there is a plan to move them to containers (via Docker and using Spring Boot). So essentially they would be converted to Spring boot apps running on Docker containers. Now my question is, what is the equivalent of doing #1 and #2 in this kind of setup? is there still a way to adjust thread pool and garbage collection? or is it a different way now? or this shouldn't be an issue because docker swarm can manage all this and scale?
Edit: for the meantime, docker swarm will be used for managing containers. Kubernetes is still not in the picture.
Hi I am new to microservices architect and have few questions related to its deployment on production environment.
I am using spring boot microservices and want to deploy them on any open source plantform like kubernetes.
Previously I have all deployments on local servers(conventional hardware). So I don't want to move to any cloud service for now, its part of future planing. I want to have a deployment strategy that should work with servers and in future when I want to move to cloud, effort should be minimum. I need some expert suggestions. I have created a POC of deploying apps to kubernetes with minikube where I am using docker container. But for docker private repositories I need to buy their enterprise services.
And for any cloud platform I also need invest money.
What are the other choices where I can deploy microservices on my conventional hardware.
Caas vs PaaS, CF vs K8s
There are Two ways of doing it. Actually three, the third being not moving to cloud. If moving to cloud, you do it either via OCF or via Docker. for OCF, we have lot of providers including Pivotal. For Docker, you go via Kubernetes. Again there are a lot of providers, AKS, PKS, GCP, etc.
The choice is always yours.
I want to install Apache Jmeter on PKS (pivotal-container-service) for testing micro service on PCF.
I am not able to find any good resources ,did anyone tried and got success?
https://hub.docker.com/r/justb4/jmeter/
https://www.virtuallyghetto.com/2018/03/getting-started-with-vmware-pivotal-container-service-pks-part-1-overview.html
https://network.pivotal.io/products/pcfdev
Looking into Pivotal Container Service main page
Production-ready Kubernetes for the enterprise
So Pivotal Container Service seems to be using Kubernetes under the hood.
The process of installation of JMeter into Kubernetes and/or Pivotal Container Service would be as easy as creating an Image with your own Docker JMeter installation (or use any existing one) and using this image in the PCS.
I want to create a distributed cluster in spring xd.
I am able to create a cluster with single admin, one zookeeper, one instance of redis and hsqldb.
But when i'm trying to do that with multiple instance of zookeeper , hsqldb, redis ,i'm not able to configure it correctly.
You should only have a single instance of zookeeper, hsqldb and redis. All xd-admins should be configured to connect to the same instance of each of these services and so should the xd-containers be.
Like Thomas has mentioned, the idea is that you have your (multiple) instances of admin and containers deployed, and all connect to the same zk,redis, hsqldb & rabbitmq.
Why do you want to start multiple instances of these applications?
Zookeeper provides the topology of the cluster and manages deployments. Also, it makes sure to note when nodes go up and down - avoiding single point of failures when you have many xd-admin instances (one is leader and the others replicate, they will become leader if the current one fails).
Or are you talking about making those instance parallel to avoid a SPOF? In that case, you should try to dedicate an entire VM for each of those applications.
In a distributed setup, how do I programatically start containers?
More specifically, does there exist any API similar to deploying and undeploying streams for setting up and tearing down containers?
There is currently no way to do this via an API. Containers are only known to the cluster after they are started. Upon initialization, the container registers itself with ZooKeeper. Running a container requires XD to be installed on that host which is currently a manual process: download,unzip,configure, as is starting the container. Some automation of operations will likely be provided in a future release.