I can use e.g. Marathon + Docker to easily run my long running applications, and applications are packaged as Docker images and retrieved from public/private registry.
However, if I were to create my own framework (such as has been done for Cassandra and Spark) how would I go about to "packaging" my application?
You can still use Docker containers, since Docker is supported directly in Mesos. That's actually how Marathon does it.
As an alternative, you can have multiple files and/or archives that have to be fetched prior to task start.
If you want to develop your own framework you can start from here: Framework Tutorial and Mesos Framework Development Guide.
Usually you use a containerizer to run your application/code inside: The two standard containerizer are Mesos Containerizer and Docker Containerizer (this means you can still use Docker for your own frameworks as alex mentioned).
Related
I'm a junior web developer working in a small web agency. We work on windows 10, with wampserver and mainly with Prestashop, Wordpress and Symfony websites. For every task I am given a "ticket" for which I must develop on a new branch (if needed). When the work is done I merge my branch on the develop branch which is hosted on a preproduction server, and if it is considered ok, it is then merged on the master branch, which is the website in production hosted on another server.
I was given the task to do some research on Docker and to find how it could improve our workflow.
But from what I understood yet, docker containers are just similar to Virtual Machines and are only useful for building isolated environments in order to test an application without having to think about dependencies.
But given the fact that we already have a preproduction server, I don't see the point of using Docker ? Did I miss something ?
Also, could Docker be of use for sharing projects between teamworkers (we all work on Windows) ? (for example a developer is working on a website locally, can he create a container and its image that could be used instantly as it is without configurations by another developer to work on it on his part ?)
But from what I understood yet, docker containers are just similar to Virtual Machines and are only useful for building isolated environments in order to test an application without having to think about dependencies.
No, Docker containers are not only useful for testing.
When building a correct workflow with Docker you can achieve 100% parity between development, staging and production if all use the same docker images.
But given the fact that we already have a preproduction server, I don't see the point of using Docker ? Did I miss something ?
This pre production server, aka what is normally called a staging server should also use docker to run the code.
Also, could Docker be of use for sharing projects between teamworkers (we all work on Windows) ? (for example a developer is working on a website locally, can he create a container and its image that could be used instantly as it is without configurations by another developer to work on it on his part ?)
Yes... You create the base docker images with only the necessary stuff to run in production and from them you can build other docker images with the necessary developers tools.
Production image company/app-name:
FROM php:alpine
# other your production dependencies here
Assumming that ones build the docker image with name company/app-name then the image for development.
Development image company/app-name-dev:
FROM company/app-name
# add here only developer tools
Now the developer uses both images company/app-name and company/app-name-dev during development and in the staging server only company/app-name docker image will be used to run the code.
After soem months of interaction in this flow you may even fill confident to start using the company/app-name to deploy the app to production and now you are with a 100% parity between development, staging and production.
Take a look to the Php Docker Stack for some inspiration, because I built it with this goal in mind for my latest job, but I end up to leave the company when we where in the process of adopting it in development...
But don't put all services you need in one single docker image because that is a bad practice in docker, instead use one service per docker image, like one service for PHP, another for the database, another for the Nginx server, etc.. See here how several services can be composed together with docker compose.
I'm a little bit confused about the concept of Docker for Windows.
Can I create a docker container for windows (and a windows host like Server 2016) and install a normal windows application into that container (simple: notepad.exe; advanced some more complex application programmed in Delphi)?
And can I run this container on every Docker enabled (windows) Host? Does the container starts automatically the application inside? Or can a windows docker container only provide service or webbased applications like an IIS website?
if you have Windows Server 2016, you will be able to launch Windows containers (and you will need a Linux server to launch Linux containers).
See those links
https://msdn.microsoft.com/en-us/virtualization/windowscontainers/quick_start/manage_docker
https://msdn.microsoft.com/en-us/virtualization/windowscontainers/quick_start/container_setup
https://msdn.microsoft.com/en-us/virtualization/windowscontainers/containers_welcome
In Windows, your Dockerfile will start with
FROM windowsservercore
instead of the more usual
FROM debian
or
FROM ubuntu
See some examples of IIS in (Windows) docker
https://msdn.microsoft.com/en-us/virtualization/windowscontainers/quick_start/manage_docker
or a SQL Server in docker
http://26thcentury.com/2016/01/03/dockerfile-to-create-sql-server-express-windows-container-image/
The types of application that are candidates for docker are applications that do not have a UI.
Unlike a VM docker images are very slim having only enough codebase to service a particular use case. You can however create a docker image and use VNC to view a desktop like environment; but you have to go through hoops to configure it. Far easier to have a VM if you need a gui surface.
The strength of docker is to easily create containers of servers and DB back ends. You can even run email servers, or a stack of restful services.
On my laptop I had installed mysql, IIS and PHP. With docker I migrated all of these into an image. I spin it up when I need it and in less than 10 seconds i have a working db backend, an IIS server with PHP interface. I can maintain different versions of mysql, IIS and PHP for different iterations, they are all isolated from each other and run in their own container. When I upgrade my laptop I will not need to install any of these, just the image will work.
I know the topic is a bit old, but since I just tried I thought I'll add my 2ct.
No, you cannot start a Windows application inside a container and expect its windows to appear on your desktop.
While starting such an application is possible, in fact, it's of little use because you won't be able to see or interact with the UI.
For example, you can start notepad.exe in your Windows Core Server container and verify that the process is running (using tasklist instead of taskmanager, which cannot be seen as well).
But you cannot type anything into this notepad instance or access the menu.
Hth,
mav
No. Docker is essentially Linux. Yes, you can run Docker on Windows, but what it in fact does is to install VirtualBox and run a Linux VM inside it. Docker servers generally run on Linux VM's in the cloud. The programs you can put in a Docker container are Linux programs.
I am starting to understand Docker and as far as I am aware the docker container runs on the default Linux dist where the container is installed - in my case it's a Mac OS X lightweight dist that comes with docker toolbox.
So why do I see many Docker files actually installing a distrib inside the container, does this not defeat the object of keeping things light?
For example, here is one Docker file starting with:
FROM debian:jessie
so this is installing a Docker image inside the container which is based on Debian.
I also see many others using Ubuntu, for example.
Can this step not be bypassed and software installed directly in the container use the underlining Linux dist where the container is installed?
Because, just as for physical or virtual machines, setting up a userland environment is going to be a pain without a distribution.
This is, IMO, one of the strong benefits of docker: Pick the most suitable distribution for a particular application.
A containerized application is probably going to have dependencies. To install these dependencies, it helps a lot to have a package manager. Some dependencies are also included by default in many distributions, which makes it a good idea for the container creator (application) to choose its own distribution.
Additionally, remember that packaging a whole distribution does not necessarily waste a lot of resources:
Docker images are stored as deltas against a common baseline, meaning that two images based on debian:jessie could reuse the same data for the baseline.
Distributions are actually not that large, as they are usually minified versions of the full system images.
If you really want to create a minimal image, try busybox. However, you will oftentimes find yourself outgrowing it quite fast for any real world container image.
All the documentation about deploying a Spark cluster on Amazon EC2 is relative to Linux environments. However, my distributed project is, at this moment, dependent of some Windows functionalities, and I would like to start working with a Windows cluster while making the necessary changes.
I would like to know if there is any method that makes us able to deploy a Windows Spark cluster on EC2 in a way relatively similar to the spark-ec2 script provided by Spark.
spark-ec2 currently only supports launching clusters in EC2 using specific Linux AMIs, so deploying a Windows Spark cluster is currently not possible using that tool. I doubt that spark-ec2 will ever have that capability, since all of the setup scripts it uses assume a Linux host.
That said, Databricks recently announced a community-managed index of Spark packages, and people are adding in stuff there all the time. For example, there is already a package to let you launch Spark clusters on Google's Compute Engine.
Though there doesn't currently appear to be anything for you, I would keep my eye on that community index for something that lets you launch Windows Spark clusters on EC2.
In a resource with Spark Packages, suggested by Nick, you can see recently added project by Sigmoid Analytics - that lets you launch the Spark cluster on Azure - spark_azure:
https://github.com/sigmoidanalytics/spark_azure
I am trying to deploy nodejs in a windows environment. As far as I can see nodejs is just a standard application run through the command-line. However running it that way is not very practical in a server environment. Is there a way to run it as a windows service, so you get all the benefits of controlling the execution as you have with other services? I found similar forum entries about this subject, however they are all related to running nodejs in a linux environment, how would you go about doing this in a window environment?
Nodejs application is not like mongodb, redis, nginx ... It's not supposed to run as an os service.
As a best practice, people usually use a process management tool like pm2 to run production nodejs applications. You can give it a try by creating a deploy.sh or deploy.bat as an execution file which trigger pm2 command to start or restart the application.