I try to deploy my container from gitlab registry to EC2 Instance, I arrived to deploy my container, but when I change something, and want to deploy again, It is required to remove the old container and the old images and deploy again, for that I create this script to remove every thing and deploy again.
...
deploy-job:
stage: deploy
only:
- master
script:
- mkdir -p ~/.ssh
- echo -e "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa
- chmod 600 ~/.ssh/id_rsa
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
- ssh -i ~/.ssh/id_rsa ec2-user#$DEPLOY_SERVER "docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN registry.gitlab.com &&
docker stop $(docker ps -a -q) &&
docker rm $(docker ps -a -q) &&
docker pull registry.gitlab.com/doesntmatter/demo:latest &&
docker image tag registry.gitlab.com/doesntmatter/demo:latest doesntmatter/demo &&
docker run -d -p 80:8080 doesntmatter/demo"
When I try this script, I got this error:
"docker stop" requires at least 1 argument. <<-------------------- error
See 'docker stop --help'.
Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...]
Stop one or more running containers
Running after script
00:01
Uploading artifacts for failed job
00:01
ERROR: Job failed: exit code 1
if you look closer, I use $(docker ps -a -q) after the the docker stop.
Questions:
I know this is not the wonderful way to make my deploys (a developer here), can you please suggest other ways, just with using gitlab and ec2.
Is there any way to avoid this error, when I have or not containers in my machine?
Probably no containers were running when the job was executed.
To avoid this behavior, you can change a bit you command to have :
docker ps -a -q | xargs -r sudo docker stop
docker ps -a -q | xargs -r sudo docker rm
These will not produce errors if no containers are running.
Afterwards, indeed there are other way to deploy a container on AWS where there are services handling containers very well like ECS, EKS or Fargate. Think also about terraform to deploy your infrastructure using IaC principle (even for you ec2 instance).
Related
I have a bash script that deploys an application called enhanced-app. It is expected to clean up all running containers first before building a new image. My current code does that, but in cases where the container doesn't exist or isn't running, I get an error.
I want to only run the cleanup command if enhanced-app is running. Please how can I achieve this?
!/bin/bash
echo "Stopping App2..."
docker container stop enhanced-app
docker container rm enhanced-app
CURPATH=$(dirname "${BASH_SOURCE[0]}")
docker build . -t enhanced-app
docker run -d -p 5000:5000 --name enhanced-app enhanced-app
I believe you can use the output of docker ps for that:
#!/bin/bash
IS_RUNNING=$(docker ps --filter name=enhanced-app --format '{{.ID}}')
if [ -n "${IS_RUNNING}" ]; then
echo "enhanced-app is running. Stopping App2 and removing container..."
docker container stop enhanced-app
docker container rm enhanced-app
else
IS_STOPPED=$(docker ps -a --filter name=enhanced-app --format '{{.ID}}')
if [ -n "${IS_STOPPED}" ]; then
echo "enhanced-app is stopped. Removing container..."
docker container rm enhanced-app
else
fi
CURPATH=$(dirname "${BASH_SOURCE[0]}")
docker build . -t enhanced-app
docker run -d -p 5000:5000 --name enhanced-app enhanced-app
You can use the exit status for docker container inspect
if docker inspect -f 'Container exists and is {{.State.Status}}' enhanced-app; then
docker container stop enhanced-app
docker container rm enhanced-app
fi
I am trying to deploy a nodejs app inside docker container on a prod machine using jenkins.
I have this shell :
ssh -tt vagrant#10.2.3.129<<EOF
cd ~/app/backend
git pull
cat <<EOM >./Dockerfile
FROM node:8
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 3000
CMD [ "npm", "start" ]
EOM
docker build -t vagrant/node-web-app .
docker kill $(docker ps -q)
docker rm $(docker ps -a -q)
docker run -p 3000:3000 -d vagrant/node-web-app
exit
EOF
this will connect via ssh to prod machine and create a Dockerfile then build and run image. but it failed.
and this a part of the jenkins logs:
Successfully built 8e5796ea9846
vagrant#ubuntu-xenial:~$ docker kill
"docker kill" requires at least 1 argument.
See 'docker kill --help'.
Usage: docker kill [OPTIONS] CONTAINER [CONTAINER...]
Kill one or more running containers
vagrant#ubuntu-xenial:~$ docker rm
"docker rm" requires at least 1 argument.
See 'docker rm --help'.
Usage: docker rm [OPTIONS] CONTAINER [CONTAINER...]
Remove one or more containers
vagrant#ubuntu-xenial:~$ docker run -p 3000:3000 -d vagrant/node-web-app
0cc8b5b67f70065ace03e744500b5b66c79941b4cb36d53a3186845445435bb5
docker: Error response from daemon: driver failed programming external connectivity on endpoint stupefied_margulis (d0e4cdd5642c288a31537e1bb8feb7dde2d19c0f83fe5d8fdb003dcba13f53a0): Bind for 0.0.0.0:3000 failed: port is already allocated.
vagrant#ubuntu-xenial:~$ exit
logout
Connection to 10.2.1.129 closed.
Build step 'Execute shell' marked build as failure
Finished: FAILURE
It seems like jenkins dont execute the " $(docker ps -q) "
and " $(docker ps -a -q) "
so docker kill and docker rm got 0 arguments.
But why this happen ?
I found the issue,
Just I have to replace "$" with "\$" .
this solve the problem.
Such a feature is useful when running multiple docker commands in one that follow this pattern:
docker do_smth $(docker query_smth)
For example:
docker stop $(docker ps -q)
or
docker rm $(docker ps -a -q)
or
docker network rm $(docker inspect ... --format ...)
If the inner docker command returns an empty list, the outer command will fail because and will display the help.
"docker stop" requires at least 1 argument.
See 'docker stop --help'.
Usage: docker stop [OPTIONS] CONTAINER [CONTAINER...] [flags]
Stop one or more running containers
Is there a way to silence docker or make docker not complain on empty lists? Something like: "Kill everybody. If there is no one, job done."
This would be similar to mkdir -p exiting_directory vs mkdir exiting_directory where the former will not complain if the directories exist.
For scripting where the result may be empty, I prefer to use xargs --no-run-if-empty:
docker ps -aq | xargs --no-run-if-empty docker rm
This question explains how to stop Docker containers started from an image.
But if there are no running containers I get the error docker stop requires a minimum of one argument. Which means I can't run this command in a long .sh script without it breaking.
How do I change these commands to work even if no results are found?
docker stop $(docker ps -q --filter ancestor="imagname")
docker rm `docker ps -aq` &&
(I'm looking for a pure Docker answer if possible, not a bash test, as I'm running my script over ssh so I don't think I have access to normal script tests)
Putting this in case we can help others:
To stop containers using specific image:
docker ps -q --filter ancestor="imagename" | xargs -r docker stop
To remove exited containers:
docker rm -v $(docker ps -a -q -f status=exited)
To remove unused images:
docker rmi $(docker images -f "dangling=true" -q)
If you are using a Docker > 1.9:
docker volume rm $(docker volume ls -qf dangling=true)
If you are using Docker <= 1.9, use this instead:
docker run -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker --rm martin/docker-cleanup-volumes
Docker 1.13 Update:
To remove unused images:
docker image prune
To remove unused containers:
docker container prune
To remove unused volumes:
docker volume prune
To remove unused networks:
docker network prune
To remove all unused components:
docker system prune
IMPORTANT: Make sure you understand the commands and backup important data before executing this in production.
I spend some time with the Vagrant & CoreOS and Docker, There's so much to learn...
I work in a development environment and constantly UP and DESTROY operation So I do not want to download the docker images every time... It takes too much time, images are very heavy.
Well, I pull the images what I use most frequently and save them.
core#core-01 ~ $ docker save ubuntu:latest > /home/core/share/ubuntu.tar
core#core-01 ~ $ docker save mysql > /home/core/share/mysql.tar
core#core-01 ~ $ docker save wordpress:latest > /home/core/share/wordpress.tar
I'm loading them again if required.
core#core-03 ~ $ docker load -i=/home/core/share/wordpress.tar
core#core-04 ~ $ docker load -i=/home/core/share/mysql.tar
so far everything is OK.
But I'm having problems when I try to build the cluster.
I have two simple service database and web
database.1.service
[Unit]
Description=Run database_1
After=docker.service
Requires=docker.service
[Service]
Restart=always
RestartSec=10s
ExecStartPre=/usr/bin/docker ps -a -q | xargs docker rm
ExecStart=/usr/bin/docker run --rm --name database_1 -e "MYSQL_DATABASE=demo" -e "MYSQL_ROOT_PASSWORD=password" -p 3306:3306 mysql
ExecStartPost=/usr/bin/docker ps -a -q | xargs docker rm
ExecStop=/usr/bin/docker kill database_1
ExecStopPost=/usr/bin/docker ps -a -q | xargs docker rm
[Install]
WantedBy=local.target
web.1.service
[Unit]
Description=Run web_1
After=database.1.service
Requires=database.1.service
[Service]
Restart=always
RestartSec=10s
ExecStartPre=/usr/bin/docker ps -a -q | xargs docker rm
ExecStart=/usr/bin/docker run --rm --name web_1 --link database_1:database_1 -e "DB_USER=root" -e "DB_PASSWORD=password" -p 80:80 wordpress
ExecStartPost=/usr/bin/docker ps -a -q | xargs docker rm
ExecStop=/usr/bin/docker kill web_1
ExecStopPost=/usr/bin/docker ps -a -q | xargs docker rm
[Install]
WantedBy=local.target
How do I load mysql image (/home/core/share/mysql.tar) before the service start.
if the service starts then download the images again.
$ fleetctl start database.1.service
$ fleetctl start web.1.service
Can I Load the images as follows?
ExecStartPre=/usr/bin/docker load -i=/home/core/share/mysql.tar
The question is;
How do I create a development environment to work without an internet connection?
I think you might be over-complicating things. You should not have to explicitly ask for an image to be saved and/or reused.
According to the CoreOS documentation
The overlay filesystem works similar to git: our image now builds off of the ubuntu base and adds another layer with Apache on top. These layers get cached separately so that you won't have to pull down the ubuntu base more than once.
While this still requires an internet connection for the initial image download, subsequent launches of the container should reuse the cached image.
If you require more control, you might want to look into maintaining a private Docker registry within your CoreOS cluster. The best way I've found to do this is using Deis, which comes with a load of goodies, including a cluster-wide fault-tolerant file-system and a private Docker registry as standard.