I have a jenkins job that will do a deployment in my CentOS machine by running the docker-compose file. This is how my shell script looks ?
#!/bin/sh
# Post steps for deployment
# Navigate to deployment scripts
cd /deployment/scripts/v1.0.784
#Execute the uninstall script
./dit_undeploy_all.sh
set +e
#Remove all docker images and containers
docker container rm $(docker ps -a -q) -f
docker rmi $(docker images -a -q)
docker volume rm $(docker volume ls -q)
set -e
#Remove and clear out the folder structure
rm -rf *.*
#Gets all the latest files from Artifactory by reading from teh input file
wget -B https://artifactory.gue.com -i /deployment/scripts/inputFile.txt
# Gives rea/write access
chmod +x *.*
# Execute docker compose file to get all the latest containers
./dit_deploy_all.sh
#Add wait time for the services to be up and running
sleep 60s #Wait 15 sec
# Need to update the URL
./dit_create_policies.sh
#Verify URL Status Code of 200
cd /deployment/scripts
sleep 60s #Wait time 60s
./verifyHttpCode
The script ./verifyHttpCode does the following:
#!/bin/bash
while read LINE; do
curl -o /dev/null --silent --head --write-out '%{http_code}' "$LINE"
echo " $LINE"
done < url-list.txt
So basically after the deployment it will verify the http status code...What's the equivalent of testNG in shell script that I can use in Jenkins to verify the http status code and generate reports ??
Have you tried this plugin : Audit2DB
https://wiki.jenkins.io/display/JENKINS/Audit+To+Database+Plugin
It will log all required job details into a DB. When you want to create a report, it will fetch it from DB.
So in this case, you'll need to fail the job (non-zero exit) on script failure. Also you can set $http_code as env variable on completion of your job and log the same into DB.
I seem to be stuck here. I'm attempting to write a bash function that starts x number of docker containers, wish an array that holds exposed ports for the given app. I don't want to loop over the array, just the commands, while referencing the array to get the value. The function looks like this:
#!/bin/bash
declare -a HOSTS=( ["app1"]="8002"
["app2"]="8003"
["app3"]="8008"
["app4"]="8009"
["app5"]="8004"
["app6"]="8007"
["app7"]="8006" )
start() {
for app in "$#"; do
if [ "docker ps|grep $app" == "$app" ]; then
docker stop "$app"
fi
docker run -it --rm -d --network example_example \
--workdir=/home/docker/app/src/projects/"$app" \
--volume "${PWD}"/example:/home/docker/app/src/example \
--volume "${PWD}"/projects:/home/docker/app/src/projects \
--volume "${PWD}"/docker_etc/example:/etc/example \
--volume "${PWD}"/static:/home/docker/app/src/static \
--name "$app" --hostname "$app" \
--publish "${HOSTS["$app"]}":"${HOSTS["$app"]}" \
example ./manage.py runserver 0.0.0.0:"${HOSTS[$app]}";
echo "$app"
done
}
And I want to pass arguments like so:
./script.sh start app1 app2 app4
Right now it isn't echoing the app so that points towards the for loop being declared incorrectly...could use some pointers on this.
This line:
if [ "docker ps|grep $app" == "$app" ];
doesn't do what you want. It looks like you mean to say:
if [ "$(docker ps | grep "$app")" == "$app" ];
but you could fail to detect two copies of the application running, and you aren't looking for the application as a word (so if you look for rm you might find perform running and think rm was running).
You should consider, therefore, using:
if docker ps | grep -w -q "$app"
then …
fi
This runs the docker command and pipes the result to grep, and reports on the exit status of grep. The -w looks for a word containing the value of "$app", but does so quietly (-q), so grep only reports success (exit status 0) if it found at least one matching line or failure (non-zero exit status) otherwise.
docker ps -f lets you conveniently check programmatically whether a particular image is running.
for app in "$#"; do
if docker ps -q -f name="$app" | grep -q .; then
docker stop "$app"
:
Unfortunately, docker ps does not set its exit code (at least not in the versions I have available -- I think it has been fixed in some development version after 17.06 but I'm not sure) so we have to use an ugly pipe to grep -q . to check whether the command produced any output. The -q flag just minimizes the amount of stuff it prints (it will print just the container ID instead of a bunch of headers and columnar output for each matching container).
Is there any shortcut command to connect to a docker container without running docker exec -it 'container_id' bash every time?
Here is a shorter command line shortcut to:
Check if a container is running
If running, connect to a running container using docker exec -it <container> bash command:
Script docker-enter:
#!/bin/bash
name="${1?needs one argument}"
containerId=$(docker ps | awk -v app="$name:" '$2 ~ app{print $1}')
if [[ -n "$containerId" ]]; then
docker exec -it $containerId bash
else
echo "No docker container with name: $name is running"
fi
Then run it as:
docker-enter webapp
I'm using the following alias on OS X:
alias dex='function _dex(){ docker exec -i -t "$(basename $(pwd) | tr -d "[\-_]")_$1_1" /bin/bash -c "export TERM=xterm; exec bash" };_dex'
In the same directory as my docker-files, I run "dex php" to enter the PHP container.
If random id is complicated. Start container with name docker run --name test image and connect with its name docker exec -it test bash.
I have a shell script which runs as follows :
image_id=$(docker ps -a | grep postgres | awk -F' ' '{print $1}')
full_id=$(docker ps -a --no-trunc -q | grep $image_id)
docker exec -i -t $full_id bash
When I run this from the base linux OS, I expect to actually enter the postgres container which is a running container. But the issue is that the shell script hangs on 3rd line during ' docker exec' step.
My end goal is using the bash script, enter a running postgres container and run another bash script inside that container.
However the same command when I run it from command line, it works fine and gets me into the postgres container.
Please help, I have spent hours and hours to solve this but no progress.
Thanks again
Your setup is a bit more complex than it needs to be.
Docker ps can filter containers directly with the --filter= option
docker ps --no-trunc --quiet --filter="ancestor=postgres"
You can also --name containers when you run them which will be less fraught with danger than the script you are attempting
docker run --detach --name postgres_whatever postgres
docker exec -ti postgres_whatever bash
I'm not sure that your script is hanging as opposed to sitting there waiting for input. Try running a command directly
Using naming
exec_test.sh
#!/usr/bin/env bash
docker exec postgres_whatever echo "I have run the test"
When run
$ ./exec_test.sh
I have run the test
Without naming
exec_filter_test.sh
#!/usr/bin/env bash
id=$(docker ps --no-trunc --quiet --filter="ancestor=postgres")
[ -z "$id" ] && echo "no id" && exit 1
docker exec "${id}" echo "I have run the test"
When run
$ ./exec_filter_test.sh
I have run the test
I'd like to find out if a Docker image with a specific tag exists locally. I'm fine by using a bash script if the Docker client cannot do this natively.
Just to provide some hints for a potential bash script the result of running the docker images command returns the following:
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
rabbitmq latest e8e654c05c91 5 weeks ago 143.5 MB
busybox latest 8c2e06607696 6 weeks ago 2.433 MB
rabbitmq 3.4.4 a4fbaad9f996 11 weeks ago 131.5 MB
I usually test the result of docker images -q (as in this script):
if [[ "$(docker images -q myimage:mytag 2> /dev/null)" == "" ]]; then
# do something
fi
But since docker images only takes REPOSITORY as parameter, you would need to grep on tag, without using -q.
docker images takes tags now (docker 1.8+) [REPOSITORY[:TAG]]
The other approach mentioned below is to use docker inspect.
But with docker 17+, the syntax for images is: docker image inspect (on an non-existent image, the exit status will be non-0)
As noted by iTayb in the comments:
The docker images -q method can get really slow on a machine with lots of images. It takes 44s to run on a 6,500 images machine.
The docker image inspect returns immediately.
As noted in the comments by Henry Blyth:
If you use docker image inspect my_image:my_tag, and you want to ignore the output, you can add --format="ignore me" and it will print that literally.
You can also redirect stdout by adding >/dev/null but, if you can't do that in your script, then the format option works cleanly.
Try docker inspect, for example:
$ docker inspect --type=image treeder/hello.rb:nada
Error: No such image: treeder/hello.rb:nada
[]
But now with an image that exists, you'll get a bunch of information, eg:
$ docker inspect --type=image treeder/hello.rb:latest
[
{
"Id": "85c5116a2835521de2c52f10ab5dda0ff002a4a12aa476c141aace9bc67f43ad",
"Parent": "ecf63f5eb5e89e5974875da3998d72abc0d3d0e4ae2354887fffba037b356ad5",
"Comment": "",
"Created": "2015-09-23T22:06:38.86684783Z",
...
}
]
And it's in a nice json format.
tldr:
docker image inspect myimage:mytag
By way of demonstration...
success, found image:
$ docker image pull busybox:latest
latest: Pulling from library/busybox
Digest: sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f
Status: Image is up to date for busybox:latest
$ docker image inspect busybox:latest >/dev/null 2>&1 && echo yes || echo no
yes
failure, missing image:
$ docker image rm busybox:latest
Untagged: busybox:latest
Untagged: busybox#sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f
$ docker image inspect busybox:latest >/dev/null 2>&1 && echo yes || echo no
no
Reference:
https://docs.docker.com/engine/reference/commandline/image_inspect/
You can use like the following:
[ -n "$(docker images -q someimage:sometag)" ] || echo "does not exist"
Or:
[ -z "$(docker images -q someimage:sometag)" ] || echo "already exists"
With the help of Vonc's answer above I created the following bash script named check.sh:
#!/bin/bash
image_and_tag="$1"
image_and_tag_array=(${image_and_tag//:/ })
if [[ "$(docker images ${image_and_tag_array[0]} | grep ${image_and_tag_array[1]} 2> /dev/null)" != "" ]]; then
echo "exists"
else
echo "doesn't exist"
fi
Using it for an existing image and tag will print exists, for example:
./check.sh rabbitmq:3.4.4
Using it for a non-existing image and tag will print doesn't exist, for example:
./check.sh rabbitmq:3.4.3
In case you are trying to search for a docker image from a docker registry, I guess the easiest way to check if a docker image is present is by using the Docker V2 REST API Tags list service
Example:-
curl $CURLOPTS -H "Authorization: Bearer $token" "https://hub.docker.com:4443/v2/your-repo-name/tags/list"
if the above result returns 200Ok with a list of image tags, then we know that image exists
{"name":"your-repo-name","tags":["1.0.0.1533677221","1.0.0.1533740305","1.0.0.1535659921","1.0.0.1535665433","latest"]}
else if you see something like
{"errors":[{"code":"NAME_UNKNOWN","message":"repository name not known to registry","detail":{"name":"your-repo-name"}}]}
then you know for sure that image doesn't exist.
Using test
if test ! -z "$(docker images -q <name:tag>)"; then
echo "Exist"
fi
or in one line
test ! -z "$(docker images -q <name:tag>)" && echo exist
In bash script I do this to check if image exists by tag :
IMAGE_NAME="mysql:5.6"
if docker image ls -a "$IMAGE_NAME" | grep -Fq "$IMAGE_NAME" 1>/dev/null; then
echo "could found image $IMAGE_NAME..."
fi
Example script above checks if mysql image with 5.6 tag exists. If you want just check if any mysql image exists without specific version then just pass repository name without tag as this :
IMAGE_NAME="mysql"
Inspired by #rubicks's response above.
To check if the image exists already
image_name_tag="alpine:3.13.3"
docker image inspect ${image_name_tag} > /dev/null
echo $?
Pull if image does not exist
docker image inspect ${image_name_tag} > /dev/null || docker pull ${image_name_tag}
Demo execution
# image_name_tag="alpine:3.13.3"
#
# docker image inspect ${image_name_tag} > /dev/null
echo $?
Error: No such image: alpine:3.13.3
# echo $?
1
# docker image inspect ${image_name_tag} > /dev/null || docker pull ${image_name_tag}
Error: No such image: alpine:3.13.3
3.13.3: Pulling from library/alpine
9aae54b2144e: Pull complete
Digest: sha256:826f70e0ac33e99a72cf20fb0571245a8fee52d68cb26d8bc58e53bfa65dcdfa
Status: Downloaded newer image for alpine:3.13.3
docker.io/library/alpine:3.13.3
# docker image inspect ${image_name_tag} > /dev/null || docker pull ${image_name_tag}
$
Just a bit from me to very good readers:
Build
#!/bin/bash -e
docker build -t smpp-gateway smpp
(if [ $(docker ps -a | grep smpp-gateway | cut -d " " -f1) ]; then \
echo $(docker rm -f smpp-gateway); \
else \
echo OK; \
fi;);
docker run --restart always -d --network="host" --name smpp-gateway smpp-gateway:latest
Watch
docker logs --tail 50 --follow --timestamps smpp-gateway
Run
sudo docker exec -it \
$(sudo docker ps | grep "smpp-gateway:latest" | cut -d " " -f1) \
/bin/bash
for specific tag name
$ docker images --filter reference='<REPOSITORY>:TAG'
for "like clause" tagname:my_image_tag --> start my_ima*
$ docker images --filter reference='<REPOSITORY>:my_ima*'
if you want to someting "the image" for example delete all images tag started "my_ima" try this
docker rmi -f $(docker images -q --filter reference='myreponame:my_ima*')
I think this functionality should be implemented inside the docker build command (using a flag?), so that it avoids a lot of code duplication.
I used the same condition as the accepted answer inside a wrapper function called docker_build so that it does the necessary checks before calling the original docker build command.
# Usage: docker_build <...> (instead of docker build)
docker_build()
{
local arguments=("$#")
local index
for (( index=0; index<$#; index++ )); do
case ${arguments[index]} in
--tag)
local tag=${arguments[index+1]}
if [[ ! -z $(docker images -q "${tag}" 2> /dev/null) ]]; then
echo "Image ${tag} already exists."
return
fi
;;
esac
done
command docker build "$#"
}
Disclaimer: This is not ready for production because it works only with space-separated arguments in long format i.e --tag hello-world:latest. Also, this just modifies the docker build command only, all other commands remain same. If anyone has improvements, please let me know.
I felt like sharing this snippet because the idea of wrapping-standard-commands-in-bash-functions, to avoid code repetition, seemed more elegant and scalable than writing wrapper statements.
I like this because it is concise and has the proper semantics. If the image exists it is true what could be easier?
if [ $(docker image ls ${build_env} --format="true") ] ;
then
echo "does exist"
fi