Looping over arguments in bash array for docker commands? - bash

I seem to be stuck here. I'm attempting to write a bash function that starts x number of docker containers, wish an array that holds exposed ports for the given app. I don't want to loop over the array, just the commands, while referencing the array to get the value. The function looks like this:
#!/bin/bash
declare -a HOSTS=( ["app1"]="8002"
["app2"]="8003"
["app3"]="8008"
["app4"]="8009"
["app5"]="8004"
["app6"]="8007"
["app7"]="8006" )
start() {
for app in "$#"; do
if [ "docker ps|grep $app" == "$app" ]; then
docker stop "$app"
fi
docker run -it --rm -d --network example_example \
--workdir=/home/docker/app/src/projects/"$app" \
--volume "${PWD}"/example:/home/docker/app/src/example \
--volume "${PWD}"/projects:/home/docker/app/src/projects \
--volume "${PWD}"/docker_etc/example:/etc/example \
--volume "${PWD}"/static:/home/docker/app/src/static \
--name "$app" --hostname "$app" \
--publish "${HOSTS["$app"]}":"${HOSTS["$app"]}" \
example ./manage.py runserver 0.0.0.0:"${HOSTS[$app]}";
echo "$app"
done
}
And I want to pass arguments like so:
./script.sh start app1 app2 app4
Right now it isn't echoing the app so that points towards the for loop being declared incorrectly...could use some pointers on this.

This line:
if [ "docker ps|grep $app" == "$app" ];
doesn't do what you want. It looks like you mean to say:
if [ "$(docker ps | grep "$app")" == "$app" ];
but you could fail to detect two copies of the application running, and you aren't looking for the application as a word (so if you look for rm you might find perform running and think rm was running).
You should consider, therefore, using:
if docker ps | grep -w -q "$app"
then …
fi
This runs the docker command and pipes the result to grep, and reports on the exit status of grep. The -w looks for a word containing the value of "$app", but does so quietly (-q), so grep only reports success (exit status 0) if it found at least one matching line or failure (non-zero exit status) otherwise.

docker ps -f lets you conveniently check programmatically whether a particular image is running.
for app in "$#"; do
if docker ps -q -f name="$app" | grep -q .; then
docker stop "$app"
:
Unfortunately, docker ps does not set its exit code (at least not in the versions I have available -- I think it has been fixed in some development version after 17.06 but I'm not sure) so we have to use an ugly pipe to grep -q . to check whether the command produced any output. The -q flag just minimizes the amount of stuff it prints (it will print just the container ID instead of a bunch of headers and columnar output for each matching container).

Related

Makefile variable value from command

I have a Docker container running and I would like to kill it using make kill.
Here's my Makefile:
kill:
CONTAINER=$(docker ps -a -q --filter ancestor=container-name); \
docker kill $$CONTAINER
It gives error:
CONTAINER=; \
docker kill $CONTAINER
"docker kill" requires at least 1 argument.
See 'docker kill --help'.
It seems that the variable CONTAINER is empty.
However running in the shell:
$(docker ps -a -q --filter ancestor=container-name)
Returns the container id, in fact it prints:
c1cddc4d19a0: command not found
I assume you have not defined a make variable named docker ps -a -q --filter ancestor=container-name, and you instead want to run that as a program and obtain its output.
If so, you need to escape the $ here like you did for the variable:
kill:
CONTAINER=$$(docker ps -a -q --filter ancestor=container-name); \
docker kill $$CONTAINER
Otherwise make thinks that $(docker ps -a -q --filter ancestor=container-name) is a reference to a non-existent variable and will substitute the empty string.

Bash Script fails with error: OCI runtime exec failed

I am running the below script and getting error.
#!/bin/bash
webproxy=$(sudo docker ps -a --format "{{.Names}}"|grep webproxy)
webproxycheck="curl -k -s https://localhost:\${nginx_https_port}/HealthCheckService"
if [ -n "$webproxy" ] ; then
sudo docker exec $webproxy sh -c "$webproxycheck"
fi
Here is my docker ps -a output
$sudo docker ps -a --format "{{.Names}}"|grep webproxy
webproxy-dev-01
webproxy-dev2-01
when i run the command individually it works. For Example:
$sudo docker exec webproxy-dev-01 sh -c 'curl -k -s https://localhost:${nginx_https_port}/HealthCheckService'
HEALTHCHECK_OK
$sudo docker exec webproxy-dev2-01 sh -c 'curl -k -s https://localhost:${nginx_https_port}/HealthCheckService'
HEALTHCHECK_OK
Here is the error i get.
$ sh healthcheck.sh
OCI runtime exec failed: exec failed: container_linux.go:348: starting container process caused "exec: \"webproxy-dev-01\": executable file not found in $PATH": unknown
Could someone please help me with the error. Any help will be greatly appreciated.
Because the variable contains two tokens (on two separate lines) that's what the variable expands to. You are running
sudo docker exec webproxy-dev-01 webproxy-dev2-01 ...
which of course is an error.
It's not clear what you actually expect to happen, but if you want to loop over those values, that's
for host in $webproxy; do
sudo docker exec "$host" sh -c "$webproxycheck"
done
which will conveniently loop zero times if the variable is empty.
If you just want one value, maybe add head -n 1 to the pipe, or pass a more specific regular expression to grep so it only matches one container. (If you have control over these containers, probably run them with --name so you can unambiguously identify them.)
Based on your given script, you are trying to "exec" the following
sudo docker exec webproxy-dev2-01
webproxy-dev-01 sh -c "curl -k -s https://localhost:${nginx_https_port}/HealthCheckService"
As you see, here is your error.
sudo docker exec webproxy-dev2-01
webproxy-dev-01 [...]
The problem is this line:
webproxy=$(sudo docker ps -a --format "{{.Names}}"|grep webproxy)
which results in the following (you also posted this):
webproxy-dev2-01
webproxy-dev-01
Now, the issue is, that your docker exec command now takes both images names (coming from the variable assignment $webproxy), interpreting the second entry (which is webproxy-dev-01 and sepetrated by \n) as the exec command. This is now intperreted as the given command which is not valid and cannot been found: That's what the error tells you.
A workaround would be the following:
webproxy=$(sudo docker ps -a --format "{{.Names}}"| grep webproxy | head -n 1)
It only graps the first entry of your output. You can of course adapt this to do this in a loop.
A small snippet:
#!/bin/bash
webproxy=$(sudo docker ps -a --format "{{.Names}}"| grep webproxy )
echo ${webproxy}
webproxycheck="curl -k -s https://localhost:\${nginx_https_port}/HealthCheckService"
while IFS= read -r line; do
if [ -n "$line" ] ; then
echo "sudo docker exec ${line} sh -c \"${webproxycheck}\""
fi
done <<< "$webproxy"

Impossible to start a script into my docker container

I am trying to create my own image based on Centos.
I don't understand why when I use CMD command in my docker file to execute a script at startup, it's impossible to start my image (Exited (0) immediatly).
If build without the CMD command and then I connect to the container and I execute "sh /opt/jbossEAP/Mock/scripts/mock_start.sh". I have no issue
I have tryied to use entrypoint command but same result :(
FROM centos:7
ENV container docker
RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in *; do [ $i == \
systemd-tmpfiles-setup.service ] || rm -f $i; done); \
rm -f /lib/systemd/system/multi-user.target.wants/*;\
rm -f /etc/systemd/system/*.wants/*;\
rm -f /lib/systemd/system/local-fs.target.wants/*; \
rm -f /lib/systemd/system/sockets.target.wants/*udev*; \
rm -f /lib/systemd/system/sockets.target.wants/*initctl*; \
rm -f /lib/systemd/system/basic.target.wants/*;\
rm -f /lib/systemd/system/anaconda.target.wants/*;
VOLUME [ "/sys/fs/cgroup" ]
CMD ["/usr/sbin/init"]
RUN yum update -y
RUN mkdir -p /opt/jbossEAP/Mock/scripts/
ADD ./scripts /opt/jbossEAP/Mock/scripts/
RUN chmod +x /opt/jbossEAP/Mock/scripts/mock_start.sh
### START SCRIPT ###
CMD sh /opt/jbossEAP/Mock/scripts/mock_start.sh
mock_start.sh
#!/bin/sh
############################################
echo "hello"
I suspect your CMD or ENTRYPOINT does work, but that the container simply finishes after outputting hello
You can check your docker's output even after it has been stopped with:
docker logs <container-id>
Read https://stackoverflow.com/a/28214133/4486184 for more information on how it works and how to avoid that.
My guesses could be wrong, so please always add to your question:
How you start your docker image
docker ps -a's output
the relevant part of docker logs <container-id>'s output
You're right!!!
I just add and now it's ok.
CMD sh /opt/jbossEAP/Mock/scripts/mock_start.sh && tail -f /dev/null
Thank you very much

How to run multiple entrypoint scripts one after another inside docker container?

I am trying to match the host UID with container UID as below.
Dockerfile
RUN addgroup -g 1000 deploy \
&& adduser -D -u 1000 -G deploy -s /bin/sh deploy
USER deploy
COPY entrypoint.sh /
ENTRYPOINT ["/entrypoint.sh"]
CMD ["php-fpm7","-F"]
entrypoint.sh
whoami # it outputs `deploy`
# Change UID of 'deploy' as per host user UID
HOST_CURRENT_USER_ID=$(stat -c "%u" /var/www/${PROJECT_NAME})
if [ ${HOST_CURRENT_USER_ID} -ne 0 ]; then
gosu root usermod -u ${HOST_CURRENT_USER_ID} deploy
gosu root groupmod -g ${HOST_CURRENT_USER_ID} deploy
fi
whoami # It outputs as unknown user id 1000.
Please note the output of whoami above. Even If I changed the UID of deploy to host uid, the entrypoint script process doesn't get changed as the entrypoint shell has been called by UID 1000.
So I came up in a solution to make two entry point script one is to change the UID and another one is for container's bootstrap process which will be run in a separate shell after I change the UID of deploy. So how can I make two entrypoint run after another. E.g something like
ENTRYPOINT ["/fix-uid.sh && /entrypoint.sh"]
It looks like you're designing a solution very similar to one that I've created. As ErikMD mentions, do not use gosu to switch from a user to root, you want to go the other way, from root to a user. Otherwise, you will have an open security hole inside your container than any user can become root, defeating the purpose of running a container as a different user id.
For the solution that I put together, I have it work whether the container is run in production as just a user with no volume mounts, or in development with volume mounts by initially starting the container as root. You can have an identical Dockerfile, and change the entrypoint to have something along the lines of:
#!/bin/sh
if [ "$(id -u)" = "0" ]; then
fix-perms -r -u deploy -g deploy /var/www/${PROJECT_NAME}
exec gosu deploy "$#"
else
exec "$#"
fi
The fix-perms script above is from my base image, and includes the following bit of code:
# update the uid
if [ -n "$opt_u" ]; then
OLD_UID=`getent passwd "${opt_u}" | cut -f3 -d:`
NEW_UID=`ls -nd "$1" | awk '{print $3}'`
if [ "$OLD_UID" != "$NEW_UID" ]; then
echo "Changing UID of $opt_u from $OLD_UID to $NEW_UID"
usermod -u "$NEW_UID" -o "$opt_u"
if [ -n "$opt_r" ]; then
find / -xdev -user "$OLD_UID" -exec chown -h "$opt_u" {} \;
fi
fi
fi
(Note, I really like your use of stat -c and will likely be updating my fix-perms script to leverage that over the ls command I have in there now.)
The important part to this is running the container. When you need the fix-perms code to run (which for me is only in development), I start the container as root. This can be a docker run -u root:root ... or user: "root:root" in a compose file. That launches the container as root initially, which triggers the first half of the if/else in the entrypoint that runs fix-perms and then runs a gosu deploy to drop from root to deploy before calling "$#" which is your command (CMD). The end result is pid 1 in the container is now running your command as the deploy user.
As an aside, if you really want an easier way to run multiple entrypoint fragments in a way that's easy to extend with child images, I use an entrypoint.d folder that is processed by an entrypoint script in my base image. To code to implement that logic is as simple as:
for ep in /etc/entrypoint.d/*.sh; do
if [ -x "${ep}" ]; then
echo "Running: ${ep}"
"${ep}"
fi
done
All of this can be seen, along with an example using nginx, at: https://github.com/sudo-bmitch/docker-base
The behavior you observe seems fairly normal: in your entrypoint script, you changed the UID associated with the username deploy, but the two whoami commands are still run with the same user (identified by the UID in the first place, not the username).
For more information about UIDs and GIDs in a Docker context, see e.g. that reference.
Note also that using gosu to re-become root is not a standard practice (see in particular that warning in the upstream doc).
For your use case, I'd suggest removing the USER deploy command and switch user in the very end, by adapting your entrypoint script as follows:
Dockerfile
(…)
RUN addgroup -g 1000 deploy \
&& adduser -D -u 1000 -G deploy -s /bin/sh deploy
COPY entrypoint.sh /
ENTRYPOINT ["/entrypoint.sh"]
CMD ["php-fpm7","-F"]
entrypoint.sh
#!/bin/sh
whoami # it outputs `root`
# Change UID of 'deploy' as per host user UID
HOST_CURRENT_USER_ID=$(stat -c "%u" /var/www/${PROJECT_NAME})
if [ ${HOST_CURRENT_USER_ID} -ne 0 ]; then
usermod -u ${HOST_CURRENT_USER_ID} deploy
groupmod -g ${HOST_CURRENT_USER_ID} deploy
fi
# don't forget the "exec" builtin
exec gosu ${HOST_CURRENT_USER_ID}:${HOST_CURRENT_USER_ID} "$#"
this can be tested using id, for example:
$ docker build -t test-gosu .
$ docker run --rm -it test-gosu /bin/sh
$ id

How to check if a Docker image with a specific tag exist locally?

I'd like to find out if a Docker image with a specific tag exists locally. I'm fine by using a bash script if the Docker client cannot do this natively.
Just to provide some hints for a potential bash script the result of running the docker images command returns the following:
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
rabbitmq latest e8e654c05c91 5 weeks ago 143.5 MB
busybox latest 8c2e06607696 6 weeks ago 2.433 MB
rabbitmq 3.4.4 a4fbaad9f996 11 weeks ago 131.5 MB
I usually test the result of docker images -q (as in this script):
if [[ "$(docker images -q myimage:mytag 2> /dev/null)" == "" ]]; then
# do something
fi
But since docker images only takes REPOSITORY as parameter, you would need to grep on tag, without using -q.
docker images takes tags now (docker 1.8+) [REPOSITORY[:TAG]]
The other approach mentioned below is to use docker inspect.
But with docker 17+, the syntax for images is: docker image inspect (on an non-existent image, the exit status will be non-0)
As noted by iTayb in the comments:
The docker images -q method can get really slow on a machine with lots of images. It takes 44s to run on a 6,500 images machine.
The docker image inspect returns immediately.
As noted in the comments by Henry Blyth:
If you use docker image inspect my_image:my_tag, and you want to ignore the output, you can add --format="ignore me" and it will print that literally.
You can also redirect stdout by adding >/dev/null but, if you can't do that in your script, then the format option works cleanly.
Try docker inspect, for example:
$ docker inspect --type=image treeder/hello.rb:nada
Error: No such image: treeder/hello.rb:nada
[]
But now with an image that exists, you'll get a bunch of information, eg:
$ docker inspect --type=image treeder/hello.rb:latest
[
{
"Id": "85c5116a2835521de2c52f10ab5dda0ff002a4a12aa476c141aace9bc67f43ad",
"Parent": "ecf63f5eb5e89e5974875da3998d72abc0d3d0e4ae2354887fffba037b356ad5",
"Comment": "",
"Created": "2015-09-23T22:06:38.86684783Z",
...
}
]
And it's in a nice json format.
tldr:
docker image inspect myimage:mytag
By way of demonstration...
success, found image:
$ docker image pull busybox:latest
latest: Pulling from library/busybox
Digest: sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f
Status: Image is up to date for busybox:latest
$ docker image inspect busybox:latest >/dev/null 2>&1 && echo yes || echo no
yes
failure, missing image:
$ docker image rm busybox:latest
Untagged: busybox:latest
Untagged: busybox#sha256:32f093055929dbc23dec4d03e09dfe971f5973a9ca5cf059cbfb644c206aa83f
$ docker image inspect busybox:latest >/dev/null 2>&1 && echo yes || echo no
no
Reference:
https://docs.docker.com/engine/reference/commandline/image_inspect/
You can use like the following:
[ -n "$(docker images -q someimage:sometag)" ] || echo "does not exist"
Or:
[ -z "$(docker images -q someimage:sometag)" ] || echo "already exists"
With the help of Vonc's answer above I created the following bash script named check.sh:
#!/bin/bash
image_and_tag="$1"
image_and_tag_array=(${image_and_tag//:/ })
if [[ "$(docker images ${image_and_tag_array[0]} | grep ${image_and_tag_array[1]} 2> /dev/null)" != "" ]]; then
echo "exists"
else
echo "doesn't exist"
fi
Using it for an existing image and tag will print exists, for example:
./check.sh rabbitmq:3.4.4
Using it for a non-existing image and tag will print doesn't exist, for example:
./check.sh rabbitmq:3.4.3
In case you are trying to search for a docker image from a docker registry, I guess the easiest way to check if a docker image is present is by using the Docker V2 REST API Tags list service
Example:-
curl $CURLOPTS -H "Authorization: Bearer $token" "https://hub.docker.com:4443/v2/your-repo-name/tags/list"
if the above result returns 200Ok with a list of image tags, then we know that image exists
{"name":"your-repo-name","tags":["1.0.0.1533677221","1.0.0.1533740305","1.0.0.1535659921","1.0.0.1535665433","latest"]}
else if you see something like
{"errors":[{"code":"NAME_UNKNOWN","message":"repository name not known to registry","detail":{"name":"your-repo-name"}}]}
then you know for sure that image doesn't exist.
Using test
if test ! -z "$(docker images -q <name:tag>)"; then
echo "Exist"
fi
or in one line
test ! -z "$(docker images -q <name:tag>)" && echo exist
In bash script I do this to check if image exists by tag :
IMAGE_NAME="mysql:5.6"
if docker image ls -a "$IMAGE_NAME" | grep -Fq "$IMAGE_NAME" 1>/dev/null; then
echo "could found image $IMAGE_NAME..."
fi
Example script above checks if mysql image with 5.6 tag exists. If you want just check if any mysql image exists without specific version then just pass repository name without tag as this :
IMAGE_NAME="mysql"
Inspired by #rubicks's response above.
To check if the image exists already
image_name_tag="alpine:3.13.3"
docker image inspect ${image_name_tag} > /dev/null
echo $?
Pull if image does not exist
docker image inspect ${image_name_tag} > /dev/null || docker pull ${image_name_tag}
Demo execution
# image_name_tag="alpine:3.13.3"
#
# docker image inspect ${image_name_tag} > /dev/null
echo $?
Error: No such image: alpine:3.13.3
# echo $?
1
# docker image inspect ${image_name_tag} > /dev/null || docker pull ${image_name_tag}
Error: No such image: alpine:3.13.3
3.13.3: Pulling from library/alpine
9aae54b2144e: Pull complete
Digest: sha256:826f70e0ac33e99a72cf20fb0571245a8fee52d68cb26d8bc58e53bfa65dcdfa
Status: Downloaded newer image for alpine:3.13.3
docker.io/library/alpine:3.13.3
# docker image inspect ${image_name_tag} > /dev/null || docker pull ${image_name_tag}
$
Just a bit from me to very good readers:
Build
#!/bin/bash -e
docker build -t smpp-gateway smpp
(if [ $(docker ps -a | grep smpp-gateway | cut -d " " -f1) ]; then \
echo $(docker rm -f smpp-gateway); \
else \
echo OK; \
fi;);
docker run --restart always -d --network="host" --name smpp-gateway smpp-gateway:latest
Watch
docker logs --tail 50 --follow --timestamps smpp-gateway
Run
sudo docker exec -it \
$(sudo docker ps | grep "smpp-gateway:latest" | cut -d " " -f1) \
/bin/bash
for specific tag name
$ docker images --filter reference='<REPOSITORY>:TAG'
for "like clause" tagname:my_image_tag --> start my_ima*
$ docker images --filter reference='<REPOSITORY>:my_ima*'
if you want to someting "the image" for example delete all images tag started "my_ima" try this
docker rmi -f $(docker images -q --filter reference='myreponame:my_ima*')
I think this functionality should be implemented inside the docker build command (using a flag?), so that it avoids a lot of code duplication.
I used the same condition as the accepted answer inside a wrapper function called docker_build so that it does the necessary checks before calling the original docker build command.
# Usage: docker_build <...> (instead of docker build)
docker_build()
{
local arguments=("$#")
local index
for (( index=0; index<$#; index++ )); do
case ${arguments[index]} in
--tag)
local tag=${arguments[index+1]}
if [[ ! -z $(docker images -q "${tag}" 2> /dev/null) ]]; then
echo "Image ${tag} already exists."
return
fi
;;
esac
done
command docker build "$#"
}
Disclaimer: This is not ready for production because it works only with space-separated arguments in long format i.e --tag hello-world:latest. Also, this just modifies the docker build command only, all other commands remain same. If anyone has improvements, please let me know.
I felt like sharing this snippet because the idea of wrapping-standard-commands-in-bash-functions, to avoid code repetition, seemed more elegant and scalable than writing wrapper statements.
I like this because it is concise and has the proper semantics. If the image exists it is true what could be easier?
if [ $(docker image ls ${build_env} --format="true") ] ;
then
echo "does exist"
fi

Resources