I want to compress (and later send with rsync to other server) my last backup Docker images for automated.
I try this:
sudo docker save -o dockdebian.tar.gz | sudo docker images | awk 'NR==2{ print $3 }'
Select the the first ID of the image list.
But gave me this error:
"docker save" requires at least 1 argument.
See 'docker save --help'.
Usage: docker save [OPTIONS] IMAGE [IMAGE...]
Save one or more images to a tar archive (streamed to STDOUT by default)
xxxxxxxxxxxx
With regards to the compression part, you can omit the -o so docker save writes to stdout then pipe that into the compressor of your choice. Eg:
docker save my-image:latest | xz > my-image.tar.xz
I'm not sure the intent with your sudo docker images | awk 'NR==2{ print $3 }' bit, for me (as an SRE) that ID could be any number of images as I'm constantly shuffling between projects. Even if you only ever build one image you should probably specify the image tag intentionally to prevent issues down the line. Also, if you're finding you need sudo with docker, you may want to peruse this page: https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user
You have to use command substitution in shell. More info here: https://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html
docker save -o dockdebian.tar $(sudo docker images | awk 'NR==2{ print $3 }')
or
docker save -o dockdebian.tar `sudo docker images | awk 'NR==2{ print $3 }'`
Related
It has been mentioned that when employing x86_64 Docker images on an M1 Mac, when no ARM64 image is available, that container will start under qemu emulation for compatibility. (At the cost of performance.)
Often times when I'm running a collection of containers (and integration tests against the lot), I'll see qemu-system-aarch64 pegging a few cores.
My question: How can I determine, for a given list of running containers (ie. docker ps), which ones are running natively and which are being emulated?
This is true also for Docker running on amd64 CPU, when the image is build for arm64, and the whole mechanism is explained in this SO
The mechanism for emulation is using the information in the elf to recognize the architecture for which the process is build, and if the architecture of the binary differs from the architecture of the CPU, it starts the qemu emulation. Though the recognizing of the architecture is more related to the process, there is still information about the targeted architecture of the docker image. The targeted architecture is determined from the "Architecture" flag on the image which was set when the image was build. Any of the containers that will run the image will be associated (trough the image) with this flag.
It should be noted that the "Architecture" flag on the image will not prevent a single process inside the image, which is compiled for a different architecture than the flagged one to run. The reason for this is that bitfmt (which is the underlying mechanism sitting inside the linux kernel) will always try to recognize the architecture from the magic numbers of the elf and will start the the emulation if the magic number is recognized.
To list the architecture of the containers, you can use the following "quick" query:
for i in `docker ps --format "{{.Image}}"` ; do docker image inspect $i --format "$i -> {{.Architecture}} : {{.Os}}" ;done
The command will print the container name, the architecture and the os of the image.
To avoid typing this command multiple times, you can add alias in .bashrc as follows:
alias docker-arch-ps='for i in `docker ps --format "{{.Image}}"` ; do docker image inspect $i --format "$i -> {{.Architecture}} : {{.Os}}" ;done';
After this, you can use simple docker-arch-ps to get the list of the containers and their architecture.
As an improvement of the #jordanvrtanoski's answer, I've done two additional commands
docker-ps-arch:
#!/bin/bash
OPT=$#
set -euo pipefail
docker container ls $OPT --format "{{.ID}}\t{{.Image}}\t{{.Command}}\t{{.Status}}\t{{.Names}}" |
awk -F '\t' 'BEGIN {OFS=FS} { "docker image inspect --format \"{{.Os}}/{{.Architecture}}\" "$2" #"NR | getline $6; print }' |
column --table --table-columns "CONTAINER ID,IMAGE,COMMAND,STATUS,NAME,ARCH" -o ' ' -s $'\t'
and
docker-images-arch:
#!/bin/bash
OPT=$#
set -euo pipefail
docker image ls $OPT --format "{{.Repository}}\t{{.Tag}}\t{{.ID}}\t{{.Size}}" |
awk -F '\t' 'BEGIN {OFS=FS} { "docker image inspect --format \"{{.Os}}/{{.Architecture}}\" "$3" #"NR | getline $5; print }' |
column --table --table-columns "REPOSITORY,TAG,IMAGE ID,SIZE,ARCH" -o ' ' -s $'\t'
They produce outputs close to the original commands and support options of docker container ls and docker image ls.
$ docker-ps-arch -a
CONTAINER ID IMAGE COMMAND STATUS NAME ARCH
261767e38db2 hello-world "/hello" Exited (0) About an hour ago practical_moore linux/amd64
16e364572d08 18e5af790473 "/hello" Exited (0) 3 hours ago peaceful_lalande linux/arm64
PS: the column command used here is the one from util-linux not the one from BSD utils. util-linux is a standard package distributed by the Linux Kernel Organization. On macOS, to get it, use brew install util-linux; rockylinux uses it by default and unfortunately on Debian/Ubuntu, the opposite choice has been done (cf https://askubuntu.com/q/1098248).
I have following bash script code try to run a docker container through bash script but I am retreiving error.
#!/bin/bash
name=sudo docker ps | grep 'test' | awk '{print
$1}'
sudo docker exec -it $name bash
error:
docker exec requires at least two arguments
Assuming that you have a docker container actually called test running, the way the id is attained is incorrect. In order to expand a command into a variable, it needs to be contained within $() and so:
#!/bin/bash
name=$(docker ps | grep 'test' | awk '{print $1}')
sudo docker exec -it $name bash
A further note is the fact that you don't need sudo permissions to run docker ps ...
Additionally, you never need to pipe grep into awk as awk can do this for you:
name=$(docker ps | awk '/test/ {print $1}')
Further more, there is no need to pipe at all given the native capabilities built into docker-ps and so:
name=$(docker ps -q -f name=test)
This will print only the container id of a container named test. I'm assuming that the name is test here but it could be something else i.e. the label that is named test, in which case the filter would need to change:
-f, --filter=[]
Filter output based on these conditions:
- exited=<int> an exit code of <int>
- label=<key> or label=<key>=<value>
- status=(created|restarting|running|paused|exited|dead)
- name=<string> a container's name
- id=<ID> a container's ID
- before=(<container-name>|<container-id>)
- since=(<container-name>|<container-id>)
- ancestor=(<image-name>[:tag]|<image-id>| â¨image#digestâ©) - conâ
tainers created from an image or a descendant.
- volume=(<volume-name>|<mount-point-destination>)
- network=(<network-name>|<network-id>) - containers connected to
the provided network
I'm new to dockers and other concepts. I've lots of unused dockers running and I would want to remove all of them. Now I'm manually killing it. Can we do it at once?
docker rm $(docker ps -a -q)
This command deletes all stopped containers. The command docker ps -a -q above returns all existing container IDs and passes them to the docker rm command which deletes them. Running containers are not deleted.
OR, try this one.
docker system prune -a
Remove all unused images not just dangling ones. also it will remove all build cache
As #Emon mentioned you can either go for docker prune. But keep in mind that pruning the system will remove any stopped containers and all unused images.
Since you've tagged shell, I'm adding this shell script also.
echo -e "******* Start *******"
echo -e "\n=====> Stopping & Cleaning up UNUSED containers.."
for i in `docker ps -a | grep -v CONTAINER | awk '{print $1}'`; do docker stop $i ; docker rm $i ; done
echo -e "[CLEARED]\n"
docker ps -a
echo -e "\n=====> Removing UNUSED container images.."
for i in `docker images | grep none | awk '{print $3}'`; do docker rmi $i ; done
echo -e "[CLEARED]\n"
docker images
This script can be changed accordingly to your need like if you want to remove only containers or even images also.
This can further also be done easily by
docker ps -a | grep "pattern" | awk '{print $1}' | xargs docker rm
If you want to remove only the exitted containers alone, you can use it
`docker rm $(docker ps -a -f status=exited -q)`
If I ran sudo doccker ps I got this
[user#vm1 ~]$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e8ff73dec1d5 portal-mhn:latest "nginx -g 'daemon of…" 43 minutes ago Up 43 minutes portal-mhn_portal-mhn.1.4rsfv94wy97gb333q3kfyxz32
62a7cf09d7bf portal-admin:latest "nginx -g 'daemon of…" 43 minutes ago Up 43 minutes portal-admin_portal-admin.1.s62iep4gl5g5oj2hrap14kz1t
I'm trying to grab the container ID base on ImageName.
Ex. Is there away to grab the container id of portal-mhn:latest via a command line ? which is e8ff73dec1d5
If you want to get the container id based on the image name this should work:
$ docker ps | grep '<image_name>' | awk '{ print $1 }'
Or even:
$ docker ps | awk '/<image_name>/ { print $1 }'
As others have suggested you can also directly filter by the image name using the ancestor filter:
$ docker ps -aqf "ancestor=<image_name>"
Thanks to #kevin-cui and #yu-chen.
The accepted answer works, but you might possibly have a misnamed container name that has postgres in its name but is actually running a totally different image, since the answer only uses grep to look for matching lines.
You can use Docker's built in filter flag:
docker ps --filter "ancestor=postgres" -q
as an alternative. The -q flag indicates to only return the container ID (quiet mode).
I needed only to obtain the latest running container id by image name, including stopped containers:
docker ps -a | grep 'django-content-services:' -m 1 | awk '{ print $1 }'
In docker -a to include all containers (even stopped). In grep, -m so grep only matches the first case. Cheers!
I need to dynamically delete all docker images in a server, except for the postgres image and container.
Now I need a dynamic way to get the id of that docker image so i will know to avoid it, using:
docker rmi $(docker images -q | grep -v $<id_of_postgres_container>)
For the container part, i managed to find this:
docker ps -aqf "name=postgres"
which returns only the id of the postgres container. Is there any way to do the same with images without getting too deep into bash scripting?
or any better suggestions?
docker images --format="{{.Repository}} {{.ID}}" |
grep "^postgres " |
cut -d' ' -f2
Get docker images in the format repository<space>id, then filter lines starting with postgres<space>, then leave only id.
docker images --format="{{.Repository}} {{.ID}}" |
grep "^postgres " |
cut -d' ' -f2 |
xargs docker rmi
But, if the postgres container and image is currently running or used, you can just:
docker system prune --force --all
You can just use:
$ docker images -q [image_name]
Where image_name can contain tags (appended after :), registry username with / (if applicable), etc.
The image has to be downloaded for this to work, for example:
$ docker pull hello-world
...
Status: Downloaded newer image for hello-world:latest
docker.io/library/hello-world:latest
$ docker images -q hello-world
d1165f221234
$ docker images -q hello-world:latest
d1165f221234
If the image is not locally available, the only alternative I can think of is to manually query the registry, e.g. like this.
docker rmi will never delete an image that corresponds to a running container. So if you have a container based on postgres running, and you want to delete every other image on your system, the age-old incantations will do what you want; I’m too old-school for docker system but the “get all of the image IDs, then try to delete them all” I know is
docker images -q | xargs docker rmi
Which will print out some errors, but will delete all of the images it can.