I use docker-compose for local development on my Mac. I have multiple images being built with docker compose. My docker and docker-compose set up is very standard. Now I want to share my locally built image file with someone. Where are these local files stored?
Searching a bit gave me answers like:
~/Library/Containers/com.docker.docker/Data/com.docker.driver.amd64-linux/Docker.qcow2
But then how can I extract one image from this and share? I tried running the tty that is present with it, but to no avail.
Docker Version: 18.03 Docker for Mac
Docker compose Version: 2
If you have docker-hub account (which is free), then you can use docker push command to save docker image into registry and use docker pull to pull on other machine.
Another solution is to use save + import commands.
For that you can use docker save and docker import commands.
docker#default:~$ docker save --help
Usage: docker save [OPTIONS] IMAGE [IMAGE...]
Save one or more images to a tar archive (streamed to STDOUT by default)
Options:
-o, --output string Write to a file, instead of STDOUT
docker#default:~$
After that you have TAR file on your file system (check -o value) then transfer the file to another machine and execute docker import
docker#default:~$ docker import --help
Usage: docker import [OPTIONS] file|URL|- [REPOSITORY[:TAG]]
Import the contents from a tarball to create a filesystem image
Options:
-c, --change list Apply Dockerfile instruction to the created image
-m, --message string Set commit message for imported image
docker#default:~$
Apparently, the solution with docker-compose is using the docker save command. We do not need to know the locations of images as #fly2matrix mentioned. We can use the docker save command to save the image in TAR file.
docker save --output image-name.tar image-name:tag
Then this image can be shared and loaded by other users through:
docker load --input image-name.tar
Related
I'm building a new image and copy contents from host OS folder D:\Programs\scrapy into it like so: docker build . -t scrapy
Dockerfile
FROM mcr.microsoft.com/windows/servercore:ltsc2019
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
RUN mkdir root
RUN cd root
WORKDIR /root
RUN mkdir scrapy
COPY scrapy to /root/scrapy
Now when I add new contents to the host OS folder "D:\Programs\scrapy" I want to also add it to image folder "root/scrapy", but I DON'T want to build a completely new image (it takes quite a while).
So how can I keep the existing image and just overwrite the contents of the image folder "root/scrapy".
Also: I don't want to copy the new contents EACH time I run the container (so NOT at run-time), I just want to have a SEPARATE command to add more files to an existing image and then run a new container based on that image at another time.
I checked here: How to update source code without rebuilding image (but not sure if OP tries to do the same as me)
UPDATE 1
Checking What is the purpose of VOLUME in Dockerfile and docker --volume format for Windows
I tried the commands below, all resulting in error:
docker: Error response from daemon: invalid volume specification: ''. See 'docker run --help'.
Where <pathiused> is for example D:/Programs/scrapy:/root/scrapy
docker run -v //D/Programs/scrapy:/root/scrapy scrapy
docker run -v scrapy:/root/scrapy scrapy
docker run -it -v //D/Programs/scrapy:/root/scrapy scrapy
docker run -it -v scrapy:/root/scrapy scrapy
UPDATE WITH cp command based on #Makariy's feedback
docker images -a gives:
REPOSITORY TAG IMAGE ID CREATED SIZE
scrapy latest e35e03c8cbbd 29 hours ago 5.71GB
<none> <none> 2089ad178feb 29 hours ago 5.71GB
<none> <none> 6162a0bec2fc 29 hours ago 5.7GB
<none> <none> 116a0c593544 29 hours ago 5.7GB
mcr.microsoft.com/windows/servercore ltsc2019 d1724c2d9a84 5 weeks ago 5.7GB
I run docker run -it scrapy and then docker container ls which gives:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1fcda458a14c scrapy "c:\\windows\\system32…" About a minute ago Up About a minute thirsty_bassi
If I run docker cp D:\Programs\scrapy scrapy:/root/scrapy I get:
Error: No such container:path: scrapy:\root
So in a separate PowerShell instance I then run docker cp D:\Programs\scrapy thirsty_bassi:/root/scrapy whichs show no output in PowerShell whatsoever, so I think it should've done something.
But then in my container instance when I goto /root/scrapy folder I only see the files that were already added when the image was built, not the new ones I wanted to add.
Also, I think I'm adding files to the container here, but is there no way to add it to the image instead? Without rebuilding the whole image?
UPDATE 2
My folder structure:
D:\Programs
Dockerfile
\image_addons
Dockerfile
\scrapy
PS D:\Programs>docker build . -t scrapybase
Successfully built 95676d084e28
Successfully tagged scrapybase:latest
PS D:\Programs\image_addons> docker build -t scrapy .
Step 2/2 : COPY scrapy to /root/scrapy
COPY failed: file not found in build context or excluded by .dockerignore: stat to: file does not exist
Dockerfile A
FROM mcr.microsoft.com/windows/servercore:ltsc2019
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
WORKDIR /root/scrapy
Dockerfile B
FROM scrapybase
COPY scrapy to /root/scrapy
You also can use docker cp, to manually copy files from your host to running container
docker cp ./path/to/file containername:/another/path
Docs
answer if you want it quick and dirty
docker run -it -v c:/programs/test:/root/test ubuntu:latest cat /root/test/myTestFile.txt
to update one file quickly:
If you don't have to build your code (I don't know what language you are using) you can build some base image with the initial code and when you want to change only one file (again I'm assuming you don't need to compile your project again for that, otherwise if you do that is not possible to due the nature of compiled programming language):
FROM previous-version-image:latest
COPY myfile dest/to/file
then because your CMD and ENTRYPOINT are saved from the previous stages no need to declare them. (if you don't remember use docker history <docker-image-name> to view virtual dockerfile for image to this stage).
Notice though not to repetitively use this method or you'll get a very big image with many useless layers. Use this only for quick testing and debugging.
explanation
Usually people use it for frontend development on docker containers but the basic idea persists, you create the basic working image with the dependencies installed and the directory layout setup with the last Dockerfile command being the development server start command.
example:
Dockerfile:
# pull the base image
FROM node:slim
# set the working directory
WORKDIR /app
# add `/app/node_modules/.bin` to $PATH
ENV PATH /app/node_modules/.bin:$PATH
# copy dependencies files
COPY package.json ./
COPY package-lock.json ./
# install app dependencies
RUN npm install
# add app
COPY . ./
# start development server
CMD ["npm", "start"]
startup command:
docker run -it --rm \
-v ${PWD}:/app \ <mount current working directory in host to container in path /app>
-v /app/node_modules \ <or other dependency directory if exists>
-p 80:3000 \ <ports if needs exposing>
ps-container:dev
I'm not sure if that use case will 100% work for you because it needs the code to be mounted using bind-mount all the time and when needed to be exported will have to be exported as the image and the source code directory, on the other hand, it allows you to make quick changes without waiting for the image to be built each time you add something new and in the end build the final image that contains all that's needed.
more relatable example to question provided code:
As you can see there is a file on the host machine that contains some text
the command that uses bind-mount to have access to the file:
docker run -it -v c:/programs/test:/root/test ubuntu:latest cat /root/test/myTestFile.txt
hope you find something that works for you from what I've provided here.
thanks to this tutorial and this example for starting examples and information.
EDIT:
Let's say your original Dockerfile looks like this:
FROM python:latest
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD python /app/app.py
This will build your initial image on top of we'll add layers and change the python files.
The next Dockerfile we'd use (let's call it Dockerfile.fix file) would copy the file we want to change instead of the ones already in the image
FROM previous-image-name
COPY app.py .
Now with after building with this Dockerfile the final image Dockerfile would look (sort of) like so:
FROM python:latest
WORKDIR /app
COPY . .
RUN pip install -r requirements.txt
CMD python /app/app.py
FROM previous-image-name
COPY app.py .
And each time we'll want to change the file we'll use the second Dockerfile
There's no way you can change a Docker image without (at least partially) rebuilding it. But you don't have to rebuild all of it, you can just rebuild the layer copying your scrapy content.
You can optimize your build to have two images:
First image is your static image you don't want to rebuild each time. Let's call it scrapy-base.
Second and final image is based on first image scrapy-base and will only exist for the purpose of copying your dynamic scrapy content
scrapy-base's Dockerfile:
FROM mcr.microsoft.com/windows/servercore:ltsc2019
SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
RUN mkdir root
RUN cd root
WORKDIR /root
RUN mkdir scrapy
And build it like:
docker build -t scrapy-base .
This command only needs to be run once. You won't have to build this image if you only change the content of local scrapy folder. (as you can see, the build does not use it at all)
scrapy's Dockerfile:
FROM scrapy-base
COPY scrapy /root/scrapy
With build command:
docker build -t scrapy .
This second build command will re-use the previous static image and only copy content without having to rebuild the entire image. Even with lots of files it should be pretty quick. You don't need to have a running container.
For your scenario :
docker run -v D:/test:/root/test your-image
A lots of valuable details available in this thread
I am quite new to docker technology and still learning and reading through the docs. I have an oracle base image which i would like to use as a parent image to build my own image and then pushing it towards custom docker registry/repository.
The base image already provides a full setup of oracle db. But as next steps, i would like
download a dump file (e.g. dump url) directly into the docker image (without downloading to local
workspace)
run some sql script
lastly, import the dump using data pump (impdp)
I tried to follow https://github.com/mpern/oracle-docker, but here you always need to store dump file locally and point it as volume.
Is it possible if i can use curl command to download and directly store in oracle docker container workspace? Afterwards importing it from there
You can run an interactive bash session inside your container to check if curl is installed, and if it is not installed then you need to install Curl. Using an interactive bash session, you can then download your dump file.
The ports you require will also need to be be published, if the container is connecting outside of Docker and the host machine, you can use docker run with the -p parameter.
An example is below,
docker run -p 80:80 -it (Your image) /bin/bash
More information on this regarding the docker run command, and Dockerfiles
https://docs.docker.com/engine/reference/commandline/run/
https://docs.docker.com/engine/reference/builder/
I have an image that I want to run on my local machine. I took this image from my friend and not from docker-hub or any repository. The file shared is in ".img" format.
I am able to import this image on docker but unable to run.
What I did:
Compress the image file from ".img" format to ".tar.gz" format so that the docker image can be imported. I used 7-zip tool to convert this.
From my local I imported the docker image using this new file(.tar.gz)<
Trying to run this imported image but fails.
Commands Executed:
PS C:\Users\C61464> docker import .\Desktop\regchange.tar.gz
sha256:a0008215897dd1a7db205c191edd0892f484d230d8925fd09e79d8878afa2743
PS C:\Users\C61464> docker images
REPOSITORY TAG IMAGE ID CREATED
SIZE
<none> <none> 7fdbbdcc59c4 2 minutes ago 1.05GB
PS C:\Users\C61464> docker tag 7fdbbdcc59c4 bwise:version1.0
PS C:\Users\C61464> docker images
REPOSITORY TAG IMAGE ID CREATED
SIZE
bwise version1.0 7fdbbdcc59c4 3 minutes ago 1.05GB
PS C:\Users\C61464> docker run -p 8888:80 bwise:version1.0
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: No command specified.
See 'C:\Program Files\Docker\Docker\Resources\bin\docker.exe run --help'.
I searched a lot for this error and found that for running we need to specify the path used while creating the image(In Dockerfile) but I am not sure as I am new to docker. Am I doing something wrong or I need to have the docker file to run this image?
Perhaps the Docker Image you have had no CMD or ENTRYPOINT defined when it was built, so the docker daemon doesn't know what to do with the image
Try doing
docker run -it -p 8888:80 bwise:version1.0 sh
(if it's a *nix based image). That should start an interactive shell.
You can do:
docker run -p 8888:80 bwise:version1.0 {command_you_want_to_run}
On the image when starting it.
The docker image may be broken.
Look inside. See suggestions how to in How to see docker image contents
Run this command to inspect your image
docker inspect [docker-image-name]
Inspect you will see base image and other info about that image
I have installed Docker for Windows. I have downloaded HDP_2.5_docker.tar from http://hortonworks.com/downloads/#sandbox which is a 10 GB file.
How can I load an image tar file? I have tried this command:
docker import HDP_2.5_docker.tar
You can use docker load
Usage: docker load [OPTIONS]
Load an image from a tar archive or STDIN
Git bash console:
docker load < HDP_2.5_docker.tar
Windows cmd:
docker load -i windowsservercore.tar
Firstly, put the tar file under your user folder: i.e: C:\Users\yourName\xxx.tar
Secondly, run the Docker load CMD:
docker load -i xxx.tar
After it is done, we could see the file is loaded as Docker images by running CMD:
docker images
you can do:
docker image import file.tar images_name:image_tag
Load the desired docker file, assuming you are in the same directory as the tar file, you can use -
$ docker load -i filename.tar
On successful import, you will see a success message along with the image ID
Check in the docker images for the image ID that you just received:
docker images
You will see the docker loaded successfully in the docker images list. However, there is one thing worth mentioning in case you might get confused; the date reflected in the command output might reflect the date when docker is created. Assuming, docker got created 5 days ago then the same will be shown in the output. Better way to confirm if your docker is loaded or not is to check for the image ID or repo and tag name (if you know).
You can finally run the docker using the command -
$ docker run -it image-ID
I'm new to Docker, so please allow me to describe the steps that I did. I'm using Docker (not Docker toolbox) on OS X. I built the image from Dockerfile using the following command
sudo docker build -t myImage .
Docker confirmed that building was successful.
Successfully built 7240e.....
However, I can't find the image anywhere. I looked at this question, but the answer is for Docker toolbox, and I don't have a folder /Users/<username>/.docker as suggested by the accepted answer.
You would be able to see your docker images by the below command:
docker images
And to check which all containers are running in docker:
docker ps -a
Local builds (in my case using buildkit) will create and cache the image layers but simply leave them in the cache rather than tell the docker daemon they're an actual image. To do that you need to use the --load flag.
$ docker buildx build -t myImage .
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
Doesn't show anything, but...
$ docker buildx build -t myImage --load .
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
myImage latest 538021e3d342 18 minutes ago 190MB
And there it is!
There actually is a warning about this in the output of the build command... but it's above all the build step logs so vanishes off your terminal without easily being seen.
To get list of Images
docker image ls
or
docker images
In addition to the correct responses above that discuss how to access your container or container image, if you want to know how the image is written to disk...
Docker uses a Copy on Write File System (https://en.wikipedia.org/wiki/Copy-on-write) and stores each Docker image as a series of read only layers and stores them in a list. The link below does a good job explaining how the image layers are actually stored on disk.
https://docs.docker.com/storage/storagedriver/
As already said, after the docker images
this command will show you all the images you have locally.
i.e "somth like that"
REPOSITORY TAG IMAGE ID CREATED SIZE
codestandars 1.0 a22daacf6761 8 minutes ago 622MB
bulletinboard 1.0 b73e8e68edc0 2 hours ago 681MB
ubuntu 18.04 cf0f3ca922e0 4 days ago 64.2MB
now you should
docker run -it and the IMAGE ID or the TAG related to the repository you want to run.
Command to list the docker images is :
docker images
The default docker images will show all top level images, their repository and tags, and their size.
An image will be listed more than once if it has multiple repository names or tags.
Click here for the screenshot for more details