I'm building a docker image for our specific implementation of an ElasticSearch server. It requires config files. In my Dockerfile, I have the following commands:
RUN mkdir /elasticsearch/plugins
ADD plugins/es_plugins.tar /elasticsearch/plugins
RUN \
cd /elasticsearch/plugins && \
tar -xvf es_plugins.tar && \
rm -f es_plugins.tar
Docker build succeeds on the mkdir, and the ADD. But fails with a tar error "es_plugins.tar: Cannot open: No such file or directory"
This is my first time with Docker. Any ideas why the "ADD" shows successful but the RUN fails?
It took an hour, but I figured out how to run my container and drop into a shell.
Seems that when I do "ADD file.tar /destination/dir" it actually moves the files/directories inside the tar into the destination. Which is what I wanted, but I am surprised. I will have to do more research.
Related
Seems like a basic issue but couldnt find any answers so far ..
When using ADD / COPY in Dockerfile and running the image on linux, the default file permission of the file copied in the image is 644. The onwner of this file seems to be as 'root'
However, when running the image, a non-root user starts the container and any file thus copied with 644 permission cannot execute this copied/added file and if the file is executed at ENTRYPOINT it fails to start with permission denied error.
I read in one of the posts that COPY/ADD after Docker 1.17.0+ allows chown but in my case i dont know who will be the non-root user starting so i cannot set the permission as that user.
I also saw another work around to ADD/COPY files to a different location and use RUN to copy them from the temp location to actual folder like what am doing below. But this approach doesnt work as the final image doesnt have the files in /otp/scm
#Installing Bitbucket and setting variables
WORKDIR /tmp
ADD atlassian-bitbucket-${BITBUCKET_VERSION}.tar.gz .
COPY bbconfigupdater.sh .
#Copying Entrypoint script which will get executed when container starts
WORKDIR /tmp
COPY entrypoint.sh .
RUN ls -lrth /tmp
WORKDIR /opt/scm
RUN pwd && cp /tmp/bbconfigupdater.sh /opt/scm \
&& cp /tmp/entrypoint.sh /opt/scm \
&& cp -r /tmp/atlassian-bitbucket-${BITBUCKET_VERSION} /opt/scm \
&& chgrp -R 0 /opt/ \
&& chmod -R 755 /opt/ \
&& chgrp -R 0 /scm/bitbucket \
&& chmod -R 755 /scm/bitbucket \
&& ls -lrth /opt/scm && ls -lrth /scmdata
Any help is appreciated to figure out how i can get my entrypoint script copied to the desired path with execute permissions set.
The default file permission is whatever the file permission is in your build context from where you copy the file. If you control the source, then it's best to fix the permissions there to avoid a copy-on-write operation. Otherwise, if you cannot guarantee the system building the image will have the execute bit set on the files, a chmod after the copy operation will fix the permission. E.g.
COPY entrypoint.sh .
RUN chmod +x entrypoint.sh
A better option with newer versions of docker (and which didn't exist when this answer was first posted) is to use the --chmod flag (the permissions must be specified in octal at last check):
COPY --chmod=0755 entrypoint.sh .
You do not need to know who will run the container. The user inside the container is typically configured by the image creator (using USER) and doesn't depend on the user running the container from the docker host. When the user runs the container, they send a request to the docker API which does not track the calling user id.
The only time I've seen the host user matter is if you have a host volume and want to avoid permission issues. If that's your scenario, I often start the entrypoint as root, run a script called fix-perms to align the container uid with the host volume uid, and then run gosu to switch from root back to the container user.
A --chmod flag was added to ADD and COPY instructions in Docker CE 20.10. So you can now do.
COPY --chmod=0755 entrypoint.sh .
To be able to use it you need to enable BuildKit.
# enable buildkit for docker
DOCKER_BUILDKIT=1
# enable buildkit for docker-compose
COMPOSE_DOCKER_CLI_BUILD=1
Note: It seems to not be documented at this time, see this issue.
I have built a Docker Cron Environment to run Cronjobs based on alseambusher/crontab-ui using alpine:3.15.3 & it works great.
For it to work I have had to install a number of things via the Dockerfile, editing it & adding python so it could run a python script, perl for another service, openssl so I could use a Self-signed certificate, etc.
As it stands the Container is a lot bigger, which is fine, but if I am to share the container others won't necessarily want or need the services I have added & will likely need other that I haven't.
I would like to be able to add a command in the ENV of a Docker Compose to add services at startup without having to do a full build each time. I'm sure it would be simpler to add build:>args: & have it rebuild the container each startup, but my goal is to have it add to an image only the services that each user needs & declares in the Docker-Compose with no need to have the files for the build on the system.
I know this will mean a longer startup depending on the services, I'm okay with that.
I know it's normal to run cron on the host & have it call into containers, but cron on Windows WSL has to be manually started every time the WSL starts & is easy to forget about & can't really be automated aside from on startup, & I'd like to do this entirely inside Docker.
How can I add an ENV like SERVICE_INSTALL to have it run in BASH (which is already added in the Dockerfile & present at /bin/bash) at container startup?
Ideally I'd like to be able to add multiple SERVICE_INSTALL lines if at all possible.
Example:
SERVICE_INSTALL1='apk add --update --no-cache python3 && ln -sf python3 /usr/bin/python'
SERVICE_INSTALL2='python3 -m ensurepip'
SERVICE_INSTALL3='apk add --no-cache perl perl-html-parser perl-http-cookies perl-lwp-useragent-determined perl-json perl-json-xs'
Or, if nothing else:
SERVICE_INSTALL=apk add --update --no-cache python3 && ln -sf python3 /usr/bin/python && perl perl-html-parser perl-http-cookies perl-lwp-useragent-determined perl-json perl-json-xs && && wget && curl && nodejs && npm
but then that leaves the problem of installing things through pip or npm.
I have tried adding a command: to the Docker-Compose but every variation I have tried does not work. I'm also concerned with this method as from my understanding a command: replaces the startup script in the container, not adds to it, so that is not ideal, regardless, it doesn't seem like an install command: is possible anyway
I have tried: (Each as a single command: not together)
command:
- BASH apk --update add openssl
- /bin/bash apk --update add openssl
- BASH RUN apk --update add openssl
- /bin/bash RUN apk --update add openssl
- sh apk --update add openssl
- /bin/sh apk --update add openssl
- apk --update add openssl
Each ends with a message along the lines of Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "/bin/bash run apk --update add openssl": stat /bin/bash run apk --update add openssl: no such file or directory: unknown
UPDATE: I discovered a few things trying to get this to work
for command: to work there needs to not be any - before it
anything, even on multiple lines, is considered a single command essentially as though they were all on the same line & have to be separated with an &&
it will repeat the command or show the error of it failing to execute the command & not continue to next until it is completed.
for example the command mkdir -p /test leaves no logs, but the container never actually starts. While portainer says it's running trying to bash into it gives a is restarting, wait until the container is running message
mkdir "-p /test" repeats this message
mkdir: unrecognized option:
BusyBox v1.34.1 (2022-02-02 18:21:20 UTC) multi-call binary.
Usage: mkdir [-m MODE] [-p] DIRECTORY...
Create DIRECTORY
-m MODE Mode
-p No error if exists; make parent directories as needed
3 times 3-4 seconds apart, them 7 seconds, then 8 seconds, then 15 seconds, 27 seconds, 53 seconds, then hits a minute & continues to grow a few seconds each try.
It also returns the same wait for the container to be running message when trying to bash in
mkdir -p "/test" seems to be the correct formatting, it appears to work but leaves no logs & when attempting to bash in it connects, shows the terminal, then exits, attempting to reconnect shows the same container is restarting message, likely because the container stopped once the command was finished & is set to restart: always. commenting out the restart command the container exits.
mkdir -p "/test" followed by a new line with supervisord -c /etc/supervisord.conf (the default start command) has mkdir reporting mkdir: unrecognized option: c
adding "supervisord -c /etc/supervisord.conf" leaves no logs & a restarting container.
reversing the order, with supervisord -c /etc/supervisord.conf 1st has supervisord reporting the error Error: positional arguments are not supported: ['mkdir', '-p', '/test'] For help, use /usr/bin/supervisord -h
bash -c "supervisord -c /etc/supervisord.conf with a new line & && mkdir -p /test with a new line & && mkdir -p /test2" runs with a working container, but no directories created
reversing the order seems to work & creates the directories, with a running container
command:
bash -c "mkdir -p /test
&& mkdir -p /test2
&& supervisord -c /etc/supervisord.conf"
Which indicates that it will run them in order, but only proceeds to the next after the one finishes.
a test confirmed that the same can be done with other dependencies so long as the initial startup is last. I'd rather have the container start 1st, then install the dependencies while it is running as they are not required for the container itself to run, but rather are added for use in the cronjobs that will be running on a schedule, so if the container starts & the dependencies cannot be used for the 1st 2, 3, even 5 or 10 minutes that might only affect their 1st attempt if it happens to be in that time.
This is alright, I now understand better how the command: option works, but it still requires users to know & properly include the default start command. The command: options are also a lot more particular & easy to get wrong, while ENV variables are something every docker user knows, has experience with, & is simpler to implement
im running a docker container as shown below, after the sh files are executed the container terminate by default, how i can keep the container working in background
sample from docker file
#=========================
# Copying Scripts to root
#=========================
COPY entry_point.sh \
/root/
RUN chmod +x /root/entry_point.sh
#=======================
# framework entry point
#=======================
CMD /root/entry_point.sh
the entry_point.sh file
function clone_repo() {
mkdir /root/repo
git clone git#github.com:test/tests.git /root/repo && \
rm -rf /root/.ssh
}
clone_repo
and here is the command im using to initialize the container
docker run -p 5900:5900 --name mycontainer--privileged amrka/image
You could also run CMD /root/entry_point.sh; sleep infinity
Thats how docker container works, its goes offf when application dies.
Why you need it to keep alive?
What it will do?
if you want just to have, you can run sleep 10000000 for example at end of the file, so it will keep container, but why?
if you want execute any tas on schedule, then you can use any cron image.
Solved by removing entry_point
I am a biginner with docker, I try to make my transmission container work!
First I am running on a debian 8.1.
To make things work I created this 2 folder:
mkdir -p /opt/docker/transmission/config
mkdir -p /opt/docker/transmission/downloads
after this I added the "good" right:
chown -R root:docker /opt/docker
chmod -R 775 /opt/docker
At the end I tried to create my docker by doing this:
docker run -d \
--net="host" \
--name="Transmission" \
-e USERNAME="root" \
-e PASSWORD="mdp" \
-v /opt/docker/transmission/config:/config \
-v /opt/docker/transmission/downloads:/downloads \
-v /etc/localtime:/etc/localtime:ro \
gfjardim/transmission
The command docker logs transmission gives:
Couldn't save temporary file "/config/resume/myfile.resume.tmp.hYhTPF": No such file or directory
I guessed that the folder resume in config was not created so I created it, but I didn't work.
The message in transmission GUI is:
unable to save resume file: permission denied
I cannot reproduce you errors using your commands. I suggest you delete your /opt/docker/transmission/ directory and then run the docker run command.
Docker will take care of creating those repositories.
I didn't find a solution for gfjardim/transmission container
But by changing container it worked directly.
I followed the dperson/transmission in the hub site, and it work perfectly.
Thanks for your help.
I am experiencing strange behavior when executing a Dockerfile (in https://github.com/Krijger/es-nagios-docker). Basically, I add a file to append its contents to a file in the image
ADD es-command /tmp/
RUN cat tmp/es-command >> /opt/nagios/etc/objects/commands.cfg
The problem is that, while /tmp/es-command is present in the resulting image, the commands.cfg file was not changed.
As a prelude to the accepted answer: my Dockerfile extends cpuguy83/nagios, which defines /opt/nagios/etc as a volume.
Good to the see sample code, which find the route cause.
Your docker image comes from cpuguy83/nagios, from this image https://github.com/cpuguy83/docker-nagios/blob/master/Dockerfile
You can see /opt/nagios/etc directory is set as VOLUME
VOLUME ["/opt/nagios/var", "/opt/nagios/etc", "/opt/nagios/libexec", "/var/log/apache2", "/usr/share/snmp/mibs"]
Then you can notice that docker volume can't be changed at the next commit by your new build.
And this is the reason you can see your changes when you enter into the container and lost it when exits.
Here is how I use it:
ls ./
configure.sh
commands.cfg
cat configure.sh
#!/bin/bash
script_path=$( cd "$( dirname "$0" )" && pwd )
cp ${script_path}/commands.cfg /opt/nagios/etc/objects/
docker run -d --name nagios cpuguy83/nagios
docker run --rm -v $(pwd):/tmp --volumes-from nagios --entrypoint /tmp/configure.sh cpuguy83/nagios