I have some few libraries that I have compiled on my machine. So I want to copy all the binaries into my docker container and at first, I tried to use COPY and ADD command in my Dockerfile:
# Installing zeromq
WORKDIR /${home}/${user}/master-wheel
COPY ${PWD}/libzmq ./libzmq
COPY ${PWD}/cppzmq ./cppzmq
WORKDIR /${home}/${user}/master-wheel/libzmq/binaries
ADD * /
WORKDIR /${home}/${user}/master-wheel/cppzmq/binaries
ADD * /
Note that the directories and files do exist and upon entering into my created container, I can see that the copied directories libzmq and cppzmq do exist and I can manually copy all the binaries to the root /. However, for some reason Dockerfile doesn't copy and I can't figure out what could be the problem.
Then, I have decided to do that inside my ENTRYPOINT script and it looks like this:
#!/bin/bash
#set -e
#set -u
echo "==> Executing master image entrypoint ..."
echo "-> Setting up"
cp -r /home/ed/master-wheel/libzmq/binaries/* /
cp -r /home/ed/master-wheel/cppzmq/binaries/* /
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
ldconfig
echo "==> Container ready"
exec "$#"
Everything except two cp commands executes. I tried the same command (cp command) from my container bash terminal and it worked.
What could be the problem?
EDIT:
This part of the file works and the binaries do really get copied to the root directory:
# Installing libfreespace
WORKDIR /${home}/${user}/master-wheel
COPY ${PWD}/libfreespace ./libfreespace
WORKDIR /${home}/${user}/master-wheel/libfreespace/binaries
COPY * /
EDIT 2:
It seems that if I do something like this:
WORKDIR /${home}/${user}/master-wheel
COPY ${PWD}/libzmq ./libzmq
COPY ${PWD}/cppzmq ./cppzmq
WORKDIR /${home}/${user}/master-wheel/libzmq/binaries/usr/
ADD * /usr/
WORKDIR /${home}/${user}/master-wheel/cppzmq/binaries/usr/
ADD * /usr/
It works.
Related
Seems like a basic issue but couldnt find any answers so far ..
When using ADD / COPY in Dockerfile and running the image on linux, the default file permission of the file copied in the image is 644. The onwner of this file seems to be as 'root'
However, when running the image, a non-root user starts the container and any file thus copied with 644 permission cannot execute this copied/added file and if the file is executed at ENTRYPOINT it fails to start with permission denied error.
I read in one of the posts that COPY/ADD after Docker 1.17.0+ allows chown but in my case i dont know who will be the non-root user starting so i cannot set the permission as that user.
I also saw another work around to ADD/COPY files to a different location and use RUN to copy them from the temp location to actual folder like what am doing below. But this approach doesnt work as the final image doesnt have the files in /otp/scm
#Installing Bitbucket and setting variables
WORKDIR /tmp
ADD atlassian-bitbucket-${BITBUCKET_VERSION}.tar.gz .
COPY bbconfigupdater.sh .
#Copying Entrypoint script which will get executed when container starts
WORKDIR /tmp
COPY entrypoint.sh .
RUN ls -lrth /tmp
WORKDIR /opt/scm
RUN pwd && cp /tmp/bbconfigupdater.sh /opt/scm \
&& cp /tmp/entrypoint.sh /opt/scm \
&& cp -r /tmp/atlassian-bitbucket-${BITBUCKET_VERSION} /opt/scm \
&& chgrp -R 0 /opt/ \
&& chmod -R 755 /opt/ \
&& chgrp -R 0 /scm/bitbucket \
&& chmod -R 755 /scm/bitbucket \
&& ls -lrth /opt/scm && ls -lrth /scmdata
Any help is appreciated to figure out how i can get my entrypoint script copied to the desired path with execute permissions set.
The default file permission is whatever the file permission is in your build context from where you copy the file. If you control the source, then it's best to fix the permissions there to avoid a copy-on-write operation. Otherwise, if you cannot guarantee the system building the image will have the execute bit set on the files, a chmod after the copy operation will fix the permission. E.g.
COPY entrypoint.sh .
RUN chmod +x entrypoint.sh
A better option with newer versions of docker (and which didn't exist when this answer was first posted) is to use the --chmod flag (the permissions must be specified in octal at last check):
COPY --chmod=0755 entrypoint.sh .
You do not need to know who will run the container. The user inside the container is typically configured by the image creator (using USER) and doesn't depend on the user running the container from the docker host. When the user runs the container, they send a request to the docker API which does not track the calling user id.
The only time I've seen the host user matter is if you have a host volume and want to avoid permission issues. If that's your scenario, I often start the entrypoint as root, run a script called fix-perms to align the container uid with the host volume uid, and then run gosu to switch from root back to the container user.
A --chmod flag was added to ADD and COPY instructions in Docker CE 20.10. So you can now do.
COPY --chmod=0755 entrypoint.sh .
To be able to use it you need to enable BuildKit.
# enable buildkit for docker
DOCKER_BUILDKIT=1
# enable buildkit for docker-compose
COMPOSE_DOCKER_CLI_BUILD=1
Note: It seems to not be documented at this time, see this issue.
Im trying to execute bash file called start.bash by CMD["bash" ,"start.bash"] in Dockerfile. When I create the image this command is not executed for some reason even though the bash file was of course copied properly in the dockerfile.
The point is that while im trying to run this command inside the container itself its success.
Here is my Dockerfile:
# build back end
FROM node:12.22.12 AS server_build
###ENV HOSTNAME myhost
WORKDIR /VideoServiceApp
COPY ./projConf.json /VideoServiceApp
COPY ./projVideoApp.json /VideoServiceApp
COPY ./front/UIVideo ./front/UIVideo
COPY ./front/videoService ./front/videoService
COPY --from=client_build /VideoServiceApp/front/video/dist/video /VideoServiceApp/front/video/dist/video
COPY ./start.bash /VideoServiceApp
COPY ./classes ./classes
WORKDIR /VideoServiceApp/front/UIVideo
RUN npm install
WORKDIR /VideoServiceApp/front/videoService
RUN npm install && npm install -g typescript#latest
RUN tsc
EXPOSE 7717 7708
WORKDIR /VideoServiceApp
CMD ["bash" , "start.bash"]
You need to specify the full path for the command when you are using the CMD array's syntax.
CMD ["/bin/bash", "start.bash"]
You could also switch to the shell form of CMD
CMD bash start.bash
For more information read the CMD-Documentation
Following the instructions as outlined to deploy Duo CloudMapper to AWS environment and getting an error
Docker File
FROM python:3.7-slim as cloudmapper
LABEL maintainer="https://github.com/0xdabbad00/"
LABEL Project="https://github.com/duo-labs/cloudmapper"
WORKDIR /opt/cloudmapper
ENV AWS_DEFAULT_REGION=us-east-1
RUN apt-get update -y
RUN apt-get install -y build-essential autoconf automake libtool python3.7-dev python3-tk jq awscli
COPY cloudmapper/. /opt/cloudmapper
COPY entrypoint.sh /opt/cloudmapper/entrypoint.sh
# Remove the demo data
RUN rm -rf /opt/cloudmapper/account-data/demo
# Install the python libraries needed for CloudMapper
RUN cd /opt/cloudmapper && pip install -r requirements.txt
ENTRYPOINT /opt/cloudmapper/entrypoint.sh
Now building the docker image
C:\> docker build -t cloudmapper .
When I run the docker using the below command I get an error
C:/> docker run -t cloudmapper
Error
/bin/sh: 1: /opt/cloudmapper/entrypoint.sh: not found
Verified that the file exists in the appropriate location
Using Docker on Windows 10
Image in the dockerfile is python:3.7-slim
Assuming the images are removed and replaced with text and the question doesn't get closed.
bash can return "file not found" when
the entrypoint shell script is not marked executable for the current user
the hash bang in the entrypoint shell script points to a binary that does not exist
the shell script actually does not exist.
You can fix the first problem by ensuring you use the new --chmod flag to ensure the executable bit is set. Even if the user is root it is necessary that there is at least 1 executable bit set.
COPY --chmod=0755 *.sh /opt/cloudmapper/
ENTRYPOINT ["/opt/cloudmapper/entrypoint.sh"]
ps. This integrated COPY --chmod only works with buildkit enabled builds, so you might need to force buildkit, or split the chmod into a separate explicit RUN step.
The 2nd issue can be dealt with by ensuring the first line of entrypoint.sh uses sh rather than bash if you are using a lightweight base image like alpine:
#!/bin/sh
set -e
# etc
Also, if on Windows especially, ensure ALL files, especially the entrypoint .sh file, are set to utf-8 encoding with lf style line endings. As linux doesn't understand the cr, it will try to execute /bin/sh<cr> as the shell which clearly doesn't exist.
In terms of the file not existing, verify the entrypoint.sh is being copied into a location that is referenced by env.PATH, or that the entry point directive uses a fully qualified path.
--
edited to add cr-lf revelation.
In terms of the way Docker works, is there any difference between RUN cd / and WORKDIR / inside a Dockerfile?
RUN cd / does absolutely nothing. WORKDIR / changes the working directory for future commands.
Each RUN command runs in a new shell and a new environment (and technically a new container, though you won't usually notice this). The ENV and WORKDIR directives before it affect how it starts up. If you have a RUN step that just changes directories, that will get lost when the shell exits, and the next step will start in the most recent WORKDIR of the image.
FROM busybox
WORKDIR /tmp
RUN pwd # /tmp
RUN cd / # no effect, resets after end of RUN line
RUN pwd # still /tmp
WORKDIR /
RUN pwd # /
RUN cd /tmp && pwd # /tmp
RUN pwd # /
(For the same reason, RUN export doesn't do anything that outlives the current Dockerfile instructions, and RUN . or the non-standard RUN source won't cause environment variables to be set.)
I am building a Dockerfile for an application. I want to execute a bash script with parameters when the container starts to run, so I have made it an entry point. However, Docker cannot find the directory in which my script is located. Thi script is located in the Intellij Idea project folder and the path practically looks like this: /home/user/Documents/folder1/folder2/folder3/Projectname/runapp.sh
I have tried to mount this directory as volume, but while running built image an error occurred:
docker: Error response from daemon: OCI runtime create failed: container_linux.go:348: starting container process caused "exec: \"runapp.sh\": executable file not found in $PATH": unknown.
What may be the reason of such behavior? How else can I reach this bash script from Dockerfile?
Here is how the Dockerfile looks like:
FROM java:8
ENV SCALA_VERSION 2.11.8
ENV SBT_VERSION 1.1.1
ENV SPARK_VERSION 2.2.0
ENV SPARK_DIST spark-$SPARK_VERSION-bin-hadoop2.6
ENV SPARK_ARCH $SPARK_DIST.tgz
ENV NEO4J_CONFIG default
ENV BENCHMARK_NAME default
WORKDIR /opt
# Install Scala
RUN \
cd /root && \
curl -o scala-$SCALA_VERSION.tgz http://downloads.typesafe.com/scala/$SCALA_VERSION/scala-$SCALA_VERSION.tgz && \
tar -xf scala-$SCALA_VERSION.tgz && \
rm scala-$SCALA_VERSION.tgz && \
echo >> /root/.bashrc && \
echo 'export PATH=~/scala-$SCALA_VERSION/bin:$PATH' >> /root/.bashrc
# Install SBT
RUN \
curl -L -o sbt-$SBT_VERSION.deb https://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \
dpkg -i sbt-$SBT_VERSION.deb && \
rm sbt-$SBT_VERSION.deb
# Install Spark
RUN \
cd /opt && \
curl -o $SPARK_ARCH http://d3kbcqa49mib13.cloudfront.net/$SPARK_ARCH && \
tar xvfz $SPARK_ARCH && \
rm $SPARK_ARCH && \
echo 'export PATH=$SPARK_DIST/bin:$PATH' >> /root/.bashrc
EXPOSE 9851 9852 4040 7474 7687 7473
VOLUME /home/user/Documents/folder1/folder2/folder3/Projectname /workdir1
WORKDIR /workdir1
ENTRYPOINT ["runapp.sh"]
CMD ["$NEO4J_CONFIG", "$BENCHMARK_NAME"]
You misunderstood volumes in Docker I think. (see What is the purpose of VOLUME in Dockerfile)
I'm citing #VonC answer:
A volume is a persistent data stored in /var/lib/docker/volumes/...
You can either declare it in a Dockerfile, which means each time a container is stated from the image, the volume is created (empty), even if you don't have any -v option.
You can declare it on runtime docker run -v [host-dir:]container-dir.
combining the two (VOLUME + docker run -v) means that you can mount the content of a host folder into your volume persisted by the container in /var/lib/docker/volumes/....
docker volume create creates a volume without having to define a Dockerfile and build an image and run a container. It is used to quickly allow other containers to mount said volume.
So you should use docker run -v /home/user/Documents/folder1/folder2/folder3/Projectname:/workdir1 when starting the container
And your Dockerfile volume declaration should be:
VOLUME /workdir1
That being said, you define both Entrypoint and CMD. What is the CMD being for ? You will never use your image without using runapp.sh ? I prefer using only CMD for development since you can still do docker run -it my_container bash for debugging purpose with this syntax.
This time I'm using #Daishi answer from What is the difference between CMD and ENTRYPOINT in a Dockerfile?
The ENTRYPOINT specifies a command that will always be executed when the container starts.
The CMD specifies arguments that will be fed to the ENTRYPOINT.
If you want to make an image dedicated to a specific command you will use ENTRYPOINT ["/path/dedicated_command"]
Otherwise, if you want to make an image for general purpose, you can leave ENTRYPOINT unspecified and use CMD ["/path/dedicated_command"] as you will be able to override the setting by supplying arguments to docker run
Moreover, runapp.sh isn't in your $PATH and you call it without absolute path, so it will not find the file even if the volume is mounted correctly.
You could just use:
CMD /workdir1/runapp.sh "$NEO4J_CONFIG" "$BENCHMARK_NAME"
Now be careful, on your host you mention that the shell script is named script.sh and you call runapp.sh in your Dockerfile, I hope it's a typo. By the way your script needs to be executable.