I want to pass an argument to my dockerfile such that I should be able to use that argument in my run command but it seems I am not able to do so
I am using a simple bash file that will trigger the docker build and docker run
FROM openjdk:8 AS SCRATCH
WORKDIR /
ADD . .
RUN apt install unzip
RUN unzip target/universal/rule_engine-1.0.zip -d target/universal/rule_engine-1.0
COPY target/universal/rule_engine-1.0 .
ENV MONGO_DB_HOST="host.docker.internal"
ENV MONGO_DB_PORT="27017"
EXPOSE 9000
ARG path
CMD target/universal/rule_engine-1.0/bin/rule_engine -Dconfig.file=$path
above is my dockerfile
and below is my bash file which will access this dockerfile
#!/bin/bash
# change the current path to rule engine path
cd /Users/zomato/Documents/Intern_Project/rule_engine
sbt dist
ENVIR=$1
config=""
if [ $ENVIR == "local" ]
then
config=conf/application.conf
elif [ $ENVIR == "staging" ]
then
config=conf/staging.conf
else
config=conf/production.conf
fi
echo $config
docker build --build-arg path=$config -t rule_engine_zip .
docker run -i -t -p 9000:9000 rule_engine_zip
but when i access the dockerfile through bash script which will set config variable I am not able to set path variable in last line of dockerfile to the value of config.
ARG values won't be available after the image is built, so
a running container won’t have access to those values. To dynamically set an env variable, you can combine both ARG and ENV (since ENV can't be overridden):
ARG PATH
ENV P=${PATH}
CMD target/universal/rule_engine-1.0/bin/rule_engine -Dconfig.file=$P
For further explanation, I recommend this article, which explains the difference between ARG and ENV in a clear way:
As you can see from the above image, the ARG values are available only during the image build.
Related
The startup.sh script for my node.js project is:
#!bin/sh
echo "Starting with ENV: $ENV"
cd /app
export PORT=8080
export NODE_ENV=$ENV
node main.js $ENV
The Dockerfile looks like:
FROM ...
LABLE ..
ARG env
ENV ENV=${env}
RUN mkdir /app
# COPY required files
COPY ...
RUN cd /app && npm install --silent
EXPOSE 8080
ENTRYPOINT ["/bin/sh", "/app/startup.sh"]
I'm building the image using Jenkins with command:
docker build --build-arg ENV=test -t "tag" .
The script is failing to receive the value "test".
What I tried:
Updating startup.sh to use $1(positional arg) instead of $ENV and pass the value through ENTRYPOINT:
ENTRYPOINT ["/bin/sh", "/app/startup.sh", ${ENV}]
This gives error: file or directory not found: /bin/sh /bin/sh,]
ENTRYPOINT ["/bin/sh", "/app/startup.sh", "${ENV}"]
Doesn't give error but value of $1 is "${ENV}"
Use combination of ENTRYPOINT and CMD:
ENTRYPOINT ["/bin/sh", "/app/startup.sh"] and CMD[${ENV}]
The value of $1 is coming as "-c".
I think it's because of the use of ARG env
ARG ENV
ENV ENV=${ENV}
I have custom entrypoint where a environment variable is exported. The value of the environment variable is constructed using two other variables provided at runtime.
Snippet from Dockerfile CMD ["bash", "/opt/softwareag/laas-api-server/entrypoint.sh"].
Snippet from entrypoint.sh
export URL="$SCHEME://$HOST:$PORT"
echo "URL:$URL"
Command docker run -e HOST="localhost" -e PORT="443" mycentos prints URL:localhost:443 as expected but the same variable appears to have lost the value when the following command is executed.
docker exec -ti <that-running-container-from-myimage> bash
container-prompt> echo $URL
<empty-line>
Why would the exported variable appear to have lost the value of URL? What is getting lost here?
The environment variable will not persist across all bash session. when the container run it will only available in that entrypoint session but later it will not available if it set using export.
docker ENV vs RUN export
If you want to use across all session you should set them in Dockerfile.
ENV SCHEME=http
ENV HOST=example.com
ENV PORT=3000
And in the application side, you can use them together.also it will be available for all session.
curl "${SCHEME}://${HOST}:${PORT}
#
Step 8/9 : RUN echo "${SCHEME}://${HOST}:${PORT}"
---> Running in afab41115019
http://example.com:3000
Now if we look into the way you are using, it will not work because
export URL="$SCHEME://$HOST:$PORT"
# only in this session
echo "URL:$URL"
# will be available for node process too but for this session only
node app.js
For example look into this Dockerfile
FROM node:alpine
RUN echo $'#!/bin/sh \n\
export URL=example.com \n\
echo "${URL}" \n\
node -e \'console.log("ENV URL value inside nodejs", process.env.URL)\' \n\
exec "$#" \n\
' >> /bin/entrypoint.sh
RUN chmod +x /bin/entrypoint.sh
entrypoint ["entrypoint.sh"]
So you when you Run docker container for the first time you will able to see the expected response.
docker run -it --rm myapp
example.com
ENV URL value inside nodejs example.com
Now we want to check for later session.
docker run -it --rm abc tail -f /dev/null
example.com
ENV URL value inside nodejs example.com
so the container is up during this time, we can verify for another session
docker exec -it myapp sh -c "node -e 'console.log(\"ENV URL value inside nodejs\", process.env.URL)'"
ENV URL value inside nodejs undefined
As we can same script but different behaviour because of docker, so the variable is only available in that session, you can write them to file if you are interested in later use.
If I set an environment variable, say ENV ADDRESSEE=world, and I want to use it in the entry point script concatenated into a fixed string like:
ENTRYPOINT ["./greeting", "--message", "Hello, world!"]
with world being the value of the environment varible, how do I do it? I tried using "Hello, $ADDRESSEE" but that doesn't seem to work, as it takes the $ADDRESSEE literally.
You're using the exec form of ENTRYPOINT. Unlike the shell form, the exec form does not invoke a command shell. This means that normal shell processing does not happen. For example, ENTRYPOINT [ "echo", "$HOME" ] will not do variable substitution on $HOME. If you want shell processing then either use the shell form or execute a shell directly, for example: ENTRYPOINT [ "sh", "-c", "echo $HOME" ].
When using the exec form and executing a shell directly, as in the case for the shell form, it is the shell that is doing the environment variable expansion, not docker.(from Dockerfile reference)
In your case, I would use shell form
ENTRYPOINT ./greeting --message "Hello, $ADDRESSEE\!"
After much pain, and great assistance from #vitr et al above, i decided to try
standard bash substitution
shell form of ENTRYPOINT (great tip from above)
and that worked.
ENV LISTEN_PORT=""
ENTRYPOINT java -cp "app:app/lib/*" hello.Application --server.port=${LISTEN_PORT:-80}
e.g.
docker run --rm -p 8080:8080 -d --env LISTEN_PORT=8080 my-image
and
docker run --rm -p 8080:80 -d my-image
both set the port correctly in my container
Refs
see https://www.cyberciti.biz/tips/bash-shell-parameter-substitution-2.html
I tried to resolve with the suggested answer and still ran into some issues...
This was a solution to my problem:
ARG APP_EXE="AppName.exe"
ENV _EXE=${APP_EXE}
# Build a shell script because the ENTRYPOINT command doesn't like using ENV
RUN echo "#!/bin/bash \n mono ${_EXE}" > ./entrypoint.sh
RUN chmod +x ./entrypoint.sh
# Run the generated shell script.
ENTRYPOINT ["./entrypoint.sh"]
Specifically targeting your problem:
RUN echo "#!/bin/bash \n ./greeting --message ${ADDRESSEE}" > ./entrypoint.sh
RUN chmod +x ./entrypoint.sh
ENTRYPOINT ["./entrypoint.sh"]
I SOLVED THIS VERY SIMPLY!
IMPORTANT: The variable which you wish to use in the ENTRYPOINT MUST be ENV type (and not ARG type).
EXAMPLE #1:
ARG APP_NAME=app.jar # $APP_NAME can be ARG or ENV type.
ENV APP_PATH=app-directory/$APP_NAME # $APP_PATH must be ENV type.
ENTRYPOINT java -jar $APP_PATH
This will result with executing:
java -jar app-directory/app.jar
EXAMPLE #2 (YOUR QUESTION):
ARG ADDRESSEE="world" # $ADDRESSEE can be ARG or ENV type.
ENV MESSAGE="Hello, $ADDRESSEE!" # $MESSAGE must be ENV type.
ENTRYPOINT ./greeting --message $MESSAGE
This will result with executing:
./greeting --message Hello, world!
Please verify to be sure, whether you need quotation-marks "" when assigning string variables.
MY TIP: Use ENV instead of ARG whenever possible to avoid confusion on your part or the SHELL side.
For me, I wanted to store the name of the script in a variable and still use the exec form.
Note: Make sure, the variable you are trying to use is declared an environment variable either from the commandline or via the ENV directive.
Initially I did something like:
ENTRYPOINT [ "${BASE_FOLDER}/scripts/entrypoint.sh" ]
But obviously this didn't work because we are using the shell form and the first program listed needs to be an executable on the PATH. So to fix this, this is what I ended up doing:
ENTRYPOINT [ "/bin/bash", "-c", "exec ${BASE_FOLDER}/scripts/entrypoint.sh \"${#}\"", "--" ]
Note the double quotes are required
What this does is to allow us to take whatever extra args were passed to /bin/bash, and supply those same arguments to our script after the name has been resolved by bash.
man 7 bash
-- A -- signals the end of options and disables further
option processing. Any arguments after the -- are treated
as filenames and arguments. An argument of - is
equivalent to --.
In my case worked this way: (for Spring boot app in docker)
ENTRYPOINT java -DidMachine=${IDMACHINE} -jar my-app-name
and passing the params on docker run
docker run --env IDMACHINE=Idmachine -p 8383:8383 my-app-name
I solved the problem using a variation on "create a custom script" approach above. Like this:
FROM hairyhenderson/figlet
ENV GREETING="Hello"
RUN printf '#!/bin/sh\nfiglet -W \${GREETING} \$#\n' > /runme && chmod +x /runme
ENTRYPOINT ["/runme"]
CMD ["World"]
Run like
docker container run -it --rm -e GREETING="G'Day" dockerfornovices/figlet-greeter Alec
If someone wants to pass an ARG or ENV variable to exec form of ENTRYPOINT then a temp file created during image building process might be used.
In my case I had to start the app differently depending on whether the .NET app has been published as self-contained or not.
What I did is I created the temp file and I used its name in the if statement of my bash script.
Part of my dockerfile:
ARG SELF_CONTAINED=true #ENV SELF_CONTAINED=true also works
# File has to be used as a variable as it's impossible to pass variable do ENTRYPOINT using Exec form. File name allows to check whether app is self-contained
RUN touch ${SELF_CONTAINED}.txt
COPY run-dotnet-app.sh .
ENTRYPOINT ["./run-dotnet-app.sh", "MyApp" ]
run-dotnet-app.sh:
#!/bin/sh
FILENAME=$1
if [ -f "true.txt" ]; then
./"${FILENAME}"
else
dotnet "${FILENAME}".dll
fi
Here is what worked for me:
ENTRYPOINT [ "/bin/bash", "-c", "source ~/.bashrc && ./entrypoint.sh ${#}", "--" ]
Now you can supply whatever arguments to the docker run command and still read all environment variables.
I'm having trouble with Docker creating a container that does not have environment variables set that I know I set in the image definition.
I have created a Dockerfile that generates an image of OpenSuse 42.3. I need to have some environment variables set up in the image so that anyone that starts a container from the image can use a code that I've compiled and placed in the image.
I have created a shell file called "image_env_setup.sh" that contains the necessary environment variable definitions. I also manually added those environment variable definitions to the Dockerfile.
USER codeUser
COPY ./docker/image_env_setup.sh /opt/MyCode
ENV PATH="$PATH":"/opt/MyCode/bin:/usr/lib64/mpi/gcc/openmpi/bin"
ENV LD_LIBRARY_PATH="/usr/lib64:/opt/MyCode/lib:"
ENV PS1="[\u#docker: \w]\$ "
ENV TERM="xterm-256color"
ENV GREP_OPTIONS="--color=auto"
ENV EDITOR=/usr/bin/vim
USER root
RUN chmod +x /opt/MyCode/image_env_setup.sh
USER codeUser
RUN /opt/MyCode/image_env_setup.sh
RUN /bin/bash -c "source /opt/MyCode/image_env_setup.sh"
The command that I use to create the container is:
docker run -it -d --name ${containerName} -u $userID:$groupID \
-e USER=$USER --workdir="/home/codeUser" \
--volume="${home}:/home/codeUser" ${imageName} /bin/bash \
The only thing that works is to pass the shell file to be run again when the container starts up.
docker start $MyImageTag
docker exec -it $MyImageTag /bin/bash --rcfile /opt/MyCode/image_env_setup.sh
I didn't think it would be that difficult to just have the shell variables setup within the container so that any entry into it would provide a user with them already defined.
RUN entries cannot modify environment variables (I assume you want to set more variables in image_env_setup.sh). Only ENV entries in the Dockerfile (and docker options like --rcfile can change the environment).
You can also decide to source image_env_setup.sh from the .bashrc, of course.
For example, you could either pre-fabricate a .bashrc and pull it in with COPY, or do
RUN echo '. /opt/MyCode/image_env_setup.sh' >> ~/.bashrc
you can put /opt/MyCode/image_env_setup.sh in ~/.bash_profile or ~/.bashrc of the container so that everytime you get into the container you have the env's set
I am having trouble creating and using variables in a Dockerfile - I build a Docker image via a Dockerfile with this command:
$ docker build --build-arg s=scripts/a.sh -t a .
(So because I use --build-arg, $s will be an available argument in the Dockerfile, and this part works)
The Dockerfile is like so:
ARG s
RUN echo $s
RUN useradd -ms /bin/bash newuser
USER newuser
WORKDIR /home/newuser
ENV fn=$(filename $s) # fails on this line
COPY $s .
ENTRYPOINT ["/bin/bash", "/home/newuser/$fn"]
The problem I have is that the Docker build is failing on the line indicated above.
Error response from daemon: Syntax error - can't find = in "$s)". Must be of the form: name=value
If I change that line to this:
RUN fn=$(filename $s)
I get this error:
Error: Command failed: docker build --build-arg s=scripts/a.sh -t a .
The command '/bin/sh -c fn=$(filename $s)' returned a non-zero code: 127
Anyone know the correct way to
Create a variable inside the docker file
Use string interpolation with that variable so that I can reference the variable in the ENTRYPOINT arguments like so:
ENTRYPOINT ["/bin/bash", "/home/newuser/$var"]
Even if I do this:
ARG s
ARG sname
RUN echo $s # works as expected
RUN echo $sname # works as expected
RUN useradd -ms /bin/bash newuser
USER newuser
WORKDIR /home/newuser
COPY $s . # works as expected (I believe)
ENTRYPOINT /bin/bash /home/newuser/$sname # does not work as expected
even though I am using the "non-JSON" version of ENTRYPOINT, it still doesn't seem to pick up the value for the $sname variable.
I would avoid using variable in ENTRYPOINT at all. It's tricky and requires a deep understanding of what is going on. And is easy to break it by accident. Just consider one of the following.
Create link with the known name to your start script.
RUN ln -s /home/newuser/$sname /home/newuser/docker_entrypoint.sh
ENTRYPOINT ["/home/newuser/docker_entrypoint.sh"]
or write standalone entrypoint script that runs what you need.
But if you want to know how and why solutions in your questions work just keep reading.
First some definitions.
ENV - is environment variable available during buildtime (docker build) and runtime (docker run)
ARG - is environment variable available only during buildtime
If you look at https://docs.docker.com/engine/reference/builder/#environment-replacement you see the list of dockerfile instructions that support those environment variables directly. This is why COPY "picks up the variable" as you said.
Please note that there is no RUN nor ENTRYPOINT. How does it work?
You need to dig into the documentation. First RUN (https://docs.docker.com/engine/reference/builder/#run). There are 2 forms. The first one executes command through the shell and this shell has access to buildtime environment variables.
# this works because it is called as /bin/sh -c 'echo $sname'
# the /bin/sh replace $sname for you
RUN echo $sname
# this does NOT work. There is no shell process to do $sname replacement
# for you
RUN ["echo", "$sname"]
Same thing applies to the ENTRYPOINT and CMD except only runtime variables are available during container start.
# first you need to make some runtime variable from builtime one
ENV sname $sname
# Then you can use it during container runtime
# docker runs `/bin/sh -c '/bin/bash /home/newuser/$sname'` for you
# and this `/bin/sh` proces interprets `$sname`
ENTRYPOINT /bin/bash /home/newuser/$sname
# but this does NOT work. There is no process to interpolate `$sname`
# docker runs what you describe.
ENTRYPOINT ["/bin/bash", "/home/newuser/$sname"]
edit 2017-04-03: updated links to the docker documentations and slight rewording to avoid confusion that I sense from other answers and comments.
I requested #Villem to answer, and his answer is much more definitive, but the following will work (just is not a stable solution). His answer is basically saying that this answer is not a good way to do it:
ARG s # passed in via --build-arg s=foo
ARG sname # passed in via --build-arg sname=bar
RUN echo $s
RUN echo $sname
ENV sname $sname # this is the key part
RUN useradd -ms /bin/bash newuser
USER newuser
WORKDIR /home/newuser
COPY $s .
ENTRYPOINT /bin/bash /home/newuser/$sname # works, but is not stable!
don't ask me why the COPY command picks up the variable that was declared via ARG, but that the ENTRYPOINT command does not seem to pick up the variable declared via ARG, but only picks up the variable declared via ENV. At least, this appears to be the case.
I spent a lot of time to find out that.
Don't works !
ARG install="bundle install --jobs=4"
FROM ruby:2.6.3-alpine
RUN eval $install
But this works...
FROM ruby:2.6.3-alpine
ARG install="bundle install --jobs=4"
RUN eval $install
Then to build the docker image:
docker build -t server --no-cache --build-arg install="bundle install --without development test" .`
I wanted both variable substitution and arguments passing.
Let's say our Dockerfile has:
ENV VARIABLE=replaced
And we want to run this:
docker run <image> arg1 arg2
I obviously wanted this output:
replaced arg1 arg2
I eventually found this one:
ENTRYPOINT [ "sh", "-c", "echo $VARIABLE $0 $#" ]
It works!!!
But I feel SOOOO dirty!
Obviously, in real life, I wanted to do something more useful:
docker run <image> --app.options.additional.second=true --app.options.additional.third=false
ENTRYPOINT [ "sh", "-c", "java -Xmx$XMX $0 $#", \
"-jar", \
"/my.jar", \
"--app.options.first=true" ]
Why a so complicated answer?
"ENTRYPOINT java..." would not pass docker arguments to the entrypoint => "ENTRYPOINT [..." is mandatory for that
"ENTRYPOINT [..." will NOT call the shell, so no variable substitution is done at all => "sh -c" is mandatory for that
"sh -c" only take the FIRST argument passed to it and split it in command+arguments => so everything must be in the first argument of "sh -c" for variables to be visible by the command
Docker arguments are passed as extra array entries of "ENTRYPOINT [..." => so the "$#" is necessary to "copy" the remainings arguments of the "sh -c" into the "echo ..." command to be executed (and as a bonus, we can reuse the additional array entries of ENTRYPOINT[] to place forced arguments in a readable way in the Dockerfile)
"$#" removes the first argument => so explicit "$0" must be used before "$#"
Fiou...
I added a comment to this issue, for Docker developers to see what we are forced to do and perhaps change their mind to make ENTRYPOINT[] replace environment variables: https://github.com/moby/moby/issues/4783#issuecomment-442466609