Here's my Dockerfile:
FROM golang
RUN apt-get update
RUN go get -u github.com/golang/protobuf/{proto,protoc-gen-go}
This generates this error:
package github.com/golang/protobuf/{proto,protoc-gen-go}: invalid github.com/ import path "github.com/golang/protobuf/{proto,protoc-gen-go}"
But, if I take out that RUN directive, and just load up /bin/bash in the docker container, I can run the go get command just fine.
What's going on?
It's happening because the default shell is not /bin/bash, it's sh. You have two possible solutions, either you can explicitly define a shell in your RUN command like so:
RUN ["/bin/bash", "-c", "go get -u github.com/golang/protobuf/{proto,protoc-gen-go}"]
Or you can change the shell that RUN uses by default like so:
SHELL ["/bin/bash", "-c"]
RUN go get -u github.com/golang/protobuf/{proto,protoc-gen-go}
Source: https://docs.docker.com/engine/reference/builder/#/shell
It works if I use another shell, like bash, and put double quotes for the go command
RUN bash -c "go get github.com/golang/protobuf/{proto,protoc-gen-go}"
By the way, you can do all in only one RUN, see Dockerfiles best practices
https://docs.docker.com/engine/userguide/eng-image/dockerfile_best-practices/
Related
I’m setting up a docker container to just be a simple environment for Ocaml, since I don’t wanna have to manage two OPAM tool chains on two computers. (Windows desktop, Linux laptop) My goal is to have the container load in to a bash command prompt on docker-compose run with ocaml ready to go, and to do this I need to enter in to bash and then run eval $(opam env) on startup. This is my current docker file:
FROM ocaml/opam:alpine-3.12
# Create folder and assign owner
USER root
RUN mkdir /code
WORKDIR /code
RUN chown opam:opam /code
USER opam
# Install ocaml
RUN opam init
RUN opam switch create 4.11.1
RUN opam install dune
# bash env
CMD [ "/bin/bash" ]
ENTRYPOINT [ "eval", "\$(opam env)" ]
Building and trying to run this gives me the error:
sh: $(opam env): unknown operand
ERROR: 2
I tried making a run.sh script but that ran into some chmod/permission issues that are probably harder to debug than this. What do I do to open this container in bash and then run the eval $(opam env) command? I don’t want to do this with command line arguments, I’d like to do this all in a dockerfile or docker-compose file
The trick is to use opam exec1 as the entry point, e.g.,
ENTRYPOINT ["opam", "exec", "--"]
Then you can either run directly a command from the installed switch or just start an interactive shell with run -it --rm <cont> sh and you will have the switch fully activated, e.g.,
$ docker run -it --rm binaryanalysisplatform/bap:latest sh
$ which ocaml
/home/opam/.opam/4.09/bin/ocaml
As an aside, since we're talking about docker and OCaml, let me share some more tricks. First of all, you can look into our collection of dockerfiles in BAP for some inspiration. And another important trick that I would like to share is using multistage builds to shrink the size of the image, here's an example Dockerfile. In our case, it gives us a reduction from 7.5 Gb to only 750 Mb, while still preserving the ability to run and build OCaml programs.
And another side note :) You also should run your installation in a single RUN entry, otherwise your layers will eventually diverge and you will get weird missing packages errors. Basically, here's the Dockerfile that you're looking for,
FROM ocaml/opam2:alpine
WORKDIR /home/opam
RUN opam switch 4.11.1 \
&& eval "$(opam env)" \
&& opam remote set-url default https://opam.ocaml.org \
&& opam update \
&& opam install dune \
&& opam clean -acrs
ENTRYPOINT ["opam", "exec", "--"]
1)Or opam config exec, i.e., ENTRYPOINT ["opam", "config", "exec", "--"] for the older versions of opam.
There's no way to tell Docker to do something after the main container process has started, or to send input to the main container process.
What you can do is to write a wrapper script that does some initial setup and then runs whatever the main container process is. Since that eval command will just set environment variables, those will carry through to the main shell.
#!/bin/sh
# entrypoint.sh
# Set up the version-manager environment
eval $(opam env)
# Run the main container command
exec "$#"
In the Dockerfile, make this script be the ENTRYPOINT:
COPY entrypoint.sh /usr/local/bin
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
CMD ["/bin/bash"]
It also might work to put this setup in a shell dotfile, and run bash -l as the main container command to force it to read dotfiles. However, the $HOME directory isn't usually well-defined in Docker, so you might need to set that variable. If you expand this setup to run a full application, the entrypoint-wrapper approach will will there too, but that sequence probably won't read shell dotfiles at all.
What you show looks like an extremely straightforward installation sequence and I might not change it, but be aware that there are complexities around using version managers in Docker. In particular every Dockerfile RUN command has a new shell environment and the eval command won't "stick". I'd ordinarily suggest picking a specific version of the toolchain and directly installing it, maybe in /usr/local, without a version manager, but that approach will be much more complex than what you have currently. For more mainstream languages you can also usually use e.g. a node:16.13 prebuilt image.
What's with the error you're getting? For ENTRYPOINT and CMD (and also RUN) Docker has two forms. If something is a JSON array then Docker runs the command as a sequence of words, with one word in the array translating to one word in the command, and no additional interpretation or escaping. If it isn't a JSON array – even if it's mostly a JSON array, but has a typo – Docker will interpret it as a shell command and run it with sh -c. Docker applies this rule separately to the ENTRYPOINT and CMD, and then combines them together into a single command.
In particular in your ENTRYPOINT line, RFC 8259 §7 defines the valid character escapes in JSON, so \n is a newline and so on, but \$ is not one of those. That makes the embedded string invalid, and therefore the ENTRYPOINT line isn't valid, and Docker runs it via a shell. The single main container command is then
sh -c '[ "eval", "\$(opam env)" ]' '/bin/bash'
which runs the shell command [, as in if [ "$1" = yes ]; then ...; fi. That command doesn't understand the $(...) string as an argument, which is the error you're getting.
The JSON array already has escaped the things that need to be escaped, so it looks like you could get around this immediate error by removing the erroneous backslash
ENTRYPOINT ["eval", "$(opam env)"] # won't actually work
Docker will run this as-is, combining it with the CMD, and you get
'eval' '$(opam env)' '/bin/bash'
But eval isn't a "real" command – there is no /bin/eval binary – and Docker will pass on the literal string $(opam env) without interpreting it at all. That's also not what you want.
In principle it's possible to do this without writing a script, but you lose a lot of flexibility. For example, consider
# no ENTRYPOINT; shell-form CMD
CMD eval $(opam env) && exec /bin/bash
Again, though, if you replace this CMD with anything else you won't have done the initial setup step.
I'm testing an app on docker (search engine) but when I use docker run the bashrc doesn't work if for example there was an alias inside bashrc, I can't use it.
The file bashrc is copied to the container but still can't use it.
My question is why not? is it only because that bashrc needs to be reloaded or there is another reason?
sudo docker run \
--mount type=bind,source=$(pwd)/remise/bashrc,destination=/root/.bashrc,readonly \
--name="s-container" \
ubuntu /go/bin/s qewrty
If you start your container as
docker run ... image-name \
/go/bin/s qwerty
when Docker creates the container, it directly runs the command /go/bin/s qwerty; it does not invoke bash or any other shell to do it. Nothing will ever know to look for a .bashrc file.
Similarly, if your Dockerfile specifies
CMD ["/go/bin/s", "qwerty"]
it runs the command directly without a shell.
There's an alternate shell form of CMD that takes a command string, and runs it via /bin/sh -c. That does involve a shell; but it's neither an interactive nor a login shell, and it's invoked as sh, so it won't read any shell dotfiles (for the specific case where /bin/sh happens to be GNU Bash, see Bash Startup Files).
Since none of these common paths to specify the main container command will read .bashrc or other shell dotfiles, it usually doesn't make sense to try to write or inject these files. If you need to set environment variables, consider the Dockerfile ENV directive or an entrypoint wrapper script instead.
I have a dockerfile and the last command is
CMD ["/opt/startup.sh"]
Now i have another shell script i.e replacevariables.sh and i want to execute the following command in my dockerfile.
sh replacevariables.sh ${app_dir} dev
How can i execute this command. It is a simple script which is basically going to replace some characters of files in ${app_dir}. What can be the solution for this because when i see any kind of documentation they all suggest to run only one sh script.
You can use a Docker ENTRYPOINT to support this. Consider the following Dockerfile fragment:
COPY entrypoint.sh .
RUN chmod +x entrypoint.sh replacevariables.sh
ENTRYPOINT ["./entrypoint.sh"]
# Same as above
CMD ["/opt/startup.sh"]
The ENTRYPOINT becomes the main container process, and it gets passed the CMD as arguments. So your entrypoint can do the first-time setup, and then run the special shell command exec "$#" to replace itself with the command it was given.
#!/bin/sh
./replacevariables.sh "${app_dir}" dev
exec "$#"
Even if you're launching some alternate command in your container (docker run --rm -it yourimage bash to get a debugging shell, for example) this will only replace the "command" part, so bash becomes the "$#" in the script, and you still do the first-time setup before launching the shell.
The important caveats are that ENTRYPOINT must be the JSON-array form (CMD can be a bare string that gets wrapped in /bin/sh -c, but this setup breaks ENTRYPOINT) and you only get one ENTRYPOINT. If you already have an ENTRYPOINT (many SO questions seem to like naming an interpreter there) move it into the start of CMD (CMD ["python3", "./script.py"]).
I'm running the following RUN command in my Dockerfile, expecting a "logs" directory to be created under each of the listed subdirectories:
RUN mkdir -p /opt/seagull/{diameter-env,h248-env,http-env,msrp-env,octcap-env,radius-env,sip-env,synchro-env,xcap-env}/logs
But when I check the image, I see a directory literally called "{diameter-env,h248-env,http-env,msrp-env,octcap-env,radius-env,sip-env,synchro-env,xcap-env}" created under /opt/seagull, instead of brace expansion taking place.
What could I be doing wrong?
You're not using brace expansion, because you're not using Bash. If you look at the documentation for the RUN command:
RUN (shell form, the command is run in a shell, which by default is /bin/sh -c on Linux or cmd /S /C on Windows)
And also:
Note: To use a different shell, other than ‘/bin/sh’, use the exec form passing in the desired shell. For example, RUN ["/bin/bash", "-c", "echo hello"]
So, just change the command to use the exec form and explicitly use a Bash shell:
RUN [ "/bin/bash", "-c", "mkdir -p /opt/seagull/{diameter-env,h248-env,http-env,msrp-env,octcap-env,radius-env,sip-env,synchro-env,xcap-env}/logs" ]
If /bin/bash is available in your image, you can change the shell that the docker build system uses to execute your RUN command, like this:
SHELL ["/bin/bash", "-c"]
Now, your RUN command should work unchanged.
I have some script that I need to run inside the container, and somehow it only run if I run it inside a bash --login.
I normally run my docker: docker build -t sometags . and I noticed it only run bash without --login.
I know I can just use bash -l -c "some-command-here" but I'd say it's my final fallback if nothing can helps.
so, tl;dr: how can I achieve something like this in my Dockerfile
#dockerfile
RUN bash --login
RUN some-script
and then, I'll just run it with: docker build -t x/y:z .
updated:
the scripts I want to run is things like: gem install bundler, and bundle install.