Following the instructions as outlined to deploy Duo CloudMapper to AWS environment and getting an error
Docker File
FROM python:3.7-slim as cloudmapper
LABEL maintainer="https://github.com/0xdabbad00/"
LABEL Project="https://github.com/duo-labs/cloudmapper"
WORKDIR /opt/cloudmapper
ENV AWS_DEFAULT_REGION=us-east-1
RUN apt-get update -y
RUN apt-get install -y build-essential autoconf automake libtool python3.7-dev python3-tk jq awscli
COPY cloudmapper/. /opt/cloudmapper
COPY entrypoint.sh /opt/cloudmapper/entrypoint.sh
# Remove the demo data
RUN rm -rf /opt/cloudmapper/account-data/demo
# Install the python libraries needed for CloudMapper
RUN cd /opt/cloudmapper && pip install -r requirements.txt
ENTRYPOINT /opt/cloudmapper/entrypoint.sh
Now building the docker image
C:\> docker build -t cloudmapper .
When I run the docker using the below command I get an error
C:/> docker run -t cloudmapper
Error
/bin/sh: 1: /opt/cloudmapper/entrypoint.sh: not found
Verified that the file exists in the appropriate location
Using Docker on Windows 10
Image in the dockerfile is python:3.7-slim
Assuming the images are removed and replaced with text and the question doesn't get closed.
bash can return "file not found" when
the entrypoint shell script is not marked executable for the current user
the hash bang in the entrypoint shell script points to a binary that does not exist
the shell script actually does not exist.
You can fix the first problem by ensuring you use the new --chmod flag to ensure the executable bit is set. Even if the user is root it is necessary that there is at least 1 executable bit set.
COPY --chmod=0755 *.sh /opt/cloudmapper/
ENTRYPOINT ["/opt/cloudmapper/entrypoint.sh"]
ps. This integrated COPY --chmod only works with buildkit enabled builds, so you might need to force buildkit, or split the chmod into a separate explicit RUN step.
The 2nd issue can be dealt with by ensuring the first line of entrypoint.sh uses sh rather than bash if you are using a lightweight base image like alpine:
#!/bin/sh
set -e
# etc
Also, if on Windows especially, ensure ALL files, especially the entrypoint .sh file, are set to utf-8 encoding with lf style line endings. As linux doesn't understand the cr, it will try to execute /bin/sh<cr> as the shell which clearly doesn't exist.
In terms of the file not existing, verify the entrypoint.sh is being copied into a location that is referenced by env.PATH, or that the entry point directive uses a fully qualified path.
--
edited to add cr-lf revelation.
Related
I’m setting up a docker container to just be a simple environment for Ocaml, since I don’t wanna have to manage two OPAM tool chains on two computers. (Windows desktop, Linux laptop) My goal is to have the container load in to a bash command prompt on docker-compose run with ocaml ready to go, and to do this I need to enter in to bash and then run eval $(opam env) on startup. This is my current docker file:
FROM ocaml/opam:alpine-3.12
# Create folder and assign owner
USER root
RUN mkdir /code
WORKDIR /code
RUN chown opam:opam /code
USER opam
# Install ocaml
RUN opam init
RUN opam switch create 4.11.1
RUN opam install dune
# bash env
CMD [ "/bin/bash" ]
ENTRYPOINT [ "eval", "\$(opam env)" ]
Building and trying to run this gives me the error:
sh: $(opam env): unknown operand
ERROR: 2
I tried making a run.sh script but that ran into some chmod/permission issues that are probably harder to debug than this. What do I do to open this container in bash and then run the eval $(opam env) command? I don’t want to do this with command line arguments, I’d like to do this all in a dockerfile or docker-compose file
The trick is to use opam exec1 as the entry point, e.g.,
ENTRYPOINT ["opam", "exec", "--"]
Then you can either run directly a command from the installed switch or just start an interactive shell with run -it --rm <cont> sh and you will have the switch fully activated, e.g.,
$ docker run -it --rm binaryanalysisplatform/bap:latest sh
$ which ocaml
/home/opam/.opam/4.09/bin/ocaml
As an aside, since we're talking about docker and OCaml, let me share some more tricks. First of all, you can look into our collection of dockerfiles in BAP for some inspiration. And another important trick that I would like to share is using multistage builds to shrink the size of the image, here's an example Dockerfile. In our case, it gives us a reduction from 7.5 Gb to only 750 Mb, while still preserving the ability to run and build OCaml programs.
And another side note :) You also should run your installation in a single RUN entry, otherwise your layers will eventually diverge and you will get weird missing packages errors. Basically, here's the Dockerfile that you're looking for,
FROM ocaml/opam2:alpine
WORKDIR /home/opam
RUN opam switch 4.11.1 \
&& eval "$(opam env)" \
&& opam remote set-url default https://opam.ocaml.org \
&& opam update \
&& opam install dune \
&& opam clean -acrs
ENTRYPOINT ["opam", "exec", "--"]
1)Or opam config exec, i.e., ENTRYPOINT ["opam", "config", "exec", "--"] for the older versions of opam.
There's no way to tell Docker to do something after the main container process has started, or to send input to the main container process.
What you can do is to write a wrapper script that does some initial setup and then runs whatever the main container process is. Since that eval command will just set environment variables, those will carry through to the main shell.
#!/bin/sh
# entrypoint.sh
# Set up the version-manager environment
eval $(opam env)
# Run the main container command
exec "$#"
In the Dockerfile, make this script be the ENTRYPOINT:
COPY entrypoint.sh /usr/local/bin
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
CMD ["/bin/bash"]
It also might work to put this setup in a shell dotfile, and run bash -l as the main container command to force it to read dotfiles. However, the $HOME directory isn't usually well-defined in Docker, so you might need to set that variable. If you expand this setup to run a full application, the entrypoint-wrapper approach will will there too, but that sequence probably won't read shell dotfiles at all.
What you show looks like an extremely straightforward installation sequence and I might not change it, but be aware that there are complexities around using version managers in Docker. In particular every Dockerfile RUN command has a new shell environment and the eval command won't "stick". I'd ordinarily suggest picking a specific version of the toolchain and directly installing it, maybe in /usr/local, without a version manager, but that approach will be much more complex than what you have currently. For more mainstream languages you can also usually use e.g. a node:16.13 prebuilt image.
What's with the error you're getting? For ENTRYPOINT and CMD (and also RUN) Docker has two forms. If something is a JSON array then Docker runs the command as a sequence of words, with one word in the array translating to one word in the command, and no additional interpretation or escaping. If it isn't a JSON array – even if it's mostly a JSON array, but has a typo – Docker will interpret it as a shell command and run it with sh -c. Docker applies this rule separately to the ENTRYPOINT and CMD, and then combines them together into a single command.
In particular in your ENTRYPOINT line, RFC 8259 §7 defines the valid character escapes in JSON, so \n is a newline and so on, but \$ is not one of those. That makes the embedded string invalid, and therefore the ENTRYPOINT line isn't valid, and Docker runs it via a shell. The single main container command is then
sh -c '[ "eval", "\$(opam env)" ]' '/bin/bash'
which runs the shell command [, as in if [ "$1" = yes ]; then ...; fi. That command doesn't understand the $(...) string as an argument, which is the error you're getting.
The JSON array already has escaped the things that need to be escaped, so it looks like you could get around this immediate error by removing the erroneous backslash
ENTRYPOINT ["eval", "$(opam env)"] # won't actually work
Docker will run this as-is, combining it with the CMD, and you get
'eval' '$(opam env)' '/bin/bash'
But eval isn't a "real" command – there is no /bin/eval binary – and Docker will pass on the literal string $(opam env) without interpreting it at all. That's also not what you want.
In principle it's possible to do this without writing a script, but you lose a lot of flexibility. For example, consider
# no ENTRYPOINT; shell-form CMD
CMD eval $(opam env) && exec /bin/bash
Again, though, if you replace this CMD with anything else you won't have done the initial setup step.
I have a docker image inside which some installations require adding exports to .bashrc.
My export variables are inside /root/.bashrc on the image.
Here is the dockerfile
FROM ubuntu:20.04
RUN apt-get update && apt-get install -y python3
RUN echo "export PATH=/path/to/stuff:\$PATH" >> /root/.bashrc
CMD ["python3"]
The environment variables are present when using the following command
docker run -it image /bin/bash
When I run the following command, environment variables are not present.
docker run -it image
It is expected since /bin/sh is the default entry point of docker
But after the following change, the environment variable are not set either.
docker commit --change='ENTRYPOINT ["/bin/bash","-c"]' container image
I tried different combinations such as
docker commit --change='CMD ["/bin/bash","-c","python3 myProgram.py"]' container image
or
docker commit --change='ENTRYPOINT ["/bin/bash","-c"]' --change='CMD ["source /root/.bashrc && python3 myProgram.py"]' container image
But the environment variables are not present.
How do I run the CMD statement with the environment variable from .bashrc loaded ?
In order to see the path variable, I use echo $PATH when I run /bin/bash and import os followed by os.getenv("PATH") when I run python3 from CMD.
Edit:
The exports are part of the installation of a library. In order to use the library, the updated exports (such as PYTHONPATH and LD_LIBRARY_PATH) needs to be set.
If .bashrc is not intended to be launched, as mentioned in the comments. How can I make this library work in the docker environment ?
As mentioned by #itshosyn in the comments, the standard way to override environment variables such as PATH consists in using the ENV directive.
So you may try writing something like this:
FROM ubuntu:20.04
RUN apt-get update && apt-get install -y python3
ENV PATH="/path/to/stuff:$PATH"
CMD ["python3"]
[this is the error I'm getting after build command ]
Step 7/9 : RUN chmod +x /main.sh
---> Running in 6e880a009c7d
chmod: cannot access '/main.sh': No such file or directory
The command '/bin/sh -c chmod +x /main.sh' returned a non-zero code: 1
and here is my docker file
FROM centos:latest
MAINTAINER Aditya Gupta
#install git
RUN yum -y update
RUN yum -y install git
#make git repo folder, change GIT_LOCATION
RUN mkdir -p /home/centos/doimages/dockimg;cd /home/centos/doimages/dockimg;
RUN git clone https://(username):(password)#gitlab.com/abc/xyz.git (foldername);cd (foldername)/
Run chmod +x ./main.sh
RUN echo " ./main.sh\n "
EXPOSE Portnumber
When you perform a RUN step in a Dockerfile, a temporary container is launched, often with a shell parsing your command. When that command finishes, the container exits, and docker packages the filesystem changes as an image layer. That process is repeated from the beginning for each RUN line.
The key piece there is the shell exits, losing environment variables you've set, background processes you've run, and in this case, the current working directory you tried to set here:
RUN git clone https://(username):(password)#gitlab.com/abc/xyz.git (foldername);cd (foldername)/
Instead of a cd in a RUN command, you can update the value of WORKDIR:
RUN git clone https://(username):(password)#gitlab.com/abc/xyz.git (foldername)
WORKDIR foldername
You want to execute a shell file which does not exist on your docker machine. use ADD command to add your script to your docker image!
-- somewehe inside your dockerfile befor the execution ---
ADD ./PATH/ON/HOST/main.sh /PATH/YOU/LIKE/ON/DOCKER/MACHINE
Then try to build your docker machine
issue is resolved with workdir and cloning manually without docker file and then give the path to mainsh in dockerfile.
I have a custom package with install.sh script, which I want to run while building a docker image (meaning - put ./install.sh inside Dockerfile). I could have ran it along with the container, but I want to have an image that contains the required packages (which are mentioned in the install script).
What I tried:
RUN /bin/sh/ -c "./install.sh"
RUN ./install.sh
It errors out saying -
/bin/sh install.sh not found
or
/bin/sh ./install.sh not found
This might be a repeated question, but I haven't found an answer to this anywhere. Any help would be appreciated.
You must copy your install.sh into docker image with this command in your dockerfile:
COPY install.sh /tmp
Then use your RUN command to run it:
RUN /bin/sh -c "/tmp/install.sh"
or
RUN sh /tmp/install.sh
Don't forget to make install.sh executable before run it:
chmod +x /tmp/install.sh
I have following Dockerfile:
FROM ubuntu:16.04
ARG path1=def_path1
RUN mkdir ${path1}
When I build this Dockerfile using following command:
docker build --build-arg path1=/home/dragan -t build_arg_ex .
I get following error when I execute it in MINGW bash on Windows 10:
$ ./build.sh --no-cache
Sending build context to Docker daemon 6.144kB
Step 1/3 : FROM ubuntu:16.04
---> 2a4cca5ac898
Step 2/3 : ARG path1=def_path1
---> Running in a35241ebdef3
Removing intermediate container a35241ebdef3
---> 01475e50af4c
Step 3/3 : RUN mkdir ${path1}
---> Running in 2759e683cbb1
mkdir: cannot create directory 'C:/Program': No such file or directory
mkdir: cannot create directory 'Files/Git/home/dragan': No such file or
directory
The command '/bin/sh -c mkdir ${path1}' returned a non-zero code: 1
Building same Dockerfile in Windows Command Prompt or on Linux or Mac is ok. The problem is only in MINGW bash terminal on Windows because it adds 'C:/Program Files/Git' before the path that is passed as argument.
Is there a way to execute this in MINGW bash so it does not add the 'C:/Program Files/Git' prefix?
Thanks
This is actually a bug/limitation of Git for Windows as described in the Release Notes under Known issues:
If you specify command-line options starting with a slash, POSIX-to-Windows path conversion will kick in converting e.g. "/usr/bin/bash.exe" to "C:\Program Files\Git\usr\bin\bash.exe". When that is not desired -- e.g. "--upload-pack=/opt/git/bin/git-upload-pack" or "-L/regex/" -- you need to set the environment variable MSYS_NO_PATHCONV temporarily, like so:
MSYS_NO_PATHCONV=1 git blame -L/pathconv/ msys2_path_conv.cc
Alternatively, you can double the first slash to avoid POSIX-to-Windows path conversion, e.g. "//usr/bin/bash.exe".
Further to #mat007's answer:
This bash function solved the problem more permanently for docker, without enabling MSYS_NO_PATHCONV globally, which causes another world of pain.
.bashrc
# See https://github.com/docker/toolbox/issues/673#issuecomment-355275054
# Workaround for Docker for Windows in Git Bash.
docker()
{
(export MSYS_NO_PATHCONV=1; "docker.exe" "$#")
}
You may need to do the same for docker-compose