I've wrote a shell script file on mac to deploy some docker application. But when I copy it to Windows10, it seems that it cannot run properly.
The path after "docker exec xx /foo/bar" is always transferred into an absolute path in windows, and leads to an error like: "sh: D:/DevTools/Git/root/start_zk.sh: No such file or directory".
What's more, when executing "docker exec xx /foo/bar" in powershell or git bash, it can run properly. This problem only occurs when I write it into a file, and run it through "sh deploy.sh".
I've tried to add "\" before "/", but it doesn't help. My code is listed below.
echo "deploying zookeepers..."
docker exec ac sh /root/start_zk.sh
Is there any fault in my script? The command "sh /root/start_zk.sh" after "docker exec" should be a parameter for the docker container to execute, but it seems that it is recognized by windows as a command or path. Is there any way to solve this?
Related
I am trying to copy a folder outside of a container using docker cp, but I am running in an unexpected issue: the command works perfectly outside of a shell script yet fails when running the script.
For example: copy_indices.sh
for x in "${find_container_id_arr[#]}"; do
CONTAINER_NAME="${x}"
CONTAINER_ID=$(docker ps -aqf "name=${x}")
idx_name=$(docker exec -it "$CONTAINER_ID" ls -1 /usr/share/elasticsearch/data/nodes/0/indices)
docker cp "$CONTAINER_ID":/usr/share/elasticsearch/data/nodes/0/indices/"$idx_name" "$ALL_INDICES"/"$idx_name"
done
I determine the container ID using CONTAINER_ID=$(docker ps -aqf "name=${x}"), find the name of folder I need using idx_name=$(docker exec -it "$CONTAINER_ID" ls -1 /usr/share/elasticsearch/data/nodes/0/indices) and then copy it on the host filesystem: docker cp "$CONTAINER_ID":/usr/share/elasticsearch/data/nodes/0/indices/"$idx_name" "$ALL_INDICES"/"$idx_name"
My issue is that every command evaluates and run as expected when not put inside this script. I can run the command docker cp <my_container>:/usr/share/elasticsearch/data/nodes/0/indices/<index_name> ./all_indices/<index_name> and the target folder is indeed found and copied onto the host.
Once these commands are inside a script however, I get an "Error: No such container:path:" error and I can't pinpoint what is going wrong, because the mentionned path indeed exists in the container and the container is correct as I tested it running the "final" command supposed to be executed (the docker cp one).
What could be the reason these commands suddenly stop working when put in a shell script?
I'm trying to execute a script on the master node of AWS EMR cluster. The intention is to create a new conda env and link it to jupyter. I'm following this doc from AWS. Problem is, whatever be the content of the script, I'm getting the same error: bash: /home/hadoop/scripts/bootstrap.sh: No such file or directory while executing sudo docker exec jupyterhub bash /home/hadoop/scripts/bootstrap.sh. I've made sure the sh file is in the correct location.
But if I copy the bootstrap.sh file inside the container, and then run the same docker exec cmd, it's working fine. What am I missing here? I've tried with a simple script with the following entries, but it throws the same error:
#!/bin/bash
echo "Hello"
The doc clearly says:
Kernels are installed within the Docker container. The easiest way to
accomplish this is to create a bash script with installation commands,
save it to the master node, and then use the sudo docker exec
jupyterhub script_name command to run the script within the jupyterhub
container.
The docker exec command runs a command within the container's namespaces. One of those namespaces is the filesystem. So unless the command is part of the image, written into the container directly, or you have mounted a host volume to map a host directory into the container, you won't be able to execute it. A host volume could look like:
docker run -v /host/scripts:/container/scripts -n your_container $your_image
docker exec -it your_container /container/scripts/test.sh
That host volume could be the same path on both the host and the container.
If it is a shell script, you could use I/O redirection, e.g.:
docker exec -i $container_id /bin/bash <local_script.sh
but be aware that you cannot do interactive stuff this way since the script content has replaced your terminal as stdin. This works because the shell inside the container is just processing commands from stdin.
Other than those scenarios, I don't know what to tell you other than the documentation from AWS appears to be wrong.
I have bash script that performing some Docker commands:
#!/usr/bin/env bash
echo "Create and start database"
cd ../../database
cp -R ../../../scripts/db db/
docker build -t a_database:1 .
docker run --rm --name a_db -e POSTGRES_PASSWORD=docker -d -p 5432:5432 a_database:1
docker network connect --ip 172.23.0.5 a_network a_db
sleep 15
echo "Initialize database"
docker exec a_db /root/db/dev/init_db.sh
echo "Cleanup"
rm -rf db
On mac everything works fine, problem occurs when I try to start this script on windows machine. When I'm running it I receive an error:
OCI runtime exec failed: exec failed: container_linux.go:344: starting container process caused "exec: \"C:/Program Files/Git/root/db/dev/init_db.sh\": stat C:/Program Files/Git/root/db/dev/init_db.sh: no such file or directory": unknown
Directory and script (/root/db/dev/init_db.sh) exist inside docker container. I don't know why it tries to find script on host machine? Also when I perform command:
docker exec a_db /root/db/dev/init_db.sh
directly in command line (on windows) script is executed. Any idea what is wrong and why it's trying to use git ?
I had a similar problem... absolute paths with windows variables fixed mine:
$HOME/docker/...
Thanks to igaul answer I was able to run this on windows machine. There were two problems:
Path to script in docker container. Instead of:
docker exec a_db /root/db/dev/init_db.sh
should be:
docker exec a_db root/db/dev/init_db.sh
Line endings in init_db.sh. On windows machine after pulling repository from bitbucket line ending of init_db.sh was setup to CRLF what caused problem. I've added .gitattribute file to my repo and now init_db.sh file always has LF endings.
It's not a bug in Docker, but the way mingw handles these paths. Here is some more information about that "feature"; http://www.mingw.org/wiki/Posix_path_conversion. Prefixing the path with a double slash (//bin/bash) should prevent this, or you can set MSYS_NO_PATHCONV=1, see How to stop MinGW and MSYS from mangling path names given at the command line
it's been a few days now but I really can't understand how to run a bash script correctly in ubuntu/xenial64 using docker. Any clarifications will be very appreciated.
I created a Dockerfile like this
FROM ubuntu:16.04
COPY setup.sh /setup.sh
RUN chmod +x /setup.sh
ENTRYPOINT [ "/setup.sh" ]
The error returned is: standard_init_linux.go:195: exec user process caused "no such file or directory"
But why?? If I run ls the file is correctly placed on the root. I also tried using CMD ["/setup.sh"]. My script file has a shebang like this #!/bin/bash.
So I am running a script that calls:
docker-compose run --rm web sh /data/bin/install_test_db.sh
and it seems to work fine when running it on ubuntu but when I run it on Windows using Git Bash I get this error:
sh: 0: Can't open C:/Program Files/Git/data/bin/install_test_db.sh
It seems to be trying to run the script on my machine and not the actual container from which I am trying to call it from.
Any ideas on why this is happening?
First, try passing a command to your sh shell:
docker-compose run --rm web sh -c "/data/bin/install_test_db.sh"
Or:
docker-compose run --rm web "sh -c /data/bin/install_test_db.sh"
That will avoid your host shell (the git bash) to interpret an absolute path (starting with '/') as one from the Git installation path.
That will keep /data/... as an argument to be passed to the shell executed in the container.
Note: If you are using Docker for Windows (meaning not VirtualBox, but HyperV), you don't need git bash at all.
Try the same command from a regular CMD (with docker.exe in your %PATH%, which Docker for Windows set for you)
vonc#VONCAVN7 C:\Users\vonc
> where docker
C:\Program Files\Docker\Docker\Resources\bin\docker.exe
If you need Linux-like commands from that same CMD session, then yes, add git paths:
set GH=C:\path\to\git
set PATH=%GH%\bin;%GH%\usr\bin;%GH%\mingw64\bin;%PATH%