On my computer bash starts and lasts on docker:git.
~ # bash
bash-4.4# ps
PID USER TIME COMMAND
1 root 0:00 sh
26 root 0:00 bash
32 root 0:00 ps
bash-4.4# echo $0
bash
bash-4.4# echo $SHELL
/bin/ash
ash seems a bit uneasy but I'm able to run #!/bin/bash file so it's fine so far.
However, on Gitlab CI on gitlab.com, command bash doesn't return anything but it doesn't seem to keep running. Why is this?
$ apk add --update bash
$ bash
$ ps && pwd && echo $0 && echo $SHELL && ls /bin
PID USER TIME COMMAND
1 root 0:00 /bin/sh
10 root 0:00 /bin/sh
25 root 0:00 ps
/builds/230s/industrial_calibration
/bin/sh
ash
base64
bash
bashbug
:
More detailed output on my computer:
$ lsb_release -a|grep Description
No LSB modules are available.
Description: Ubuntu 16.04.4 LTS
$ docker pull docker:git
$ docker images | grep -i docker
docker git 5c58d1939c5d 10 days ago 152MB
$ docker run -it docker:git
~ # apk add --update bash
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
:
(10/10) Installing tar (1.29-r1)
Executing busybox-1.27.2-r11.trigger
OK: 37 MiB in 31 packages
~ # bash
bash-4.4# ps
PID USER TIME COMMAND
1 root 0:00 sh
26 root 0:00 bash
32 root 0:00 ps
bash-4.4# echo $0
bash
bash-4.4# echo $SHELL
/bin/ash
.gitlab-ci.yml used (this fails at the last line as the passed file uses bash specific syntax):
image: docker:git
before_script:
- apk add --update bash coreutils tar # install industrial_ci depedencies
- bash
- git clone https://github.com/plusone-robotics/industrial_ci.git .ci_config -b gitlab_modularize
- ps && pwd && echo $0 && echo $SHELL && ls /bin
- source ./.ci_config/industrial_ci/src/tests/gitlab_module.sh
UPDATE: Sourcing bash-based file in bash -c indeed works, but it's probably not useful to me as what I really want is to use a function defined in that file and because bash -c line terminates and doesn't carry the context, the function won't be available in the later lines IMO.
- /bin/bash -c "source ./.ci_config/industrial_ci/src/tests/gitlab_module.sh"
- pass_sshkey_docker
image: alpine
before_script:
- apk add --update bash coreutils tar
- bash
- echo smth
Now imagine your are the computer. You wait for each command to finish before executing the next one, you don't use the keyboard. So what do you do? Let's try it with alpine, substituting newline with ;:
$ docker run -ti --rm alpine sh -c 'apk add --update bash; bash; echo smth'
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
(1/6) Installing pkgconf (1.3.10-r0)
(2/6) Installing ncurses-terminfo-base (6.0_p20171125-r0)
(3/6) Installing ncurses-terminfo (6.0_p20171125-r0)
(4/6) Installing ncurses-libs (6.0_p20171125-r0)
(5/6) Installing readline (7.0.003-r0)
(6/6) Installing bash (4.4.19-r1)
Executing bash-4.4.19-r1.post-install
Executing busybox-1.27.2-r7.trigger
OK: 13 MiB in 17 packages
bash-4.4#
YOU DON'T TOUCH THE KEYBOARD. You can wait endlessly for the bash-4.4# line to disapear as bash will wait endlessly for you to type anything. The command echo smth will never execute, gitlab will timeout waiting for bash to end, the end.
Now if you want to execute something in alpine using bash using gitlab-ci i suggest doing it this way: create a executable script ci-myscript.sh that you git add&commit to your repo:
$ cat ci-myscript.sh
#!/bin/bash
git clone https://github.com/plusone-robotics/industrial_ci.git .ci_config -b gitlab_modularize
ps && pwd && echo $0 && echo $SHELL && ls /bin
source ./.ci_config/industrial_ci/src/tests/gitlab_module.sh
The first line #!/bin/bash tells that shell should execute this script under bash. Now from your gitlab-ci you run:
image: docker:git
before_script:
- apk add --update bash coreutils tar
- ./ci-myscript-sh
Creating such scripts is actually a good workflow, because you can test the script locally on your computer before testing it in gitlab-ci.
The other option is to single call bash as suggested by #Mazel in comments:
image: docker:git
before_script:
- apk add --update bash coreutils tar
- bash -c 'git clone https://github.com/plusone-robotics/industrial_ci.git .ci_config -b gitlab_modularize; ps && pwd && echo $0 && echo $SHELL && ls /bin; source ./.ci_config/industrial_ci/src/tests/gitlab_module.sh'
That way you need to call everything in single line, because the next line won't have the same enviroment as the previous line.
Related
I built a container using the docker-compose script below:
services:
client:
image: alpine
environment:
- BACKUP_ENABLED=1
- BACKUP_INTERVAL=60
- BACKUP_PATH=/data
- BACKUP_FILENAME=db_backup
networks:
- dbnet
entrypoint: |
sh -c 'sh -s << EOF
apk add --no-cache mysql-client
while true
do
then
sleep $$BACKUP_INTERVAL
echo "$$(date +%FT%H.%M) - Making Backup to : $$BACKUP_PATH/$$(date +%F)/$$BACKUP_FILENAME-$$(date +%FT%H.%M).sql.gz"
mysqldump -u root -ppassword -h dblb --all-databases | gzip > $$BACKUP_PATH/$$(date +%F)/$$BACKUP_FILENAME-$$(date +%FT%H.%M).sql.gz
done
EOF'
But I encounter an issue where the date won't be updated and causes the loop keep backup to the same created file.
Every 60s the log will some the same date value. Here the container's log:
The same thing happened when I tried to manually write the script inside the container:
The timestamp always displays correctly when I only type date inside the container console.
Why won't the date update? What did I miss in the script?
Why won't the date update?
Because it is expanded by outer shell. Compare a shell script:
#!/bin/sh
# in a script
# this is running inside a shell
cat <<EOF # cat just prints data
$(date) # not cat, **but the shell**, expands $(date)
EOF
vs:
sh -c '
# this is running inside a shell
sh -s <<EOF # sh -s executes input data
echo $(date) # not sh -s, but **the outer shell**, expands $(date). ONCE
EOF
'
That sh -c sh -s and entrypoint is all unorthodox, just run the command that you want to run.
command:
- sh
- -c
- |
apk add --no-cache mysql-client
while sleep $$BACKUP_INTERVAL; do
echo "$$(date +%FT%H.%M) - Making Backup to : $$BACKUP_PATH/$$(date +%F)/$$BACKUP_FILENAME-$$(date +%FT%H.%M).sql.gz"
done
When using bash to execute "docker exec" commands, I am trying to install nvm inside the Docker container. Those are the following commands inside the bash script:
Update:
I've added the whole script.
#!/bin/bash
ubuntu_version="ubuntu:$1"
node_version=$2
container_name="ubuntu_container"
green=`tput setaf 2`
reset=`tput sgr0`
red=`tput setaf 1`
echo "${green}Pulling Ubuntu version: $ubuntu_version ${reset}"
docker pull $ubuntu_version
echo "${green}Running Ubuntu in background...${reset}"
# Stop and remove an existing docker container
docker stop $container_name
docker rm $container_name
docker run -d --name $container_name --rm $ubuntu_version sleep inf
echo "${green}Updating Ubuntu...${reset}"
docker exec $container_name apt update
docker exec $container_name apt upgrade
echo "${green}Installing curl${reset}"
docker exec $container_name apt install -y curl
echo "${green}Installing nvm: ${red}$node_version${reset}"
docker exec -it $container_name bash -c "curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash"
docker exec $container_name bash -c ". ~/.bashrc; nvm"
This leads to the following output:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 15037 100 15037 0 0 54481 0 --:--:-- --:--:-- --:--:-- 54481
=> Downloading nvm as script to '/root/.nvm'
=> Appending nvm source string to /root/.bashrc
=> Appending bash_completion source string to /root/.bashrc
=> Close and reopen your terminal to start using nvm or run the following to use it now:
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh" # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion" # This loads nvm bash_completion
OCI runtime exec failed: exec failed: container_linux.go:380: starting container process caused: exec: "nvm": executable file not found in $PATH: unknown
Seems like nvm is either not in the path or not installed. What am I missing here?
The problem is with the third docker exec.
You have two options
Run :
docker exec -it test_container bash
in bash session, you can run nvm
Or
Run :
docker exec test_container bash -c '. ~/.bashrc; nvm'
Update
Regarding 1), I thought(wrongly) you were running your docker exec manually.
Regarding 2), save follwoing in test.sh :
#!/bin/bash
docker run -d --name test_container2 --rm node sleep inf
docker exec test_container2 apt install -y curl
docker exec -it test_container2 bash -c "curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.1/install.sh | bash"
docker exec test_container2 bash -c '. ~/.bashrc; nvm'
and run bash test.sh
When I run it, I don't have nvm: command not found
Update 2
Your version of ~/.bashrc contains a test on $PS1, so a workaround, change the last line to :
docker exec $container_name bash -c "PS1=x; . ~/.bashrc; nvm"
How do I make this work?
[alan#stormfather-0be642-default-1 ~]$ kubectl exec -it my-pod-0 -- bash -c "/bin/bash && cd /tmp"
[root#my-pod-0 /]# pwd
/
Change directory first and then sh into it.
kubectl exec -it my-pod-0 -- bash -c "cd /tmp && /bin/bash"
Mohsin Amjad's answer is both simple and correct, if you are getting the
..."bash": executable file not found in $PATH...
error, this just means the container inside the pod does not have bash installed, instead try sh or other shells. I.e. something like:
kubectl exec -it my-pod-0 -- sh -c "cd /tmp && echo $0 $SHELL"
I have running docker ubuntu container with just a bash script inside. I want to start my application inside that container with docker exec like that:
docker exec -it 0b3fc9dd35f2 ./main.sh
Inside main script I want to run another application with nohup as this is a long running application:
#!/bin/bash
nohup ./java.sh &
#with this strange sleep the script is working
#sleep 1
echo `date` finish main >> /status.log
The java.sh script is as follow (for simplicity it is a dummy script):
#!/bin/bash
sleep 10
echo `date` finish java >> /status.log
The problem is that java.sh is killed immediately after docker exec returns. The question is why?
The only solution I found out is to add some dummy sleep 1 into the first script after nohup is started. Than second process is running fine. Do you have any ideas why it is like that?
[EDIT]
Second solution is to add some echo or trap command to java.sh script just before sleep. Than it works fine. Unfortunately I cannot use this workaround as instead of this script I have java process.
This is not an answer, but I still don't have the required reputation to comment.
I don't know why the nohup doesn't work. But I did a workaround that worked, using your ideas:
docker exec -ti running_container bash -c 'nohup ./main.sh &> output & sleep 1'
Okay, let's join two answers above :D
First rcmgleite say exactly right: use
-d
options to run process as 'detached' background.
And second (the most important!) if you run detached process, you don't needed nohup!
deploy_app.sh
#!/bin/bash
cd /opt/git/app
git pull
python3 setup.py install
python3 -u webui.py >> nohup.out
Execute this inside a container
docker exec -itd container_name bash -c "/opt/scripts/deploy_app.sh"
Check it
$ docker attach container_name
$ ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 11768 1940 pts/0 Ss Aug31 0:00 /bin/bash
root 887 0.4 0.0 11632 1396 pts/1 Ss+ 02:47 0:00 /bin/bash /opt/scripts/deploy_app
root 932 31.6 0.4 235288 32332 pts/1 Sl+ 02:47 0:00 python3 -u webui.py
I know this is a late response but I will add it here for documentation reasons.
When using nohup on bash and running it with 'exec' on a docker container, you should use
$ docker exec -d 0b3fc9dd35f2 /bin/bash -c "./main.sh"
The -d option means:
-d, --detach Detached mode: run command in the
background
for more information about docker exec, see:
https://docs.docker.com/engine/reference/commandline/exec/
This should do the trick.
This works:
# echo 1 and exit:
$ docker run -i -t image /bin/bash -c "echo 1"
1
# exit
# echo 1 and return shell in docker container:
$ docker run -i -t image /bin/bash -c "echo 1; /bin/bash"
1
root#4c064f2554de:/#
Question: How could I source a file into the shell? (this does not work)
$ docker run -i -t image /bin/bash -c "source <(curl -Ls git.io/apeepg) && /bin/bash"
# content from http://git.io/apeepg is sourced and shell is returned
root#4c064f2554de:/#
In my case, I use RUN source command (which will run using /bin/bash) in a Dockerfile to install nvm for node.js
Here is an example.
FROM ubuntu:14.04
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
...
...
RUN source ~/.nvm/nvm.sh && nvm install 0.11.14
I wanted something similar, and expanding a bit on your idea, came up with the following:
docker run -ti --rm ubuntu \
bash -c 'exec /bin/bash --rcfile /dev/fd/1001 \
1002<&0 \
<<<$(echo PS1=it_worked: ) \
1001<&0 \
0<&1002'
--rcfile /dev/fd/1001 will use that file descriptor's contents instead of .bashrc
1002<&0 saves stdin
<<<$(echo PS1=it_worked: ) puts PS1=it_worked: on stdin
1001<&0 moves this stdin to fd 1001, which we use as rcfile
0<&1002 restores the stdin that we saved initially
You can use .bashrc in interactive containers:
RUN curl -O git.io/apeepg.sh && \
echo 'source apeepg.sh' >> ~/.bashrc
Then just run as usual with docker run -it --rm some/image bash.
Note that this will only work with interactive containers.
I don't think you can do this, at least not right now. What you could do is modify your image, and add the file you want to source, like so:
FROM image
ADD my-file /my-file
RUN ["source", "/my-file", "&&", "/bin/bash"]