I have the following Dockerfile:
# => Build container
FROM node:14-alpine AS builder
WORKDIR /app
# split the dependecies from our source code so docker builds can cache this step
# (unless we actually change dependencies)
COPY package.json yarn.lock ./
RUN yarn install
COPY . ./
RUN yarn build
# => Run container
FROM nginx:alpine
# Nginx config
WORKDIR /usr/share/nginx/html
RUN rm -rf ./*
COPY ./nginx/nginx.conf /etc/nginx/conf.d/default.conf
# Default port exposure
EXPOSE 8080
# Copy .env file and shell script to container
COPY --from=builder /app/dist .
COPY ./env.sh .
COPY .env .
USER root
# Add bash
RUN apk update && apk add --no-cache bash
# Make our shell script executable
RUN chmod +x env.sh
# Start Nginx server
CMD ["/bin/bash", "-c", "/usr/share/nginx/html/env.sh && nginx -g \"daemon off;\""]
But when I try to run it I get the following errors when trying to add bash:
------
> [stage-1 8/9] RUN apk update && apk add --no-cache bash:
#19 0.294 fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/main/x86_64/APKINDEX.tar.gz
#19 0.358 140186175183688:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914:
#19 0.360 fetch https://dl-cdn.alpinelinux.org/alpine/v3.14/community/x86_64/APKINDEX.tar.gz
#19 0.360 ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.14/main: Permission denied
#19 0.360 WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.14/main: No such file or directory
#19 0.405 140186175183688:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:1914:
#19 0.407 ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.14/community: Permission denied
#19 0.407 WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.14/community: No such file or directory
#19 0.408 2 errors; 42 distinct packages available
------
executor failed running [/bin/sh -c apk update && apk add --no-cache bash]: exit code: 2
Which I need to add to execute this script:
#!/bin/bash
# Recreate config file
rm -rf ./env-config.js
touch ./env-config.js
# Add assignment
echo "window._env_ = {" >> ./env-config.js
# Read each line in .env file
# Each line represents key=value pairs
while read -r line || [[ -n "$line" ]];
do
# Split env variables by character `=`
if printf '%s\n' "$line" | grep -q -e '='; then
varname=$(printf '%s\n' "$line" | sed -e 's/=.*//')
varvalue=$(printf '%s\n' "$line" | sed -e 's/^[^=]*=//')
fi
# Read value of current variable if exists as Environment variable
value=$(printf '%s\n' "${!varname}")
# Otherwise use value from .env file
[[ -z $value ]] && value=${varvalue}
# Append configuration property to JS file
echo " $varname: \"$value\"," >> ./env-config.js
done < .env
echo "}" >> ./env-config.js
From what I have read adding USER root should fix this but it does not in this case.
Any ideas on how to fix?
You should be able to use /bin/sh as a standard Bourne shell; also, you should be able to avoid the sh -c wrapper in the CMD line.
First, rewrite the script using POSIX shell syntax. Scanning over the script, it seems like it is almost okay as-is; change the first line to #!/bin/sh, and correct the non-standard ${!varname} expansion (also see Dynamic variable names in Bash)
# Read value of current variable if exists as Environment variable
value=$(sh -c "echo \$$varname")
# Otherwise use value from .env file
[[ -z $value ]] && value=${varvalue}
You can try testing it using an alpine or busybox image with a more restricted shell, or setting the POSIXLY_CORRECT environment variable with bash.
Secondly, there's a reasonably standard pattern of using ENTRYPOINT and CMD together. The CMD gets passed as arguments to the ENTRYPOINT, so if the ENTRYPOINT ends with exec "$#", it will replace itself with that command.
#!/bin/sh
# ^^ not bash
# Recreate config file
...
echo "window._env_ = {" >> ./env-config.js
...
echo "}" >> ./env-config.js
# Run the main container CMD
exec "$#"
Now, in your Dockerfile, you don't need to install GNU bash, because you're not using it, but you do need to correctly split out the ENTRYPOINT and CMD.
ENTRYPOINT ["/usr/share/nginx/html/env.sh"] # must be JSON-array syntax
CMD ["nginx", "-g", "daemon off;"] # can be either syntax
As an aside, the cmd1 && cmd2 syntax is basic shell syntax and not a bash extension, so you could write CMD ["/bin/sh", "-c", "cmd1 && cmd2"]; but, if you write a command without the JSON array syntax, Docker will insert the sh -c wrapper for you. You should almost never need to write out sh -c in a Dockerfile.
Related
I need to split a comma-separated string into array and run k6 for each of the array values parallely. Since shell script doesn't support arrays, I m using bash command in the script. I am not able to run it as bash using Dockerfile in TeamCity.
Dockerfile:
FROM loadimpact/k6:0.34.1
COPY ./src/lib /lib
COPY ./src/scenarios /scenarios
COPY ./src/k6-run-all.sh /k6-run-all.sh
WORKDIR /
ENTRYPOINT []
RUN bash -c "./k6-run-all.sh"
Shell script:
#!/bin/bash
K6_RUN_OPTIONS=${K6_RUN_OPTIONS}
ENV_NAME=${ENV_NAME:-qa}
IS_TEST_RUN=${IS_TEST_RUN:-true}
SCENARIO_NAME=${SCENARIO_NAME:-"full-card-visa"}
GWC_PC_ID=${GWC_PC_ID}
IFS=',' read -r -a PCIds <<< "$GWC_PC_ID"
echo "Number of PC ids provided in environment variables=" ${#PCIds[#]}
if [[ ${#PCIds[#]} > 0 ]]; then
for pcId in "$#"
do
ENV_NAME=$ENV_NAME RUN_OPTIONS=$SCENARIO_NAME-$ENV_NAME$OPTIONS_VARIANT GWC_PC_ID=$pcId k6 run $K6_RUN_OPTIONS ''$SCENARIO/index.js'' &
done
fi
existCode=$?
if [ $existCode -ne 0 ]; then
echo "Scenario $SCENARIO_NAME completed with the error"
exit $existCode
fi
Error:
#9 [5/6] RUN bash -c "./k6-run-all.sh"
17:02:02 #9 0.356 /bin/sh: bash: not found
17:02:02 #9 ERROR: executor failed running [/bin/sh -c bash -c "./k6-run-all.sh"]: exit code: 127
17:02:02 ------
17:02:02 > [5/6] RUN bash -c "./k6-run-all.sh":
17:02:02 #9 0.356 /bin/sh: bash: not found
17:02:02 ------
17:02:02 failed to solve: executor failed running [/bin/sh -c bash -c "./k6-run-all.sh"]: exit code: 127
How to modify Dockerfile or shell script to run this shell script as bash?
Previously to run it as bash script the last line of Dockerfile used to be:
CMD ["sh", "-c", "./k6-run-all.sh"]
******* Edit: **********
Updated full script after knittl's answer (current issue is after adding & for parallel runs it is not working, it is not running anything inside the for loop and it is not giving any extra error or information in logs, it is like it is skipping it):
K6_RUN_OPTIONS=${K6_RUN_OPTIONS}
ENV_NAME=${ENV_NAME:-qa}
IS_TEST_RUN=${IS_TEST_RUN:-true}
SCENARIO_NAME=${SCENARIO_NAME:-"full-card-visa"}
GWC_PC_ID=${GWC_PC_ID}
OPTIONS_VARIANT=""
if $IS_TEST_RUN; then
OPTIONS_VARIANT="-test"
fi
SCENARIO_DIR="$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
SCENARIO_PATH="${SCENARIO_DIR}/scenarios"
SCENARIO="${SCENARIO_PATH}/${SCENARIO_NAME}"
echo "Executing scenario path $SCENARIO"
SCENARIO_NAME=${SCENARIO:${#SCENARIO_PATH}+1:${#SCENARIO}}
echo "Scenario Name: $SCENARIO_NAME"
echo "Run option: $SCENARIO_NAME-$ENV_NAME$OPTIONS_VARIANT"
echo "pc ids provided in environment variable=" $GWC_PC_ID
if [ -z "$GWC_PC_ID" ]
then
ENV_NAME=$ENV_NAME RUN_OPTIONS=$SCENARIO_NAME-$ENV_NAME$OPTIONS_VARIANT k6 run $K6_RUN_OPTIONS ''$SCENARIO/index.js''
else
for pcId in $(printf '%s' "$GWC_PC_ID" | tr , ' ');
do
ENV_NAME=$ENV_NAME RUN_OPTIONS=$SCENARIO_NAME-$ENV_NAME$OPTIONS_VARIANT GWC_PC_ID=$pcId k6 run $K6_RUN_OPTIONS ''$SCENARIO/index.js'' &
done
fi
existCode=$?
if [ $existCode -ne 0 ]; then
echo "Scenario $SCENARIO_NAME completed with the error"
exit $existCode
fi
k6 Docker containers do not come with bash preinstalled, but with busybox. I see two options:
Create your own Docker image based off grafana/k6 and manually install bash in your image.
Rewrite your script to not rely on bashisms. Should be fairly easy: split your list of tests to run into one path per line and while read -r path; do …; done them.
Or if support for whitespace in filenames is not required, then for path in $(printf '%s' "$GWC_PC_ID" | tr , ' '); do …; done
Note that your current script will return with the exit code of your last k6 process, meaning that if any other test failed but the last one was successfull, that will mask the error.
PS. Time to upgrade your base Docker image too. loadimpact/k6:0.34.1 is really old (exactly 1 year). It's better to switch to grafana/k6:0.40.0, which was released a week ago.
I have a Dockerfile, where I am finding myself constantly needing to call source /opt/ros/noetic/setup.bash.
e.g.:
RUN source /opt/ros/noetic/setup.bash \
&& SOME_COMMAND
RUN source /opt/ros/noetic/setup.bash \
&& SOME_OTHER_COMMAND
Is there a method to have this initialised in every RUN call in a Dockerfile?
Have tried adding to ~/.bash_profile and Docker's ENV command with no luck.
TL;DR: what you want is feasible by copying your .sh script in /etc/profile.d/ and using the SHELL Dockerfile command to tweak the default shell.
Details:
To start with, consider the following sample script setup.sh:
#!/bin/sh
echo "# setup.sh"
ENV_VAR="some value"
some_fun() {
echo "## some_fun"
}
Then, it can be noted that bash provides the --login CLI option:
When Bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable. The --noprofile option may be used when the shell is started to inhibit this behavior.
When an interactive login shell exits, or a non-interactive login shell executes the exit builtin command, Bash reads and executes commands from the file ~/.bash_logout, if it exists.
− https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Bash-Startup-Files
Furthermore, instead of appending the setup.sh code in /etc/profile, you can take advantage of the /etc/profile.d folder that is read in the following way by most distributions:
$ docker run --rm -i debian:10 cat /etc/profile | tail -n 9
if [ -d /etc/profile.d ]; then
for i in /etc/profile.d/*.sh; do
if [ -r $i ]; then
. $i
fi
done
unset i
fi
Note in particular that the .sh extension is mandatory, hence the naming of the minimal-working-example above: setup.sh (not setup.bash).
Finally, it is possible to rely on the SHELL command to replace the default shell used by RUN (in place of ["/bin/sh", "-c"]) to incorporate the --login option of bash.
Concretely, you could phrase your Dockerfile like this:
FROM debian:10
# WORKDIR /opt/ros/noetic
# COPY setup.sh .
# RUN . /opt/ros/noetic/setup.sh && echo "ENV_VAR=$ENV_VAR"
# empty var here
RUN echo "ENV_VAR=$ENV_VAR"
# enable the extra shell init code
COPY setup.sh /etc/profile.d/
SHELL ["/bin/bash", "--login", "-c"]
# nonempty var and function
RUN echo "ENV_VAR=$ENV_VAR" && some_fun
# DISABLE the extra shell init code!
RUN rm /etc/profile.d/setup.sh
# empty var here
RUN echo "ENV_VAR=$ENV_VAR"
Outcome:
$ docker build -t test .
Sending build context to Docker daemon 6.144kB
Step 1/7 : FROM debian:10
---> ef05c61d5112
Step 2/7 : RUN echo "ENV_VAR=$ENV_VAR"
---> Running in 87b5c589ec60
ENV_VAR=
Removing intermediate container 87b5c589ec60
---> 6fdb70be76f9
Step 3/7 : COPY setup.sh /etc/profile.d/
---> e6aab4ebf9ef
Step 4/7 : SHELL ["/bin/bash", "--login", "-c"]
---> Running in d73b0d13df23
Removing intermediate container d73b0d13df23
---> ccbe789dc36d
Step 5/7 : RUN echo "ENV_VAR=$ENV_VAR" && some_fun
---> Running in 42fd1ae14c17
# setup.sh
ENV_VAR=some value
## some_fun
Removing intermediate container 42fd1ae14c17
---> de74831896a4
Step 6/7 : RUN rm /etc/profile.d/setup.sh
---> Running in bdd969a63def
# setup.sh
Removing intermediate container bdd969a63def
---> 5453be3271e5
Step 7/7 : RUN echo "ENV_VAR=$ENV_VAR"
---> Running in 0712cea427f1
ENV_VAR=
Removing intermediate container 0712cea427f1
---> 216a421f5659
Successfully built 216a421f5659
Successfully tagged test:latest
This question already has answers here:
Are shell scripts sensitive to encoding and line endings?
(14 answers)
Closed 3 years ago.
So I have a relatively simple bash script:
#!/bin/bash
declare ALL="";
while IFS="" read -r line || [ -n "$line" ]
do
if [[ $line == 'ENV ms'* ]]; then
words=( $line )
if [[ ${#ALL} > 0 ]]; then
ALL="$ALL;${words[1]}=${words[2]}"
else
ALL="${words[1]}=${words[2]}"
fi
if [[ ${#ALL} > 0 ]]; then
printf '%s\n' "$ALL"
echo "${#ALL}"
fi
fi
done < Dockerfile
echo "$ALL"
echo "${#ALL}"
parsing a Dockerfile that looks like this:
#
# Configuration Stage
#
FROM maven:3.6.1-jdk-12 AS build
ENV HOME=/usr/local/ms-cards
RUN mkdir -p $HOME
WORKDIR $HOME
COPY maven-settings.xml /root/.m2/settings.xml
COPY pom.xml $HOME
RUN mvn -Dmaven.test.skip=true clean verify --fail-never
COPY . $HOME
RUN mvn -Dmaven.test.skip=true clean package
#
# Package stage
#
FROM openjdk:12
COPY --from=build /usr/local/ms-cards/target/ms-cards-1.0-exec.jar /usr/local/lib/ms-cards.jar
ENV ms_oauth_ip ms-oauth
ENV ms_oauth_port 48001
ENV ms_cards_client_id clientapp
ENV ms_cards_client_secret 123456
ENV ms_cards_port 48002
ENV ms_connection_port 48003
ENV ms_connection_ip ms-connection
EXPOSE 48002
ENTRYPOINT ["java","-jar","/usr/local/lib/ms-cards.jar"]
and it gives me this output:
ms_oauth_ip=ms-oauth
21
;ms_oauth_port=48001
42
;ms_cards_client_id=clientapp
72
;ms_cards_client_secret=123456
103
;ms_cards_port=48002ret=123456
124
;ms_connection_port=4800323456
150
;ms_connection_ip=ms-connection
182
;ms_connection_ip=ms-connection
182
So I can see that my ALL variable is growing in length...but when printf'ing it never seems to be growing...Anyone know what I'm doing wrong?
The Dockerfile has Windows \r\n line endings. Each \r that is printed causes the cursor to jump back to the beginning of the line and overwrite the previous setting.
Debugging tip: Use declare -p var to see exactly what's in a variable.
I'm looking for a simple command that would allow me to create a terminal alias to another command from the command line and preferably use it without having to restart my terminal.
The use case is as follows, I just created a small bash script and put it on gist.
# Babel in your project
npm i json -g
npm install --save-dev babel-core babel-cli babel-preset-es2015 babel-preset-react
json -f package.json -I -e 'this.scripts.test = "mocha --compilers js:babel-core/register --recursive"'
json -f package.json -I -e 'this.babel = {}'
json -f package.json -I -e 'this.babel.presets = ["es2015", "react"]'
json -f package.json -I -e 'this.main = "./lib/index"'
json -f package.json -I -e 'this.esnext = "./src/index"'
I can run it here from the terminal like this:
bash -c "$(curl -fsSL https://gist.githubusercontent.com/reggi/8035dcbdf0fb73b8c8703a4d244f15cf/raw/767ec8c2fb54b554ce122cc85953da5b277dbaf4/babel-ready.sh)"
I'm curious if there's an easily way to create an alias from the terminal. Something simple like:
add-alias babel-ready -- bash -c "$(curl -fsSL https://gist.githubusercontent.com/reggi/8035dcbdf0fb73b8c8703a4d244f15cf/raw/767ec8c2fb54b554ce122cc85953da5b277dbaf4/babel-ready.sh)"
A rough attempt (not flow-control aware, not idempotent, not able to recognize and remove old versions of the same alias) might look something like this:
# non-POSIX-compliant function keyword used to allow dash in function name
function add-alias() {
# declare locals explicitly to prevent scope leakage
local content_line cmd
(( $# == 1 )) && [[ $1 ]] || {
echo "Usage: add_alias name" >&2
return 1
}
IFS= read -r content_line
# reject inputs containing newlines
content_line=${content_line%$'\n'}
if [[ $content_line = *$'\n'* ]]; then
echo "ERROR: Only one line may be provided" >&2
return 1
fi
# generate a command
printf -v cmd "alias %q=%q" "$1" "$content_line"
# eval it in the local shell, and add to the rc if that succeeds
eval "$cmd" && printf '%s\n' "$cmd" >>"$HOME/.bashrc"
}
...used as:
add-alias babel-ready <<'EOF'
bash -c "$(curl -fsSL https://gist.githubusercontent.com/reggi/8035dcbdf0fb73b8c8703a4d244f15cf/raw/767ec8c2fb54b554ce122cc85953da5b277dbaf4/babel-ready.sh)"
EOF
Note the use of <<'EOF' -- quoting the heredoc is important, as this prevents expansions from being run when the alias is being defined (as opposed to when it's run). This is why usage is passing code on stdin rather than on the command line, which would make it the user's responsibility to quote command line arguments appropriately.
I have a script called autoinstall:
#!/usr/bin/sh
echo "Installasi membutuhkan free space minimal 2MB, pastikan ada punya cukup space di router anda"
read -p "Anda yakin ingin melanjutkan installasi?(y/n) " -n 1 -r
echo ""
if [[ $REPLY = ^[Yy]$ ]]
then
cd /
cd /tmp/
tar -xvf OpenWrt_Angel_Beats_Edition_v1.3.3.tar -C /
chmod -R 744 /root/crt
chmod 744 /www/wget/wget_download.sh
chmod 744 /usr/bin/gsm
chmod 744 /usr/bin/profile
opkg update && opkg install elinks
cp /etc/rc.local /etc/rc.local.backup
cat > /etc/rc.local << END
#!bin/sh
# /etc/rc.local: Local system initialization script.
#
# Put any local startup commands in here. Also, if you have
# anything that needs to be run at shutdown time you can
# make an /etc/rc.d/rc.local_shutdown script and put those
# commands in there.
sh /www/wget/wget_download.sh > /dev/null 2>&1 &
exit 0
END
killall sh /www/wget/wget_download.sh
sh /www/wget/wget_download.sh > /dev/null 2>&1 &
echo "File backup /etc/rc.local.backup telah dibuat, gunakan file ini untuk mengembalikan konfigurasi rc.local anda yang dulu jika diperlukan"
echo "Installasi selesai. Jangan lupa di akun openvpn yang digunakan (/root/crt/xxx.ovpn) tambahkan baris ini:
script-security 2
up client-connect.sh"
else
echo ""
echo "Installasi dibatalkan"
fi
Every command that I put in the first line always gets the error above (line 1:xxx not found) and I'm sure I've typed in the correct command, even echo gives the error like that, how do I solve this?
There can be two problems here:
The file doesn't exist. Usually, for sh, the path is /bin/sh, so it should be #!/bin/sh
You're editing the file on Windows. Windows uses CR+LF as line ending. Unix (and Linux) uses just LF. So for Linux, the command reads "execute /bin/sh<CR> and sh<CR> doesn't exist.
Solution: When editing the file, make sure you use Unix line endings.
The file might have been edited with an editor that insert a Unicode BOM (Byte Order Mark).
Have a look to the first line contents with:
od -c autoinstall | head -1
or
hd -n 16 autoinstall
If you see unexpected characters before #!/usr/bin/sh, you might try one of the methods described here Using awk to remove the Byte-order mark to remove the BOM.