Script with docker-compose commands. A line is not executed - bash

My script looks like this:
#!/usr/bin/env bash
db_name="dh$1";
db_user="dh$1";
db_backup="dh$1-$2.sql";
docker stop $(docker ps -aq) \
&& docker start platform_db_$1_1 \
&& docker start platform_$1_1 \
&& docker cp $db_backup platform_db_$1_1:/$db_backup \
&& cd ../platform \
&& docker-compose exec db_$1 bash -c "dropdb -U $db_user $db_name && createdb -U $db_user $db_name && psql -U $db_user -d $db_name -f $db_backup" \
&& docker-compose exec $1 php artisan d:d:u \
&& docker-compose exec $1 php artisan settings:sync \
&& docker-compose up -d
Everything works great but the last line docker-compose up -d and probably the one before docker-compose exec $1 php artisan settings:sync don't get executed, and i have no idea why since there are no error messages. Any thoughts?
I believe the question is what can stop a script without error messages.

#Quasimodo for some reason the d:d:u command is stopping the script (after running successfuly). This is the handle method.
Anyways, i've set that command at the end since i don't need it before and all solved. Still I don't know why that happened.
public function handle()
{
$this->checkEnvironment();
$this->info('Pre-migrations');
Artisan::call('migrate', [
'--force' => true,
'--path' => 'database/pre-migrations',
]);
$this->info('Doctrine schema update');
Artisan::call('doctrine:schema:update');
$this->info('Migrations');
Artisan::call('migrate');
$this->info('Done');
}

Related

Environment variables not defined in SSH AuthorizedKeysCommand (Docker)

I'm trying to make the private key SSH connection with LDAP.
/etc/ssh/sshd_config
AuthorizedKeysCommand /etc/ldap_ssh_authorized_keys.sh
AuthorizedKeysCommandUser nobody
Script to get public keys from LDAP server
/etc/ldap_ssh_authorized_keys.sh
#!/bin/bash
USERSLIST=$( ldapsearch -x -D "${LDAP_USER}" -w "${LDAP_PASSWORD}" -H $LDAP_URI -b "${LDAP_BASEDN}" -s sub '(objectClass=posixAccount)' -u 'uid' \
grep '^uid:' | sed -n '/^ /{H;d};/uid:/x;$g;s/\n *//g;s/uid: //gp' \
)
while IFS= read -r line; do
exists=$(ldapsearch -x -D "${LDAP_USER}" -w "${LDAP_PASSWORD}" -H $LDAP_URI -b "${LDAP_BASEDN}" \
-s sub "(&(objectClass=posixGroup)(cn=sysadmin)(memberUid=${line}))" | grep "^# numEntries:")
if [[ ! -z $exists ]]
then
ldapsearch -x -D "${LDAP_USER}" -w "${LDAP_PASSWORD}" -H $LDAP_URI -b "${LDAP_BASEDN}" \
-s sub "(&(objectClass=posixAccount)(uid=${line}))" \
-u 'sshPublicKey' \
| sed -n '/^ /{H;d};/sshPublicKey:/x;$g;s/\n *//g;s/sshPublicKey: //gp'
echo -e "";
fi;
done <<< "$USERSLIST"
When I'm running script with /bin/bash it's working well and return my public keys.
All environment variables defined normally.
LDAP_URI
LDAP_BASEDN
LDAP_USER
LDAP_PASSWORD
The script also running normally when trying to make an SSH connection. But environment variables not available.
I'm trying also with AuthorizedKeysCommandUser as root. But nothing changed.
I solved this problem by getting the environment variables from /proc/1/environ.
Reference

/bin/sh: 1: Syntax error: Unterminated quoted string in dockerfile

I need to have variable inside file constants.js with quotes ('here variable'). Trying do it in dockerfile, but getting his error, can't figure it out. Tried escaping quotes, but its not working. Doing it with double quotes - works, but I need single. Any idea?
FROM debian:stretch-slim
ENV GIT_BRANCH=develop
ENV GIT_COMMIT_TAG=26bf94075e6d5f2c575a291680e905e15aa0c81f
COPY constants.js .
RUN { \
/bin/bash -c '{ \
sed -i -r "s/^([[:blank:]]*APP_VERSION:[[:blank:]]*).*/\1\'\${GIT_COMMIT_TAG:0:7}\',/" constants.js ; \
}' ; \
}
CMD ["sleep", "inf"]
File constants.js looks like:
const constants = {
APP_NAME: 'app',
APP_VERSION: '10.0.0',
};
module.exports = {
constants,
};
#UPDATE1
Longer
/bin/bash -c
in Dockerfile:
if [ ! -z "$GIT_BRANCH" ]; then \
GIT_BRANCH="${GIT_BRANCH}/" ; \
fi ; \
echo "${GIT_BRANCH}${GIT_COMMIT_TAG:0:7}" > .release.txt ; \
sed -i -r "s/^([[:blank:]]*APP_VERSION:[[:blank:]]*).*/\1'\${GIT_COMMIT_TAG:0:7}',/" constants.js ; \
appver=$(awk '"'"'/APP_VERSION/ { print $2 }'"'"' constants.js) ; \
echo "${appver:1:-2}" > .appversion ; \
so thats why I use bash -c.
You don't need to specify /bin/bash -c. Instead, just use RUN and immediately the command you need to execute. It is necessary to remove a few extra /. The final Dockerfile is given below
Dockerfile
FROM debian:stretch-slim
ENV GIT_BRANCH=develop
ENV GIT_COMMIT_TAG=26bf94075e6d5f2c575a291680e905e15aa0c81f
COPY constants.js .
RUN sed -i -r "s/^([[:blank:]]*APP_VERSION:[[:blank:]]*).*/\1\'$GIT_COMMIT_TAG:0:7',/" constants.js ;
CMD ["sleep", "inf"]
Resulted constants.js file
const constants = {
APP_NAME: 'app',
APP_VERSION: '26bf94075e6d5f2c575a291680e905e15aa0c81f:0:7',
};
module.exports = {
constants,
};
UPD #1
For execute some logic that does not fit on one line, the best solution would be to put it in a separate bash script.
Thus, you add setup.sh and change the Dockerfile
Dockerfile
FROM debian:stretch-slim
ENV GIT_BRANCH=develop
ENV GIT_COMMIT_TAG=26bf94075e6d5f2c575a291680e905e15aa0c81f
COPY ["constants.js", "setup.sh", "./"]
RUN chmod +x ./setup.sh && ./setup.sh
setup.sh
if [ ! -z "$GIT_BRANCH" ]; then
GIT_BRANCH="${GIT_BRANCH}/"
fi
CUTTED_GIT_COMMIT_TAG=$(echo "$GIT_COMMIT_TAG" | cut -c -7)
echo "${GIT_BRANCH}${CUTTED_GIT_COMMIT_TAG}" > .release.txt
sed -i -r "s/^([[:blank:]]*APP_VERSION:[[:blank:]]*).*/\1\'${CUTTED_GIT_COMMIT_TAG}',/" constants.js
APP_VERSION=$(awk '"'"'/APP_VERSION/ { print $2 }'"'"' constants.js)
CUTTED_APP_VERSION=$(echo "$appver" | cut -c 2- | rev | cut -c 1- | rev)
echo $CUTTED_APP_VERSION > .appversion
OK, so sed should looks like this:
sed -r -i "/APP_VERSION/ s/'\''(.*)'\''/'\''${GIT_COMMIT_TAG:0:7}'\''/" constants.js ; \
Then its working like it should.

How to rewrite grep in #!/bin/sh?

I have the following shell script, that does not compile:
#!/bin/sh
.....
.....
clean_up() {
echo "Clean up"
docker stop $KC_TEST_SVC
docker stop $KC_NAME
docker stop $POSTGRES_NAME
docker network rm $KC_NETWORK
}
# Check network if exists, if not then create
docker network inspect $KC_NETWORK
if [ $? -eq 1 ]; then
docker network create $KC_NETWORK
if [ $? -eq 1 ]; then
exit 1
fi
fi
docker run -d --rm --name $POSTGRES_NAME \
-e POSTGRES_DB=$POSTGRES_DB \
-e POSTGRES_USER=$POSTGRES_USER \
-e POSTGRES_PASSWORD=$POSTGRES_PW \
--network=$KC_NETWORK \
postgres:12.3
if [ $? -eq 1 ]; then
exit 1
fi
docker build --build-arg STAGE=int --tag $KC_TAG .
if [ $? -eq 1 ]; then
exit 1
fi
docker run -d --rm --name $KC_NAME \
-e DB_VENDOR=POSTGRES \
-e DB_ADDR=$POSTGRES_NAME \
-e DB_DATABASE=$POSTGRES_DB \
-e DB_USER=$POSTGRES_USER \
-e DB_PASSWORD=$POSTGRES_PW \
-e KEYCLOAK_USER=$KC_USER \
-e KEYCLOAK_PASSWORD=$KC_PW \
-e KEYCLOAK_LOGLEVEL=DEBUG \
--network=$KC_NETWORK \
$KC_TAG "-Dkeycloak.migration.action=import -Dkeycloak.migration.provider=dir -Dkeycloak.migration.dir=/opt/jboss/keycloak/import-dir -Dkeycloak.migration.strategy=OVERWRITE_EXISTING"
if [ $? -eq 1 ]; then
exit 1
fi
# Wait until Keycloak get started
sleep 30
# Run test jetty service that is protected with Keycloak
docker run -d --rm --name $KC_TEST_SVC \
-v $(pwd)/app:/app \
--network $KC_NETWORK \
hub.databaker.io/devops/jetty-keycloak:0.1.6
if [ $? -eq 1 ]; then
exit 1
fi
# Request Tokens for credentials
KC_URL=http://$KC_SERVER/$KC_CONTEXT/realms/$KC_REALM/protocol/openid-connect/token
echo "Keycloak URL => $KC_URL"
echo "Test the client connection to Keycloak with user $KC_TEST_USER"
KC_RESPONSE=$(
docker run --rm --network=$KC_NETWORK curlimages/curl:7.71.1 -X POST \
-d "username=$KC_TEST_USER" \
-d "password=$KC_TEST_PW" \
-d "grant_type=password" \
-d "client_id=$KC_TEST_CLIENT" \
$KC_URL | docker run --rm -i stedolan/jq .
)
echo "Response from Keycloak $KC_RESPONSE"
if grep -q "error" <<< "$KC_RESPONSE"; then
echo "++++++++Error+++++++++"
exit 1
fi
KC_ACCESS_TOKEN=$(echo "$KC_RESPONSE" | docker run --rm -i stedolan/jq -r .access_token)
echo "Access token from KC => $KC_ACCESS_TOKEN"
echo "Make request to protected service"
SVC_URL=http://$KC_TEST_SVC:8080/api/health
SVC_RES=$(docker run --rm --network=$KC_NETWORK curlimages/curl:7.71.1 -v -k \
-H "Authorization: Bearer $KC_ACCESS_TOKEN" \
-H "Accept: application/json" \
$SVC_URL | docker run --rm -i stedolan/jq .status)
clean_up
echo "$SVC_RES"
if [ "$SVC_RES" != "I am healthy" ]; then
echo "Test failed."
exit 1
else
echo "Test was successful. The service response $SVC_RES."
fi
it complains:
Syntax error: redirection unexpected
due to of the following line:
if grep -q "error" <<< "$KC_RESPONSE"; then
echo "++++++++Error+++++++++"
exit 1
fi
#!/bin/sh does not support grep. How to rewrite it?
<<< is a here string and it's a bash extension. In posix shell, just pipe the data.
if printf "%s\n" "$KC_RESPONSE" | grep -q "error"; then
Do not do if [ $? -eq 1 ]; then, it's error prone. Do if ! command; then. (You might want to research set -eu).
Change #!/bin/sh to #!/bin/bash (ev. replace bash location, which bash helps to find location of bash, and to see if bash is installed), or you may need to rewrite your script in a portable way. In this case, create a temporary file, put the string in such file, redirect such file to grep (with single <), and then delete the temporary file.
With such long description, you see why there is an extension in bash, on the other hand, it is nearly a syntactic sugar: you can easily do the same.

Getting error at the time of run .sh script from DockerFile

My DockerFile is :
FROM postgres:latest
VOLUME /var/lib/postgresql/data
ARG PG_POSTGRES_PWD=123qwe
ARG DBUSER=postgres
ARG DBUSER_PWD=123qwe
ARG DBNAME=docker_pg_sh
ARG DB_DUMP_FILE=gtma_latest.sql
ENV POSTGRES_DB docker_pg_sh
ENV POSTGRES_USER postgres
ENV POSTGRES_PASSWORD ${PG_POSTGRES_PWD}
ENV PGDATA /pgdata
COPY wait-for-pg-isready.sh /tmp/wait-for-pg-isready.sh
COPY ${DB_DUMP_FILE} /gtma_latest.sql
RUN set -e && \
nohup bash -c "docker-entrypoint.sh postgres & " && \
/tmp/wait-for-pg-isready.sh && \
psql -U postgres -c "CREATE USER ${DBUSER} WITH SUPERUSER CREATEDB CREATEROLE ENCRYPTED PASSWORD '${DBUSER_PWD}';" && \
psql -U ${DBUSER} -d ${POSTGRES_DB} -c "CREATE DATABASE ${DBNAME} TEMPLATE template0;" && \
pg_restore -v --no-owner --role=${DBUSER} --exit-on-error -U ${DBUSER} -d ${DBNAME} </gtma_latest.sql && \
psql -U postgres -c "ALTER USER ${DBUSER} WITH NOSUPERUSER;" && \
rm -rf /gtma_latest.sql
HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 \
CMD pg_isready -U postgres -d docker_pg_sh
and wait-for-pg-isready.sh file is :
#!/bin/bash
set -e
get_non_lo_ip() {
local _ip _non_lo_ip _line _nl=$'\n'
while IFS=$': \t' read -a _line ;do
[ -z "${_line%inet}" ] &&
_ip=${_line[${#_line[1]}>4?1:2]} &&
[ "${_ip#127.0.0.1}" ] && _non_lo_ip=$_ip
done< <(LANG=C /sbin/ifconfig)
printf ${1+-v} $1 "%s${_nl:0:$[${#1}>0?0:1]}" $_non_lo_ip
}
get_non_lo_ip NON_LO_IP
until pg_isready -h $NON_LO_IP -U "postgres" -d "docker_pg_sh"; do
>&2 echo "Postgres is not ready - sleeping..."
sleep 4
done
>&2 echo "Postgres is up - you can execute commands now"
Both file are in same folder.
But when I run the command docker build -t gmta-test-sh-vol:1.0.0 . , I am getting error
like this :
Step 14/15 : RUN set -e && nohup bash -c "docker-entrypoint.sh postgres & " && /tmp/wait-for-pg-isready.sh && psql -U postgres -c "CREATE USER ${DBUSER} WITH SUPERUSER CREATEDB CREATEROLE ENCRYPTED PASSWORD '${DBUSER_PWD}';" && psql -U ${DBUSER} -d ${POSTGRES_DB} -c "CREATE DATABASE ${DBNAME} TEMPLATE template0;" && pg_restore -v --no-owner --role=${DBUSER} --exit-on-error -U ${DBUSER} -d ${DBNAME} </gtma_latest.sql && psql -U postgres -c "ALTER USER ${DBUSER} WITH NOSUPERUSER;" && rm -rf /gtma_latest.sql
---> Running in a791e734c6df
/bin/sh: 1: /tmp/wait-for-pg-isready.sh: not found
The command '/bin/sh -c set -e && nohup bash -c "docker-entrypoint.sh postgres & " && /tmp/wait-for-pg-isready.sh && psql -U postgres -c "CREATE USER ${DBUSER} WITH SUPERUSER CREATEDB CREATEROLE ENCRYPTED PASSWORD '${DBUSER_PWD}';" && psql -U ${DBUSER} -d ${POSTGRES_DB} -c "CREATE DATABASE ${DBNAME} TEMPLATE template0;" && pg_restore -v --no-owner --role=${DBUSER} --exit-on-error -U ${DBUSER} -d ${DBNAME} </gtma_latest.sql && psql -U postgres -c "ALTER USER ${DBUSER} WITH NOSUPERUSER;" && rm -rf /gtma_latest.sql' returned a non-zero code: 127
But there is no error when the line for copy the file wait-for-pg-isready.sh is running .
How can the problem will be solved?
Thanks in advance.

sed not working in dockerfile but in container bash it does

I'm creating a Docker image where I use sed to modify two parameters, but when i create the images and check the file I wanted to modify it remanis the same. If i run the very sed command interactively, i t works. Why? Could sombebody help me make ma image work without having to modify every container.
Dockerfile
FROM python:slim-buster
WORKDIR /home/scr_dca
COPY . .
ENV FLASK_APP Screenly.py
RUN apt-get update && \
apt install curl gnupg -y && \
curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \
curl https://packages.microsoft.com/config/debian/10/prod.list > /etc/apt/sources.list.d/mssql-release.list && \
apt-get update && ACCEPT_EULA=Y apt-get install msodbcsql17 unixodbc-dev -y && \
apt-get install libgssapi-krb5-2 g++ gcc && \
pip3 install -r requirements.txt --trusted-host pypi.python.org
RUN sed -i "s/\(MinProtocol *= *\).*/\1TLSv1.0 /" "/etc/ssl/openssl.cnf" && \
sed -i "s/\(CipherString *= *\).*/\1DEFAULT#SECLEVEL=1 /" "/etc/ssl/openssl.cnf"
CMD ["gunicorn", "-b", ":8000", "scr_dca:app"]
I'm doing
docker run --name mycontainer -d -p 5050:8000 src_dca_v1.0
docker container exec -it mycontainer bash
:/home/myapp# cat /etc/ssl/openssl.cnf
I checked and sed didnt work during the image creation so I ran the following commands:
sed -i "s/\(MinProtocol *= *\).*/\1TLSv1.0 /" "/etc/ssl/openssl.cnf"
sed -i "s/\(CipherString *= *\).*/\1DEFAULT#SECLEVEL=1 /" "/etc/ssl/openssl.cnf"
original part of the file I want to modify:
[system_default_sect]
MinProtocol = TLSv1.2
CipherString = #SECLEVEL=1
sed expected result
[system_default_sect]
MinProtocol = TLSv1.0
CipherString = DEFAULT#SECLEVEL=1
I suspect there is something you are not seeing or that you did not explain/describe in your question. As is, I cannot reproduce your problem.
My MCVE, inspired by your current question to test:
FROM python:slim-buster
RUN cp /etc/ssl/openssl.cnf /etc/ssl/openssl.cnf.ORI && \
sed -i "s/\(MinProtocol *= *\).*/\1TLSv1.0 /" "/etc/ssl/openssl.cnf" && \
sed -i "s/\(CipherString *= *\).*/\1DEFAULT#SECLEVEL=1 /" "/etc/ssl/openssl.cnf" && \
(diff -u /etc/ssl/openssl.cnf.ORI /etc/ssl/openssl.cnf || exit 0)
Note: I ignored diff exit status and force it to 0, as it will exit with status 1 when there is a difference between the files which would fail the build.
And the result:
$ docker build --no-cache -t test:test .
Sending build context to Docker daemon 4.096kB
Step 1/2 : FROM python:slim-buster
---> 3d8f801fc3db
Step 2/2 : RUN cp /etc/ssl/openssl.cnf /etc/ssl/openssl.cnf.ORI && sed -i "s/\(MinProtocol *= *\).*/\1TLSv1.0 /" "/etc/ssl/openssl.cnf" && sed -i "s/\(CipherString *= *\).*/\1DEFAULT#SECLEVEL=1 /" "/etc/ssl/openssl.cnf" && (diff -u /etc/ssl/openssl.cnf.ORI /etc/ssl/openssl.cnf || exit 0)
---> Running in 523ddc0f4025
--- /etc/ssl/openssl.cnf.ORI 2020-01-09 16:21:44.667348574 +0000
+++ /etc/ssl/openssl.cnf 2020-01-09 16:21:44.675348574 +0000
## -358,5 +358,5 ##
system_default = system_default_sect
[system_default_sect]
-MinProtocol = TLSv1.2
-CipherString = DEFAULT#SECLEVEL=2
+MinProtocol = TLSv1.0
+CipherString = DEFAULT#SECLEVEL=1
Removing intermediate container 523ddc0f4025
---> 88c28529ceb5
Successfully built 88c28529ceb5
Successfully tagged test:test
As you can see, diff is showing the differences before/after running sed and the modifications you are expecting are there.
We can also make sure those modifications persist when starting a container from this image:
$ docker run -it --rm --name testcmd test:test bash -c "grep -A 2 '\[system_default_sect\]' /etc/ssl/openssl.cnf"
[system_default_sect]
MinProtocol = TLSv1.0
CipherString = DEFAULT#SECLEVEL=1

Resources