I need to execute a bash script containing SQL, so I am using a script to add custom configurations to a Postgres Docker container, according to the docs here:
https://github.com/docker-library/docs/tree/master/postgres#how-to-extend-this-image
But I don't know what EOSQL means. Here is an example of my script taken from the docs above:
#!/bin/bash
set -e
psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
CREATE USER docker;
CREATE DATABASE docker;
GRANT ALL PRIVILEGES ON DATABASE docker TO docker;
CREATE EXTENSION $MY_EXTENSION;
EOSQL
So, what is EOSQL? I cannot seem to find much information about this command or keyword.
EOSQL is a limit string for a Here Document block. The limit string signifies the start and end of a text block to the bash interpreter (or any POSIXy shell). The limit string can be any text that doesn't appear in your block, EOF is common in examples.
Variable substitution will work as normal in a here document:
#!/usr/bin/env bash
cat <<-EOF
a
$MY_EXTENSION
b
EOF
echo "script continues" > /dev/null
Then running that with the MY_EXTENSION variable set:
$ MY_EXTENSION=something ./test.sh
a
something
b
In Docker you will need ENV MY_EXTENSION=something in your Dockerfile or docker run -e MY_EXTENSION=something <image> on the command line for the environment to be setup.
Leading tabs
The <<-EOSQL that starts this heredoc includes a - to ignore the leading tab character on any lines of the heredoc.
Using <<EOSQL instead would leave the leading tabs in the output.
Related
I'm trying to read CSV file and writing the same into the table, CSV file was located in my local machine(client). I used /copy command and achieved the same. Here I have hardcoded my filepath in sql script. I want to parameterised my csv file path.
Based on my analysis /copy not supported :variable substitution, but not sure
I believe we can achieve this using shell variables, but I tried the same, It's not working as expected.
Following are my sample scripts
command:
psql -U postgres -h localhost testdb -a -f '/tmp/psql.sql' -v path='"/tmp/userData.csv"'
psql script:
\copy test_user_table('username','dob') from :path DELIMITER ',' CSV HEADER;
I executing this commands from shell and I'm getting no such a file not found exception. But same script is working with hardcoded path.
Anyone able to advise me on this.
Reference :
Variable substitution in psql \copy
https://www.postgresql.org/docs/devel/app-psql.html
I am new to Bash. So far your problem is way hard for me.
I can do it in one shell script. Maybe later I can make it to two scripts.
The follow is a simple one script file.
#!bin/bash
p=\'"/mnt/c/Users/JIAN HE/Desktop/test.csv"\'
c="copy emp from ${p}"
a=${c}
echo $a
psql -U postgres -d postgres -c "${a}"
I try to run docker run from bash script and docker says:
“is not a docker command”
If I print the docker command line before I called docker and I copy it to clipboard and paste it to command line it works well!
here is the command in bash script:
local args="run ${nw_param} ${opts} --name ${img} ${repository}/${img}:${tag}"
docker ${args}
the current echo of args string is:
run --net=ehvb-network -d --restart=always --name my-module my-private-registry:5000/my-module:0.0.1-1555334810
When I copied this string to the clipboard and paste it to command line it works well.
I use Debian stretch. My script is using bash (#!/bin/bash)
When I remove ${opts} it runs from bash. Opts currently contains “-d --restart=always”. When I try to use only -d or only --restart=always it works well. But when I try to use both together it doesn’t work well.
And I try to define opts like this:
opts=’–restart=always -d’
the message from docker is:
docker: Error response from daemon: invalid restart policy ‘always -d’, but the print message contains:
opts:–restart=always -d
Somebody removes --restart=
The problem was that, I used variables coming from other bash command in my script (like curl, ps etc). All of these variables end with carriage return \r. When I try to insert these variables into a docker parameter string \r are inside it. I need to add:
| sed 's/\r//' to all of these commands.
How can I run a command inside a docker container, using docker run, where bash variables are evaluated inside the container?
E.g.:
$ SOMEONE=host
$ docker run --env SOMEONE=busybox busybox echo "Hello $SOMEONE"
Hello host
How can I make it output Hello busybox?
To prevent the replacement from happening from the outer shell, one needs to use single quotes, not double.
To ensure that there is an inner shell that can do a replacement (echo doesn't have any such functionality itself!), we need to explicitly call sh -c; otherwise, Docker will just directly invoke execlp("echo", "echo", "$SOMEONE", NUL) inside the container, which doesn't actually do any substitution.
Thus:
docker run --env SOMEONE=busybox busybox sh -c 'echo "Hello $SOMEONE"'
Using docker run, where bash variables are evaluated inside
By far the easiest, non-cryptic approach is to write a bash function with all commands to be executed inside the container. Benefits:
Easy to write - no need to use special quote placement and escaping
Easy to debug - see what bash actually does inside the container
Easy to maintain - write readable scripts, not cryptic commands
Easy to write and maintain
Here's an example bash function that expands all variables inside a docker container.
-- (host) $ ./create-db.sh
#!/bin/bash
function main_inside_docker {
# all variables are expanded insider docker
DBNAME=${1:-testdb}
echo "creating database $DBNAME"
PATH=$MSSQL_PATH:$PATH
SQL="
create database $DBNAME;
select database_id, name, create_date from sys.databases;
"
sqlcmd -U SA -P $SA_PASSWORD -Q "$SQL"
}
# declare the function inside docker and run it there
CMD="$(declare -f main_inside_docker); main_inside_docker $#"
docker exec -it mssql bash -c "$CMD"
Essentially this declares the main_inside_docker function inside the container, then runs it with all arguments provided from the host invocation. All variables inside the function are expanded inside the docker container. The function just works the way one would expect.
Easy to debug
To debug the function, set "-x" as the first command in $CMD:
CMD="set -x; $(declare -f ...)"
When running it this way, it will print the bash trace from inside the container nicely:
(host) $ ./create-db.sh foodb
+ main_inside_docker
+ DBNAME=foodb
+ echo 'creating database foodb'
creating database testdb
...
I'm trying to create a script to run a docker cluster.
In my script there is a moment that I want to copy some files from the docker to my local machine. So I'm creating the CONTAINER_WORKDIR variable.
CONTAINER_WORKDIR=`docker exec -it jmeter-master /bin/pwd`
The value stored in CONTAINER_WORKDIR is:
/usr/local/apache-jmeter-3.2/bin
The problem is that there is a strange character in the end of this variable. Try to execute the line below:
echo "docker cp jmeter-master:$CONTAINER_WORKDIR/output.csv ."
My expected result is
docker cp jmeter-master:/usr/local/apache-jmeter-3.2/bin/output.csv .
But the real output is:
/output.csv .ter-master:/usr/local/apache-jmeter-3.2/bin
The pwd or the docker exec command is returning a returning character.
There is a way to remove this character from CONTAINER_WORKDIR variable
This script that executes what I presume must be
CONTAINER_WORKDIR=$(docker exec -it jmeter-master /bin/pwd)
may have been written with an editor that stores text files with DOS line-endings (like Notepad++).
Run dos2unix on that script, or use
$ tr -d '\r' <script >script-new
to fix it.
I want to write a line of text to a textfile INSIDE a running docker container. Here's what I've tried so far:
docker exec -d app_$i eval echo "server.url=$server_url" >> /home/app/.app/app.config
Response:
/home/user/.app/app.config: No such file or directory
Second try:
cfg_add="echo 'server.url=$server_url' >> /home/user/.app/app.config"
docker exec -i app_$i eval $cfg_add
Response:
exec: "eval": executable file not found in $PATH
Any ideas?
eval is a shell builtin, whereas docker exec requires an external utility to be called, so using eval is not an option.
Instead, invoke a shell executable in the container (bash) explicitly, and pass it the command to execute as a string, via its -c option:
docker exec "app_$i" bash -c "echo 'server.url=$server_url' >> /home/app/.app/app.config"
By using a double-quoted string to pass to bash -c, you ensure that the current shell performs string interpolation first, whereas the container's bash instance then sees the expanded result as a literal, as part of the embedded single-quoted string.
As for your symptoms:
/home/user/.app/app.config: No such file or directory was reported, because the redirection you intended to happen in the container actually happened in your host's shell - and because dir. /home/user/.app apparently doesn't exist in your host's filesystem, the command failed fundamentally, before your host's shell even attempted to execute the command (bash will abort command execution if an output redirection cannot be performed).
Thus, even though your first command also contained eval, its use didn't surface as a problem until your second command, which actually did get executed.
exec: "eval": executable file not found in $PATH happened, because, as stated, eval is not an external utility, but a shell builtin, and docker exec can only execute external utilities.
Additionally:
If you need to write text from outside the container, this also works:
(docker exec -i container sh -c "cat > c.sql") < c.sql
This will pipe you input into the container. Of course, this would also work for plain text (no file). It is important to leave off the -t parameter.
See https://github.com/docker/docker/pull/9537
UPDATE (in case you just need to copy files, not parts of files):
Docker v17.03 has docker cp which copies between the local fs and the container: https://docs.docker.com/engine/reference/commandline/cp/#usage
try to use heredoc:
(docker exec -i container sh -c "cat > /test/iplist") << EOF
10.99.154.146
10.99.189.247
10.99.189.250
EOF