I would like to read contents of a file specified by an environment variable and pass it to docker-compose as build arg.
So then in my Dockerfile I can do:
ARG MY_FILE
RUN echo "$MY_FILE" > /my-file
This works perfectly:
docker-compose -f ./docker-compose.yml build --build-arg MY_FILE="$(cat $PATH_TO_MY_FILE)"
However, if I try to do this in docker-compose.yml like so:
build:
context: .
args:
- MY_FILE="$(cat $PATH_TO_MY_FILE)"
it fails with this error:
ERROR: Invalid interpolation format for "build" option in service "my-service": "MY_FILE="$(cat $PATH_TO_MY_FILE)""
Any idea how do I have to construct this string to have the same effect? I tried $$ etc, but doesn't seem to work...
Thanks for your help :)
Docker compose doesn't support this, so you have to use a workaround only. Which would either mean pre-processing the compose file or generate the command you ran by reading the yaml and interpolating by generating the command in bash
You can use something like yq and parse the parameters from docker-compose.yml and generate your command. But honestly what you are doing right now is simple and effective.
In docker service 3, you can do that now.
web:
image: xxxx
env_file:
- web-variables.env
If you have specified a Compose file with docker-compose -f FILE, paths in env_file are relative to the directory that file is in.
Related
I am trying to run a local script with docker bash in windows PowerShell but not working.
My script part is another program, but the finally goal is to process a media file and zip it with the shell script.
The cmd: docker exec -it containername /bin/bash < myscript.sh -f fileone.mp4 -h output
I have an error in ps:
The '<' operator is reserved for future use.
The parameters (and also the files) are changing, if rerun the shell script, and after the script, processing is done it will create a zip file (what I need) with the output name, but random strings will be placed to the zipped filename too.
Anyone tried to use docker in that way in windows?
I figure out a solution for my own question. I just leave it here, if someone needs it.
The docker-compose file:
version: '3.8'
services:
somename:
build:
context: .
dockerfile: Dockerfile
container_name: 'name_if_you_need'
The dockerfile:
FROM debian:latest
# Install and/or config anything what you need
ADD . /newfolder
WORKDIR /newfolder
ENTRYPOINT [ "/newfolder/myscript.sh" ]
To call (with arguments and/or flags if your script need it): docker run --rm -v ${PWD}:/newfolder image_name -flag1 sample.mp4 -flag2 sample (no tty error, not need winpty)
Please note, if your script working with file or files, and you pass it via arguments like me, you need to copy them in your current folder before docker run
With this solution, if your script generates a file or files when/after executing, you will see them automatically in your current folder.
In my docker-compose file (docker-compose.yaml), I would like to set an argument based on a small shell script like this:
services:
backend:
[...]
build:
[...]
args:
PKG_NAME: $(dpkg -l <my_package>)
In the Dockerfile, I read this argument like this:
ARG PKG_NAME
First of all: I know that this approach is OS-dependent (requires dpkg), but for starters I would be happy to make it run on Debian. Also, it's fine it the value is an empty string.
However, docker-compose up throws this error:
ERROR: Invalid interpolation format for "build" option in service "backend": "$(dpkg -l <my_package>)"
Is there a way to dynamically specify an argument in the docker-compose file through a shell script (or another way)?
You can only use variable substitution as described in compose file documentation
You are trying to inject a shell construct and this is not supported.
The documentation has several examples on how to pass vars to compose file. In your case, you could:
export the var in your environment:
export MY_PACKAGE=$(dpkg -l <my_package>)
use that var in your compose file with default:
args:
PKG_NAME: "${MY_PACKAGE:-some_default_pkg}"
I have the following docker-compose service:
maxwell:
image: zendesk/maxwell:latest
restart: always
environment:
# <irrelevant config here>
MAXWELL_PRODUCER: stdout
MAXWELL_OPTIONS: >-
# <irrelevant config here>
--filter="exclude:*.*"
The file running inside the image starts the server like this:
exec `dirname $0`/maxwell --user=$MYSQL_USERNAME --password=$MYSQL_PASSWORD --host=$MYSQL_HOST --producer=$MAXWELL_PRODUCER $MAXWELL_OPTIONS
The server this image runs, Maxwell, is not starting because it's complaining about malformed options. However, if I SSH into the image (with docker-compose run maxwell bash), and input the above filter option, exactly as written, the server starts normally.
I tried parsing the above YAML and looking at the string. It looks correct, so I don't think it's a YAML formatting issue.
What could be causing the command line option to get mangled?
Here's my docker-compose:
version: '2'
services:
couchpotato:
build:
context: ./couchpotato
dockerfile: Dockerfile
ports:
- 5050:5050
volumes:
- "${PWD}/couchpotato/data:/home/CouchPotato/data/"
- "${PWD}/couchpotato/config:/home/CouchPotato/config/"
When I run it inside the shell, in the directory of the docker-compose.yml, I get:
WARNING: The PWD variable is not set. Defaulting to a blank string.
and the compose starts with PWD being empty.
I don't see any error in the file, as seen here: https://docs.docker.com/compose/environment-variables/
You don't need ${PWD} for this, you can just make the path relative and compose will expand it (one major difference between compose paths and those processed by docker run).
version: '2'
services:
couchpotato:
build:
context: ./couchpotato
dockerfile: Dockerfile
ports:
- 5050:5050
volumes:
- "./couchpotato/data:/home/CouchPotato/data/"
- "./couchpotato/config:/home/CouchPotato/config/"
As for why compose doesn't see this variable, that depends on your shell. Compose looks for an exported environment variable, contents of the .env file, and command line flags to the docker-compose command. If each of those comes up empty for the variable, you'll get that warning.
My advice: change all $PWD to .
$PWD will not work if you are running using sudo. Try the recommended settings from Docker for Linux https://docs.docker.com/engine/install/linux-postinstall/.
Sudo will run as a different user, with a different env.
$ sudo env | grep -i pwd
$ env | grep -i pwd
PWD=/home/user
OLDPWD=/
If you really need absolute paths, then call this before calling docker-compose up:
set PWD=%CD%
I had the same issue with one of my env vars. On looking at my bashrc file more closely, I found out that I hadn't exported that variable.
Before:
VAR=<value>
After:
export VAR=<value>
I have a Dockerfile which is basically the following:
FROM mhart/alpine-node:5
COPY . /project
WORKDIR /project
ENTRYPOINT ["./startup.sh"]
And my startup.sh is quite simple too:
#!/bin/sh
set -e
docker-compose up -d
I do have a docker-compose.yml, but there is no point to describe it here.
First thing I do is to build the docker image by using my Dockerfile, so:
docker build -t test .
Then run this image:
docker run -d test
Which will launch the startup.sh
Unfortunately, I have the following error showing up:
./startup.sh: line 10: docker-compose: not found
And if I do only ./startup.sh without the docker stuff, it works like a charm.
Where the issue can be possibly coming from?
Try to add the full path to the docker-compose inside the script
which docker-compose
>/usr/bin/some/path/docker-compose
Then add this to your script
#!/bin/sh
set -e
/usr/bin/some/path/docker-compose up -d
Your local PATH settings are unknown to the script called by docker. Therefore you have to name the full path.