Interactive shell using Docker Compose - shell

Is there any way to start an interactive shell in a container using Docker Compose only? I've tried something like this, in my docker-compose.yml:
myapp:
image: alpine:latest
entrypoint: /bin/sh
When I start this container using docker-compose up it's exited immediately. Are there any flags I can add to the entrypoint command, or as an additional option to myapp, to start an interactive shell?
I know there are native docker command options to achieve this, just curious if it's possible using only Docker Compose, too.

You need to include the following lines in your docker-compose.yml:
version: "3"
services:
app:
image: app:1.2.3
stdin_open: true # docker run -i
tty: true # docker run -t
The first corresponds to -i in docker run and the second to -t.

The canonical way to get an interactive shell with docker-compose is to use:
docker-compose run --rm myapp
(With the service name myapp taken from your example. More general: it must be an existing service name in your docker-compose file, myapp is not just a command of your choice. Example: bash instead of myapp would not work here.)
You can set stdin_open: true, tty: true, however that won't actually give you a proper shell with up, because logs are being streamed from all the containers.
You can also use
docker exec -ti <container name> /bin/bash
to get a shell on a running container.

The official getting started example (https://docs.docker.com/compose/gettingstarted/) uses the following docker-compose.yml:
version: "3.9"
services:
web:
build: .
ports:
- "8000:5000"
redis:
image: "redis:alpine"
After you start this with docker-compose up, you can shell into either your redis container or your web container with:
docker-compose exec redis sh
docker-compose exec web sh

docker-compose run myapp sh should do the deal.
There is some confusion with up/run, but docker-compose run docs have great explanation: https://docs.docker.com/compose/reference/run

If anyone from the future also wanders up here:
docker-compose exec service_name sh
or
docker-compose exec service_name bash
or you can run single lines like
docker-compose exec service_name php -v
That is after you already have your containers up and running.
The service_name is defined in your docker-compose.yml file

Using docker-compose, I found the easiest way to do this is to do a docker ps -a (after starting my containers with docker-compose up) and get the ID of the container I want to have an interactive shell in (let's call it xyz123).
Then it's a simple matter to execute
docker exec -ti xyz123 /bin/bash
and voila, an interactive shell.

This question is very interesting for me because I have problems, when I run container after execution finishes immediately exit and I fixed with -it:
docker run -it -p 3000:3000 -v /app/node_modules -v $(pwd):/app <your_container_id>
And when I must automate it with docker compose:
version: '3'
services:
frontend:
stdin_open: true
tty: true
build:
context: .
dockerfile: Dockerfile.dev
ports:
- "3000:3000"
volumes:
- /app/node_modules
- .:/app
This makes the trick: stdin_open: true, tty: true
This is a project generated with create-react-app
Dockerfile.dev it looks this that:
FROM node:alpine
WORKDIR '/app'
COPY package.json .
RUN npm install
COPY . .
CMD ["npm", "run", "start"]
Hope this example will help other to run a frontend(react in example) into docker container.

I prefer
docker-compose exec my_container_name bash

If the yml is called docker-compose.yml it can be launched with a simple $ docker-compose up. The corresponding attachment of a terminal can be simply (consider that the yml has specified a service called myservice):
$ docker-compose exec myservice sh
However, if you are using a different yml file name, such as docker-compose-mycompose.yml, it should be launched using $ docker-compose -f docker-compose-mycompose.yml up. To attach an interactive terminal you have to specify the yml file too, just like:
$ docker-compose -f docker-compose-mycompose.yml exec myservice sh

A addition to this old question, as I only had the case last time. The difference between sh and bash. So it can happen that for some bash doesn't work and only sh does.
So you can:
docker-compose exec CONTAINER_NAME sh
and in most cases: docker-compose exec CONTAINER_NAME bash
use.
If you have time. The difference between sh and bash is well explained here:
https://www.baeldung.com/linux/sh-vs-bash

You can do docker-compose exec SERVICE_NAME sh on the command line. The SERVICE_NAME is defined in your docker-compose.yml. For example,
services:
zookeeper:
image: wurstmeister/zookeeper
ports:
- "2181:2181"
The SERVICE_NAME would be "zookeeper".

According to documentation -> https://docs.docker.com/compose/reference/run/
You can use this docker-compose run --rm app bash
[app] is the name of your service in docker-compose.yml

Related

Why does "docker-compose up" exit but "docker-compse run" enters into bash shell

Dockerfile
FROM get some base image
ENV ProjectDir /workarea/svc
RUN mkdir -p $ProjectDir
WORKDIR $ProjectDir
docker-compose.yaml
version: "3.7"
services:
svc:
build:
context: .
dockerfile: Dockerfile
volumes:
- .:/workarea/svc
command: ["/opt/bb/bin/bash"]
When I run docker-compose up it exits immediately
r#PW02R9F3:$ docker-compose up
Creating svc_dev_1 ... done
Attaching svc_dev_1
svc_dev_1 exited with code 0
But when I run "docker-compose run --rm dev" I am able to get into bash as specified in the command session of my docker-compose.yaml file
r#PW02R9F3:$ docker-compose run --rm dev
Creating svc_dev_run ... done
[root#ad5d3d7107b4 svc]#
Why is this happening? Isnt "docker-compose up" running my command "/opt/bb/bin/bash" in the docker-compose.yaml file?
I believe this is because docker compose run spawns a container in interactive mode (unless specified otherwise) by default. docker compose up does not.
That is of importance because when running bash in a container that is not in interactive mode, it just dies immediately with status code 0 not because there's an error, but because there's no input for bash (and won't be).
It's like running docker run ubuntu and docker run -it ubuntu. The latter will keep STDIN open, "listening" for commands if you will.

How to run a shell command on all containers of a replicated docker-compose service?

I am running spark workers using docker, replicated using a docker-compose setup:
version: '2'
services:
spark-worker:
image: bitnami/spark:latest
environment:
- SPARK_MODE=worker
- SPARK_MASTER_URL=spark://1.1.1.1:7077
deploy:
mode: replicated
replicas: 4
When I run docker-compose exec spark-worker ls, for example, it only runs on usually the first replica. Is there a way to broadcast these commands to all of the replicas?
docker-compose version 1.29.2, build 5becea4c
Docker version 20.10.7, build f0df350
There's no built-in facility for this, but you could construct something fairly easily. For example:
docker ps -q --filter label=com.docker.compose.service=spark-worker |
xargs -ICID docker exec CID ls
I would propose a simpler command than what #larsks proposed, which leverages the docker-compose command itself.
SERVICE_NAME=spark-worker
for id in $(docker-compose ps -q $SERVICE_NAME); do
docker exec -t $id "echo" "hello"
done;

/bin/sh: No such file or directory when setting a docker-compose entrypoint

I have a container that runs a database migration (source):
FROM golang:1.12-alpine3.10 AS downloader
ARG VERSION
RUN apk add --no-cache git gcc musl-dev
WORKDIR /go/src/github.com/golang-migrate/migrate
COPY . ./
ENV GO111MODULE=on
ENV DATABASES="postgres mysql redshift cassandra spanner cockroachdb clickhouse mongodb sqlserver firebird"
ENV SOURCES="file go_bindata github github_ee aws_s3 google_cloud_storage godoc_vfs gitlab"
RUN go build -a -o build/migrate.linux-386 -ldflags="-s -w -X main.Version=${VERSION}" -tags "$DATABASES $SOURCES" ./cmd/migrate
FROM alpine:3.10
RUN apk add --no-cache ca-certificates
COPY --from=downloader /go/src/github.com/golang-migrate/migrate/build/migrate.linux-386 /migrate
ENTRYPOINT ["/migrate"]
CMD ["--help"]
I want to integrate it into a docker-compose and make it dependent on the Postgres database service. However, since I have to wait until the database is fully initialised I have to wrap the migrate command in a script and thus replace the entrypoint of the migration container. I'm using the wait-for script to poll the database, which is a pure shell (not bash) script and should thus work in an alpine container.
This is how the service is defined in the docker-compose:
services:
database:
# ...
migration:
depends_on:
- database
image: migrate/migrate:v4.7.0
volumes:
- ./scripts/migrations:/migrations
- ./scripts/wait-for:/wait-for
entrypoint: ["/bin/sh"]
command: ["./wait-for database:5432", "--", "./migrate", "-path", "/migrations", "-database", "postgres://test:test#database:5432/test?sslmode=disable", "-verbose", "up"]
Running docker-compose up on this fails with
migration_1 | /bin/sh: can't open './wait-for database:5432': No such file or directory
Running the migrate container for itself with
docker run -it --entrypoint /bin/sh -v $(pwd)/scripts/wait-for:/wait-for migrate/migrate:v4.7.0
does work flawlessly, the script is there and can be run with /bin/sh ./wait-for.
So why does it fail as part of the docker-compose?
If you read the error message carefully, you will see that the file that cannot be found is not ./waitfor, it is ./wait-for database:5432. This is consistent with your input file, where that whole thing is given as the first element of the command list:
command: ["./wait-for database:5432", "--", "./migrate", "-path", "/migrations", "-database", "postgres://test:test#database:5432/test?sslmode=disable", "-verbose", "up"]
It's unclear to me what you actually want instead, since the working alternatives presented do not seem to be fully analogous, but possibly it's
command: ["./wait-for", "database:5432", "--", "./migrate", "-path", "/migrations", "-database", "postgres://test:test#database:5432/test?sslmode=disable", "-verbose", "up"]
Running the migrate container for itself with does work flawlessly
When you run it like:
docker run -it --entrypoint /bin/sh -v $(pwd)/scripts/wait-for:/wait-for migrate/migrate:v4.7.0
entrypoint /bin/sh is executed.
When you run it using docker-compose:
entrypoint (/bin/sh ) + command (./wait-for database:5432) ...` is executed.
./wait-for database:5432 as whole stands for executable that will run and it can't be found, that's why you get the error No such file or directory
Try to specify an absolute path to wait-for in command: and split ./wait-for database:5432 into "./wait-for", "database:5432".
It's possible that splitting will be enough
As an alternative you can follow CMD syntax docs and use different command syntax without array: command: ./wait-for database:5432 ...
ENTRYPOINT ["/bin/sh"] is not enough, you also need the -c argument.
Example (testing a docker-compose.yml with docker-compose run --rm MYSERVICENAMEFROMTHEDOCKERCOMPOSEFILE bash here):
entrypoint: ["/bin/sh"]
Throws:
/bin/sh: 0: cannot open bash: No such file
ERROR: 2
And some wrong syntax examples like
entrypoint: ["/bin/sh -c"]
(wrong!)
or
entrypoint: ["/bin/sh, -c"]
(wrong!)
throw errors:
starting container process caused: exec: "/bin/sh, -c": stat /bin/sh, -c: no such file or directory: unknown
ERROR: 1
starting container process caused: exec: "/bin/sh -c": stat /bin/sh -c: no such file or directory: unknown
ERROR: 1
In docker-compose or Dockerfile, for an entrypoint, you need the -c argument.
This is right:
entrypoint: "/bin/sh -c"
or:
entrypoint: ["/bin/sh", "-c"]
The -c is to make clear that this is a command executed in the command line, waiting for an additional command to be used in that command line. but not starting the bash /bin/sh just on its own. You can read that between the lines at What is the difference between CMD and ENTRYPOINT in a Dockerfile?.

How to force Git for Windows' bash-shell to not convert path-string to windows path?

I'm using the bash shell provided by Git for Windows for Docker toolbox for Windows. I want to export a string representing a unix path to a environment variable to then use in a docker container. Something like:
export MY_VAR=/my/path; docker-compose up
The problem is that in my container the variable will be something like:
echo $MY_VAR # prints c:/Program Files/Git/my/path
So it seems the shell (my guess) recognizes the string as a path and converts it to windows format. Is there a way to stop this?
I've attempted to use MSYS_NO_PATHCONV=1:
MSYS_NO_PATHCONV=1; export LOG_PATH=/my/path; docker-compose up
But it did not have any effect.
I don't think it's an issue with my docker-compose and dockerfile but I'll attach them if someone is interested.
My Dockerfile:
FROM node:8-slim
RUN mkdir /test \
&& chown node:node /test
USER node
ENTRYPOINT [ "/bin/bash" ]
My docker-compose.yml:
version: '2'
services:
test:
build:
context: .
image: test
environment:
- MY_VAR
volumes:
- ${MY_VAR}:/test
command: -c 'sleep 100000'
The Final goal here is to make a directory on the host machine accessible from the docker container (for logs and such). The directory should be set by an environment variable. Setting the directory in the docker-compose.yml does work, just not for my use case.
If you want your command docker-compose up to be run with MSYS_NO_PATHCONV=1; you have two options:
export LOG_PATH=/c/Windows; export MSYS_NO_PATHCONV=1; docker-compose up This will affect your bash session as the variable is exported
export LOG_PATH=/c/Windows; MSYS_NO_PATHCONV=1 docker-compose up; (note I removed one semi-colon intentionally) This will set MSYS_NO_PATHCONV only in the context of the command to run
Test it with:
$ export LOG_PATH=/c/Windows ; cmd "/c echo %LOG_PATH%";
C:/Windows --> Fails
$ export LOG_PATH=/c/Windows ; MSYS_NO_PATHCONV=1 cmd "/c echo %LOG_PATH%"
/c/Windows --> Success
$ export LOG_PATH=/c/Windows ; export MSYS_NO_PATHCONV=1; cmd "/c echo %LOG_PATH%";
/c/Windows --> Success but MSYS_NO_PATHCONV is now "permanently" set
Seems a workaround is to remove the first / from the string and add it in the docker-compose.yml instead.
new docker-compose.yml:
version: '2'
services:
test:
build:
context: .
image: test
environment:
- MY_VAR
volumes:
- /${MY_VAR}:/test # added '/' to the beginning of the line
command: -c 'sleep 100000'
and then starting the container with:
export MY_VAR=my/path; docker-compose up # removed the '/' from the beginning of the path.
This does seem more like a "lucky" workaround than a perfect solution as when I'll build this on other systems I'll have to remind myself to remove the /. Doable but a bit annoying. Maybe someone has a better idea.

Docker compose won't find $PWD environment variable

Here's my docker-compose:
version: '2'
services:
couchpotato:
build:
context: ./couchpotato
dockerfile: Dockerfile
ports:
- 5050:5050
volumes:
- "${PWD}/couchpotato/data:/home/CouchPotato/data/"
- "${PWD}/couchpotato/config:/home/CouchPotato/config/"
When I run it inside the shell, in the directory of the docker-compose.yml, I get:
WARNING: The PWD variable is not set. Defaulting to a blank string.
and the compose starts with PWD being empty.
I don't see any error in the file, as seen here: https://docs.docker.com/compose/environment-variables/
You don't need ${PWD} for this, you can just make the path relative and compose will expand it (one major difference between compose paths and those processed by docker run).
version: '2'
services:
couchpotato:
build:
context: ./couchpotato
dockerfile: Dockerfile
ports:
- 5050:5050
volumes:
- "./couchpotato/data:/home/CouchPotato/data/"
- "./couchpotato/config:/home/CouchPotato/config/"
As for why compose doesn't see this variable, that depends on your shell. Compose looks for an exported environment variable, contents of the .env file, and command line flags to the docker-compose command. If each of those comes up empty for the variable, you'll get that warning.
My advice: change all $PWD to .
$PWD will not work if you are running using sudo. Try the recommended settings from Docker for Linux https://docs.docker.com/engine/install/linux-postinstall/.
Sudo will run as a different user, with a different env.
$ sudo env | grep -i pwd
$ env | grep -i pwd
PWD=/home/user
OLDPWD=/
If you really need absolute paths, then call this before calling docker-compose up:
set PWD=%CD%
I had the same issue with one of my env vars. On looking at my bashrc file more closely, I found out that I hadn't exported that variable.
Before:
VAR=<value>
After:
export VAR=<value>

Resources