I have set COMPOSE_PROJECT_NAME=xxx in .env file in order to customise project name. But when I use the following:
docker-compose up --abort-on-container-exit --scale influx=0 kafka=0
It throws an error saying -
ERROR: No such service: kafka=0
If I provide only one service then it scales up/down without any error but breaks down when I provide multiple services. Is there a way to fix this?
Here's the structure of docker-compose.yml file -
version: '3'
services:
influx:
image: xxx
volumes:
- xxx
ports:
- xxx
kafka:
image: xxx
volumes:
- xxx
ports:
- xxx
The syntax is docker-compose up [options] continers... and one of the option is --scale and it takes one argument. You can, for exmaple:
docker-compose up --scale influx=1 --scale kafka=2 kafka influx
^^^^^^^^^^^^ - containers, just omit if all
^^^^^^^^ - argument to --scale option
^^^^^^^ - option
Related
I have a problem in opening minio in the browser. I just created Spring Boot app with the usage of it.
Here is my application.yaml file shown below.
server:
port: 8085
spring:
application:
name: springboot-minio
minio:
endpoint: http://127.0.0.1:9000
port: 9000
accessKey: minioadmin #Login Account
secretKey: minioadmin # Login Password
secure: false
bucket-name: commons # Bucket Name
image-size: 10485760 # Maximum size of picture file
file-size: 1073741824 # Maximum file size
Here is my docker-compose.yaml file shown below.
version: '3.8'
services:
minio:
image: minio/minio:latest
container_name: minio
environment:
MINIO_ROOT_USER: "minioadmin"
MINIO_ROOT_PASSWORD: "minioadmin"
volumes:
- ./data:/data
ports:
- 9000:9000
- 9001:9001
I run it by these commands shown below.
1 ) docker-compose up -d
2 ) docker ps -a
3 ) docker run minio/minio:latest
Here is the result shown below.
C:\Users\host\IdeaProjects\SpringBootMinio>docker run minio/minio:latest
NAME:
minio - High Performance Object Storage
DESCRIPTION:
Build high performance data infrastructure for machine learning, analytics and application data workloads with MinIO
USAGE:
minio [FLAGS] COMMAND [ARGS...]
COMMANDS:
server start object storage server
gateway start object storage gateway
FLAGS:
--certs-dir value, -S value path to certs directory (default: "/root/.minio/certs")
--quiet disable startup information
--anonymous hide sensitive information from logging
--json output server logs and startup information in json format
--help, -h show help
--version, -v print the version
VERSION:
RELEASE.2022-01-08T03-11-54Z
When I write 127.0.0.1:9000 in the browser, I couldn't open the MinIo login page.
How can I fix my issue?
The MinIO documentation includes a MinIO Docker Quickstart Guide that has some recipes for starting the container. The important thing here is that you cannot just docker run minio/minio; it needs a command to run, probably server. This also needs to be translated into your Compose setup.
The first example on that page breaks down like so:
docker run \
-p 9000:9000 -p 9001:9001 \ # publish ports
-e "MINIO_ROOT_USER=..." \ # set environment variables
-e "MINIO_ROOT_PASSWORD=..." \
quay.io/minio/minio \ # image name
server /data --console-address ":9001" # command to run
That final command is important. In your example where you just docker run the image and get a help message, it's because you omitted the command. In the Compose setup you also don't have a command: line; if you look at docker-compose ps I expect you'll see the container is exited, and docker-compose logs minio will probably show the same help message.
You can include that command in your Compose setup with command::
version: '3.8'
services:
minio:
image: minio/minio:latest
environment:
MINIO_ROOT_USER: "..."
MINIO_ROOT_PASSWORD: "..."
volumes:
- ./data:/data
ports:
- 9000:9000
- 9001:9001
command: server /data --console-address :9001 # <-- add this
I'm using docker-compose to manage a multi container application. 1 of those containers needs access to the contents of a directory on the host.
This seems simple according to the various sources of documentation on docker and docker-compose but I'm struggling to get it working.
event_processor:
environment:
- COMPOSE_CONVERT_WINDOWS_PATHS=1
build: ./Docker/event_processor
ports:
- "15672:15672"
entrypoint: python -u /src/event_processor/event_processor.py
networks:
- app_network
volumes:
- C/path/to/interesting/directory:/interesting_directory"
Running this I get the error message:
ERROR: Named volume
"C/path/to/interesting/directory:/interesting_directory:rw" is used in
service "event_processor" but no declaration was found in the
volumes section.
I understand from the docs that a top level declaration is only necessary if data is to be shared between containers
which isn't the case here.
The docs for docker-compose I linked above have an example which seems to do exactly what I need:
version: "3.2"
services:
web:
image: nginx:alpine
ports:
- "80:80"
volumes:
- type: volume
source: mydata
target: /data
volume:
nocopy: true
- type: bind
source: ./static
target: /opt/app/static
networks:
webnet:
volumes:
mydata:
However when I try, I get errors about the syntax:
ERROR: The Compose file '.\docker-compose.yaml' is invalid because:
services.audio_event_processor.volumes contains an invalid type, it
should be a string
So I tried to play along:
volumes:
- type: "bind"
source: "C/path/to/interesting/directory"
target: "/interesting_directory"
ERROR: The Compose file '.\docker-compose.yaml' is invalid because:
services.audio_event_processor.volumes contains an invalid type, it should be a string
So again the same error.
I tried the following too:
volumes:
- type=bind, source=C/path/to/interesting/directory,destination=/interesting_directory
No error, but attaching to the running container, I see the following two folders;
type=bind, source=C
So it seems that I am able to create a number of volumes with 1 string (though the forward slashes are cutting the string in this case) but I am not mapping it to the host directory.
I've read the docs but I think I'm missing something.
Can someone post an example of mounting a a windows directory from a host to a linux container so that the existing contents of the windows dir is available from the container?
OK so there were multiple issues here:
1.
I had
version: '3'
at the top of my docker-compose.yml. The long syntax described here wasn't implemented until 3.4 so I stopped receiving the bizarre syntax error when I updated this to:
version: '3.6'
2.
I use my my docker account on 2 windows PCs. Following a hint from another stackoverflow post, I reset Docker to the factory settings. I had to give docker the computer username and password with the notice that this was necessary to access the contents of the local filesystem - at this point I remembered doing this on another PC so I'm not sure whether the credentials were correct on this on. With the correct credentials for the current PC, I was able to bind-mount the volume with the expected results as follows:
version: '3.6'
event_processor:
environment:
- COMPOSE_CONVERT_WINDOWS_PATHS=1
build: ./Docker/event_processor
ports:
- "15672:15672"
entrypoint: python -u /src/event_processor/event_processor.py
networks:
- app_network
volumes:
- type: bind
source: c:/path/to/interesting/directory
target: /interesting_directory
Now it works as expected. I'm not sure if it was the factory reset or the updated credentials that fixed it. I'll find out tomorrow when I use another PC and update.
I'm using docker-compose (v 3.3) and I have a set of services as shown below.
version: '3.3'
services:
jboss:
build:
context: .
dockerfile: jboss/Dockerfile
ports:
- 8080
depends_on:
- mysql
- elasticsearch
networks:
- ${NETWORK}
# - front-tier
# - back-tier
mysql:
hostname: mysql
image: mysql:latest
ports:
- 3306
networks:
- ${NETWORK}
# - back-tier
elasticsearch:
image: elasticsearch:1.7.3
ports:
- 9200
networks:
- ${NETWORK}
# - back-tier
networks:
# front-tier:
# driver: bridge
The problem/question is related to the possibility to isolate these services when I scale my containers in a kind of subnet (by the way, I'm not using swarm here) in the sense that jboss1 can only see mysql1 and elasticsearch1. The same for jboss2 - mysql2 - es2 and so on. I know this is quite strange, but the task is to parallelize some tests and they must be completely isolated.
As you probably have realized, I've tried some approaches (that are commented in the compose) of defining some networks, but if I scale the containers, they will obviously be on the same network - which means that I could randomly ping on mysql-n from jboss-1.
Then, I tried another approach explained here by Arun Gupta http://blog.arungupta.me/docker-bridge-overlay-network-compose-variable-substitution/ where he mentioned the variable substitution in the network attribute (for that the ${NETWORK} there). But it turns out that if I try to "up" my containers via:
NETWORK=isolated-net1 docker-compose up -d
and then
NETWORK=isolated-net2 docker-compose up -d
it won't scale the containers, but instead, recreate them all:
Recreating docker_mysql_1 ...
Recreating docker_elasticsearch_1 ...
Recreating docker_jboss_1 ...
In a nutshell: Is there a way to isolate a group of services when doing a docker-compose up --scale?
Thanks
Well, it turns out that after a few days where I've been beating my brains out and trying to find a proper out-of-the-box solution for the problem, the feasible way to achieve this (actually, the one which fits my needs) was doing the following:
I. Creating the specific volume for the instance that would call the other containers (in this case, JBoss):
jboss:
build:
context: jboss
dockerfile: Dockerfile
volumes:
- /var/run/docker.sock:/var/run/docker.sock
# ... omitting the rest
II. Slightly editing my Dockerfile with a script that really does the isolation:
FROM my-private-jboss:latest
ADD isolateNetwork.sh /opt/jboss/jboss-eap/jboss-as/server/default/bin/isolateNetwork.sh
RUN chmod +x /opt/jboss/jboss-eap/jboss-as/server/default/bin/isolateNetwork.sh
CMD ["/opt/jboss/jboss-eap/jboss-as/server/default/bin/isolateNetwork.sh"]
III. Finally, the isolateNetwork.sh (warning: I had to install jq on my JBoss container to help in these extractions from docker API):
#!/usr/bin/env bash
curl -s --unix-socket /var/run/docker.sock http://localhost/containers/$(hostname)/json > container.json
export DOCKER_CONTAINER_NUMBER=$(cat container.json | jq -r '.Config.Labels["com.docker.compose.container-number"]')
export DOCKER_PROJECT_NAME=$(cat container.json | jq -r '.Config.Labels["com.docker.compose.project"]')
export MYSQL_CONTAINER_NAME=${DOCKER_PROJECT_NAME}_mysql_${DOCKER_CONTAINER_NUMBER}
export MYSQL_IP=$(curl -s --unix-socket /var/run/docker.sock http://localhost/containers/$MYSQL_CONTAINER_NAME/json | jq -r '.NetworkSettings.Networks[].IPAddress')
#Now it's just required to edit the Datasources
find . -iname "MySql*-ds.xml" -exec sed -i s/mysql:3306/${MYSQL_CONTAINER_NAME}:3306/g {} +
As you can see, the volume created during step 1 serves to access docker API from inside a container. This way, I could query the info mentioned and edit the configuration to invoke just upon the same DOCKER_CONTAINER_NUMBER. Apart from that, it is important to mention that my command to scale has a very specific requirement that is: every Jboss will have the same number of data-sources, which implies that DOCKER_CONTAINER_NUMBER will ever match.
docker-compose up --scale jboss=%1 --scale elasticsearch=%1 --scale mysql=%1 --no-recreate -d
I'm working with docker-compose for a laravel project and nginx.
This is my docker-compose.yml :
version: '2'
services:
backend:
image: my_image
depends_on:
- elastic
- mysql
mysql:
image: mysql:8.0.0
nginx:
depends_on:
- backend
image: my_image
ports:
- 81:80
So, my Laravel project is in the container backend, and If I run the command : docker-compose up -d it's ok, all containers are created and my project is running on port 81.
My problem is, In the Laravel project in my container backend, I have a .env file with database login, password and other stuff.
How can I edit this file after docker-compose up ? Directly in the container is not a good idea, is there a way to link a file outside a container with docker-compose ?
Thanks
One approach to this is to use the 'env_file' directive on the docker-compose.yml, in there you can put a list of key value pairs that will be exported into the container. For example:
web:
image: nginx:latest
env_file:
- .env
ports:
- "8181:80"
volumes:
- ./code:/code
Then you can configure your application to use these env values.
One catch with this approach is that you need to recreate the containers if you change any value or add a new one (docker-compose down && docker-compose up -d).
Here is my docker-compose.yml file:
version:'2':
services:
redis:
image: redis
environment:
- HOST='localhost'
- PORT=6379
ports:
-"0.0.0.0:${PORT}:6379"
I get this error on running docker-compose up:
ERROR: The Compose file './docker-compose.yml' is invalid because:
Invalid service name 'services' - only [a-zA-Z0-9\._\-] characters are allowed
Unsupported config option for services: 'redis'
There are multiple problems with your file. The one causing the syntax error is that you have an extra colon on the first line:
version:'2':
that way you define a scalar string key version:'2' with value of null. Since you are therefore not defining the version of the docker compose file, the rest of the file (which is version 2 oriented) fails. This is best resolved by adding a space after version:
In addition your ports definition is incorrect, the value for that should be a sequence/list, and you again specify a scalar string -"0.0.0.0:${PORT}:6379" because there is no space after the initial dash.
Change your docker_compose.yaml file to:
version: '2'
# ^ no colon here
# ^ space here
services:
redis:
image: redis
environment:
- HOST='localhost'
- PORT=6379
ports:
- "0.0.0.0:${PORT}:6379"
# ^ extra space here
just remove last character ":" into string version:'2':
after it docker-compose.yml must be like
version:'2'
services:
redis:
image: redis
environment:
- HOST='localhost'
- PORT=6379
ports:
-"0.0.0.0:${PORT}:6379"