i have crated the docker stack file as below, it created the 3 services, as excepted but i am unable to access out side of the host. and its not creating any port also. i have created a overlay network called test01. When i create a this manually via command line it works perfectly.
version: '3.0'
networks:
default:
external:
name: test01
services:
mssql:
image: microsoft/mssql-server-windows-developer
environment:
- SA_PASSWORD=Password1
- ACCEPT_EULA=Y
ports:
- 1433:1433
volumes:
- c:\Databases:c:\Databases
deploy:
placement:
constraints: [node.labels.os==Windows]
web:
image: iiswithdb:latest
ports:
- 8080:8080
deploy:
replicas: 3
lbs:
image: nginx:latest
ports:
- 80: 80
deploy:
placement:
constraints: [node.labels.os==Windows]
Your services need to explicitly join the network you are defining. You can do this in the compose file. Otherwise they will use the default network created by the stack/compose. https://docs.docker.com/compose/compose-file/#networks
c:\Program Files\docker>docker network inspect test01
[
{
"Name": "test01",
"Id": "8ffz8xihux13gx1uuhalub5by",
"Created": "2017-09-11T12:30:35.7747711+05:30",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "10.0.0.0/24",
"Gateway": "10.0.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"2f283e7c21608d09a57a7cdef25a836d77c0ceb8030ae15796ff692e43b0eb73": {
"Name": "test_web.1.jti1pyrgxv3v4yet9m9cpk0i4",
"EndpointID": "bed2a5e0d077fcf48ab2d6fe419a8a69a45c3033e1a8602cf6395f93bec405b8",
"MacAddress": "00:15:5d:f3:aa:1a",
"IPv4Address": "10.0.0.5/24",
"IPv6Address": ""
},
"8c55fad8ad54e5286bb7fc54da52ad1958854bceacbf0260400e7dc3c00c1c45": {
"Name": "test_mssql.1.mn31bwoh8iwg5sge5rllh7gc9",
"EndpointID": "00c6e68d6a22ee0dc5ad90cda7ab958323a0b07206ce4583f11baa8b3476de8f",
"MacAddress": "00:15:5d:f3:aa:23",
"IPv4Address": "10.0.0.3/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "4097",
"com.docker.network.windowsshim.hnsid": "b76fa7e3-530d-4133-b72a-1d1818cd3c16"
},
"Labels": {},
"Peers": [
{
"Name": "node2-f3dedf0e26d9",
"IP": "10.30.50.10"
},
{
"Name": "node3-2e1ad7fb91be",
"IP": "10.30.50.13"
}
]
}
]
Below is the output
c:\Program Files\docker>docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
bo9uovidd4z3 test_web replicated 3/3 iiswithdb:latest *:8080->8080/tcp
sujwg53gjnp3 test_lbs replicated 0/1 nginx:latest *:80->80/tcp
vyxyoaji8jkd test_mssql replicated 1/1 microsoft/mssql-server-windows-developer:latest *:1433->1433/tcp
c:\Program Files\docker>docker service ps test_mssql
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
mn31bwoh8iwg test_mssql.1 microsoft/mssql-server-windows-developer:latest node2 Running Running 6 minutes ago
When i inspect SQL server container i can't find any port taged
c:\Program Files\docker>docker service ps test_lbs
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
j4x806u1ucdr test_lbs.1 nginx:latest Running Pending 32 minutes ago
c:\Program Files\docker>docker service ps test_web
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
jti1pyrgxv3v test_web.1 iiswithdb:latest node2 Running Running 22 minutes ago
1gudznmi9ufz \_ test_web.1 iiswithdb:latest node2 Shutdown Failed 27 minutes ago "task: non-zero exit (21479434…"
xxkr98na4qsy test_web.2 iiswithdb:latest node3 Running Running 29 minutes ago
7j1y6vc90qvf test_web.3 iiswithdb:latest node3 Running Running 29 minutes ago
C:\Users\Administrator>docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
19qeljqt3wuf test_mssql replicated 1/1 microsoft/mssql-server-windows-developer:latest *:1433->1433/tcp
48gamfl4j4rl test_web replicated 3/3 iiswithdb:latest *:8080->8080/tcp
nxycxrigmz4u test_lbs replicated 1/1 nginx:latest *:80->80/tcp
C:\Users\Administrator>docker service ps test_lbs
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
81fm4xplekig test_lbs.1 nginx:latest node2 Running Running 25 minutes ago
C:\Users\Administrator>docker service ps test_web
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
aivzt7eagf4f test_web.1 iiswithdb:latest node1 Running Running about an hour ago
sny1zf7osibq test_web.2 iiswithdb:latest node2 Running Running about an hour ago
lwzlpaks1b4t \_ test_web.2 iiswithdb:latest node2 Shutdown Failed about an hour ago "task: non-zero exit (21479434…"
iav5mxqdbzoy test_web.3 iiswithdb:latest node3 Running Running about an hour ago
C:\Users\Administrator>docker service ps test_mssql
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
pfu8qyw7vqxp test_mssql.1 microsoft/mssql-server-windows-developer:latest node2 Running Running 26 minutes ago
Related
I have a simple Springboot app connecting to two different SQL Server database. When all of them are hosted locally, I have no issues. But I need to have each of them in a separated docker container, when I do this I get an SQLServerException at the start of my Springboot app telling me :
com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host '172.21.0.3', port 1434 has failed. Error: "'172.21.0.3'. Verify the connection properties.
Where 172.21.0.3 is the IP of one of my database and 1434 it's port.
I use a docker network called network_gls (which doesn't seems to work) to connect my containers (gls_app, mssql_1 & mssql_2) together, when I execute :
docker inspect network_gls
(NOTE : The execution of this line is after the start of the Springboot app container & before it's error)
I get the following result :
[
{
"Name": "network_gls",
"Id": "88895acb2247b3b63b0cc29656fcb6d1a0d4a8192a8c7c1bb7b79362509e0742",
"Created": "2020-09-28T15:21:39.995019917Z",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
{
"Subnet": "172.21.0.0/16",
"Gateway": "172.21.0.1"
}
]
},
"Internal": false,
"Attachable": false,
"Ingress": false,
"ConfigFrom": {
"Network": ""
},
"ConfigOnly": false,
"Containers": {
"0754d8766736806549e99500c143420c556e9370c14f897f6beb82c24a3c1124": {
"Name": "mssql_1",
"EndpointID": "6d886cf8f2aed256d8cbc7141d9ea5242f7ce61d95ae5412c16905d1b490f133",
"MacAddress": "02:42:ac:15:00:02",
"IPv4Address": "172.21.0.2/16",
"IPv6Address": ""
},
"54d20a9a409053eaf53eb5c7e73e340ab29c12ceaf8ac20b109d1403cba0c3d3": {
"Name": "mssql_2",
"EndpointID": "e675f72fc6c737201a31dd485496e749d386165eaa90a6647e0bf13507683028",
"MacAddress": "02:42:ac:15:00:03",
"IPv4Address": "172.21.0.3/16",
"IPv6Address": ""
},
"7e4ae1a46358fe9081c5277cb52ec49681b44631d6d9c1cdcaf6116326277d37": {
"Name": "gls_app",
"EndpointID": "d9051cd0134f5074b2b756b44b60cced85d2cac2fd04653e0f52ddb9ada339b9",
"MacAddress": "02:42:ac:15:00:04",
"IPv4Address": "172.21.0.4/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {}
}
]
And in my Springboot application, my connection string looks like this (example of the database in mssql_2) :
jdbc:sqlserver://172.21.0.3:1433;DatabaseName=gls
The docker networking aspect is new to me, tell me if I'm missing important information in this question
Thanks in advance
In my case it does work, when I use the Container Name instead of the IP address.
So instead of:
jdbc:sqlserver://172.21.0.3:1433;DatabaseName=gls
try this:
jdbc:sqlserver://mssql_2:1433;DatabaseName=gls
Also you can try to publish your ports, if you want to test if you have problems with your networking.
https://docs.docker.com/config/containers/container-networking/
Edit: Thanks to David for pointing out container_name is not required. It can be connected using service name.
You can create a docker compose and use it to start your DB. Given is an example of docker compose you use and application.properties.
You can use your docker service name while connecting to db from another container. To connect from localhost, in most cases port are exposed as port:port, it can be accessed as localhost.
docker-compose build web .
docker-compose up db
docker-compose up web
or
docker-compose up
You can use localhost when not accessing it from container
Docker Compose File
version: "3.3"
services:
web:
build:
context: ./
dockerfile: Dockerfile
image: web:latest
ports:
- 8080:8080
environment:
POSTGRES_JDBC_USER: UASENAME
POSTGRES_JDBC_PASS: PASSWORD
SPRING_DATASOURCE_URL: "jdbc:postgresql://db:5432/DATABASE"
SPRING_PROFILES_ACTIVE: dev
command: mvn spring-boot:run -Dspring.profiles.active=dev
depends_on:
- db
- rabbitmq
db:
image: "postgres:9.6-alpine"
ports:
- 5432:5432
expose:
- 5432
volumes:
- postgres:/var/lib/postgresql/data
environment:
POSTGRES_USER: USERNAME
POSTGRES_PASSWORD: PASSWORD
POSTGRES_DB: DATABASE
volumes:
postgres:
app:
This is application properties (for local development):
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
spring.datasource.url=jdbc:postgresql://localhost:5432/database
spring.datasource.username=USERNAME
spring.datasource.password=PASSWORD
Hope this will answer.
I am using the Consul Docker Image on dockerhub. I wanted to know if there is a way to store the Key/Value settings in a config that the docker images can load on boot. I understand that the Image has the /consul/config and /consul/data volumes that can be used. but I have not found a way to achieve this.
The following is how I run consul
version: '3.4'
service:
consul:
container_name: consul
image: consul:latest
ports:
- "8500:8500"
- "8300:8300"
volumes:
- ./consul:/consul/config
In my host consul dir I have a file called config.json which contains the following
{
"node_name": "consul_server",
"data_dir": "/data",
"log_level": "INFO",
"client_addr": "0.0.0.0",
"bind_addr": "0.0.0.0",
"ui": true,
"server": true,
"bootstrap_expect": 1
}
I have a laravel project which should be a wrap to the docker containers. I added containers for PHP, MySQL, Redis, and Nginx. But laravel-echo-server doesn't work. All connections are failed.
My docker-compose.yml config:
version: '3.7'
services:
redis:
image: redis:alpine
container_name: wex_redis
volumes:
- redis_data:/data
command: redis-server --appendonly yes
networks:
- wex-network
app:
build:
context: ./
dockerfile: docker/containers/app/Dockerfile
working_dir: /app
container_name: wex_app
volumes:
- ./:/app
environment:
- DB_PORT=3306
- DB_HOST=database
networks:
- wex-network
web:
build:
context: ./
dockerfile: docker/containers/web/Dockerfile
working_dir: /app
container_name: wex_web
ports:
- 8082:80
- 6001:6001
networks:
- wex-network
database:
image: mysql:8
container_name: wex_db
volumes:
- dbdata:/var/lib/mysql
environment:
- MYSQL_DATABASE=${DB_DATABASE:-wex}
- MYSQL_ROOT_PASSWORD=root
ports:
- 33062:3306
command: --default-authentication-plugin=mysql_native_password
networks:
- wex-network
networks:
wex-network:
driver: bridge
volumes:
dbdata:
redis_data:
"web" container need for install npm dependensies and set nginx. Dockerfile for web container:
FROM node:10-alpine as build
WORKDIR /app
# Load dependencies
COPY ./package*.json ./yarn.lock ./
COPY ./webpack.mix.js ./
RUN yarn install
COPY ./resources/assets /app/resources/assets
COPY ./public/assets /app/public/assets
# Build
RUN yarn run production
# Laravel-echo-server
RUN yarn global add --prod --no-lockfile laravel-echo-server \
&& yarn cache clean
EXPOSE 6001
CMD ["laravel-echo-server", "start"]
# NGINX
FROM nginx:alpine
EXPOSE 80
COPY ./docker/containers/web/nginx.conf /etc/nginx/conf.d/default.conf
COPY ./public /app/public
COPY --from=build /app/public/assets /app/public/assets
Settings for laravel-echo-server:
{
"authHost": "http://localhost",
"authEndpoint": "/broadcasting/auth",
"clients": [],
"database": "redis",
"databaseConfig": {
"redis": {},
"sqlite": {
"databasePath": "/database/laravel-echo-server.sqlite"
}
},
"devMode": true,
"host": null,
"port": "6001",
"protocol": "http",
"socketio": {},
"sslCertPath": "",
"sslKeyPath": "",
"sslCertChainPath": "",
"sslPassphrase": "",
"subscribers": {
"http": true,
"redis": true
},
"apiOriginAllow": {
"allowCors": false,
"allowOrigin": "",
"allowMethods": "",
"allowHeaders": ""
}
}
Connection to the laravel-echo-server from js:
import Echo from 'laravel-echo'
window.io = require('socket.io-client');
window.Echo = new Echo({
broadcaster: 'socket.io',
host: window.location.hostname + ':6001'
});
"docker ps" command:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dddc2f5867b6 wex_app "/var/entrypoint.sh …" 43 minutes ago Up 43 minutes 9000/tcp wex_app
5fb70cf7da97 wex_web "nginx -g 'daemon of…" 19 hours ago Up 43 minutes 0.0.0.0:6001->6001/tcp, 0.0.0.0:8082->80/tcp wex_web
f2f331bf37e0 redis:alpine "docker-entrypoint.s…" 7 days ago Up 43 minutes 6379/tcp wex_redis
224e2af8c151 mysql:8 "docker-entrypoint.s…" 7 days ago Up 43 minutes 33060/tcp, 0.0.0.0:33062->3306/tcp wex_db
Could you please ask when I did a mistake? Why all connections to the laravel-echo server don't work?
Have a look for your Dockerfile, it use multi-stage builds.
There are 2 stages here, CMD ["laravel-echo-server", "start"] just in the first stage. But multi-stage builds's aim is to generate a final image with last stage. All previous stage is just intermediate to afford things to last stage.
So, when your docker-compose run web service, it just use the final image to start a container, so the container will just call ENTRYPOINT or CMD of last stage which will not start laravel-echo-server.
I have a docker swarm running a number of services. I'm using the elastic stack (kibana, elastic, filebeat, etc) for monitoring.
For the business logic I'm writing logs and using filebeat to move them to logstash and analyze the data in kibana.
But I'm having trouble in monitoring the liveness of my docker services. some of them are deployed globally (like filebeat) and some of them have a number of replicas. I wan't to be able to see in kibana that the number of running containers is equal to the number that the service should have. I'm trying to use metricbeat with docker module, the most useful metricset I've found is container, but it doesn't seem to contain enough information for me to display or analyze the number of instances of a service.
I'd appreciate any advice how to achieve this.
The metricbeat config
metricbeat.autodiscover:
providers:
- type: docker
hits.enabled: true
metricbeat.modules:
- module: docker
enabled: true
metricsets:
- container
- healthcheck
- info
period: 10s
hosts: [ "unix:///var/run/docker.sock" ]
processors:
- add_docker_metadata: ~
- add_locale:
format: offset
output.logstash:
hosts: [ "mylogstash.com" ]
The metricset container log data (the relevant docker part)
...
"docker" : {
"container": {
"id": "60983ad304e13cb0245a589ce843100da82c5fv9e093aad68abb439cdc2f3044"
"status": "Up 3 weeks",
"command": "./entrypoint.sh",
"image": "registry.com/myimage",
"created": "2019-04-08T11:38:10.000Z",
"name": "mystack_myservice.wuiqep73p99hcbto2kgv6vhr2.mufs70y24k5388jxv782in18f",
"ip_addresses": [ "10.0.0.148" ]
"labels" : {
"com_dokcer_swarm_node_id": "wuiqep73p99hcbto2kgv6vhr2",
"com_docker_swarm_task_name": "stack_service.wuiqep73p99hcbto2kgv6vhr2.mufs70y24k5388jxv782in18f",
"com_docker_swarm_service_id": "kxm5dk43yzyzpemcbz23s21xo",
"com_docker_swarn_task_id": "mufs70y24k5388jxv782in18f",
"com_docker_swarm_task" : "",
"com_docker_stack_namespace": "mystack",
"com_docker_swarm_service_name": "mystack_myservice"
},
"size": {
"rw": 0,
"root_fs": 0
}
}
}
...
For future reference:
I wrote a bash script which runs by interval and write a json log for each of the swarm services. the script is wrapped in image docker service logger
I am trying to set up a Jelastic clustered database as described in Setting Up Auto-Clusterization with Cloud Scripting but I don't see documentation there that describes how to either set or retrieve the cluster username and password.
I did try passing db_user and db_pass to the cluster, names I found in some of the sample JPS files, as well as having those as settings but the credentials were still just the Jelastic generated ones.
Here is the JPS I am trying to use; it includes a simple Debian container that requires the database credentials as environment variables. In this case the Docker container includes just the MariaDB client for testing purpose, the real environment is bit more complex than that, running scripts in the startup that need the database connection.
{
"version": "1.5",
"type": "install",
"name": "Database test",
"skipNodeEmails": true,
"globals":
{
"MYSQL_ROOT_USERNAME": "root",
"MYSQL_ROOT_PASSWORD": "${fn.password(20)}",
"MYSQL_USERNAME": "username",
"MYSQL_PASSWORD": "${fn.password(20)}",
"MYSQL_DATABASE": "database",
"MYSQL_HOSTNAME": "ProxySQL"
},
"nodes":
[
{
"image": "mireiawen/debian-sql",
"count": 1,
"cloudlets": 8,
"nodeGroup": "vds",
"displayName": "SQL worker",
"env":
{
"MYSQL_ROOT_USERNAME": "${globals.MYSQL_ROOT_USERNAME}",
"MYSQL_ROOT_PASSWORD": "${globals.MYSQL_ROOT_PASSWORD}",
"MYSQL_USERNAME": "${globals.MYSQL_USERNAME}",
"MYSQL_PASSWORD": "${globals.MYSQL_PASSWORD}",
"MYSQL_DATABASE": "${globals.MYSQL_DATABASE}",
"MYSQL_HOSTNAME": "${globals.MYSQL_HOSTNAME}"
}
},
{
"nodeType": "mariadb-dockerized",
"nodeGroup": "sqldb",
"count": "2",
"cloudlets": 16,
"cluster":
{
"scheme": "master"
}
}
]
}
This JPS seems to launch the MariaDB master-master cluster correctly with the ProxySQL included, I am just lacking on the documentation about how to either provide the database credentials to the database cluster, or a way to retrieve the generated ones to be used as variables in the JPS to send those to the containers.
The mechanism has been improved so now you can pass custom credentials to cluster using either environment variables or cluster settings:
type: install
name: env. variables
nodes:
nodeType: mariadb-dockerized
nodeGroup: sqldb
count: 2
cloudlets: 8
env:
DB_USER: customuser
DB_PASS: custompass
cluster:
scheme: master
or
type: install
name: cluster settings
nodes:
nodeType: mariadb-dockerized
nodeGroup: sqldb
count: 2
cloudlets: 8
cluster:
scheme: master
db_user: customuser
db_pass: custompass
Thank you for the good question. The mechanism of passing custom credentials should be and will be improved soon. At the moment you can use the example below. In short, we disable automated clustering and enable it again with custom username and password.
---
version: 1.5
type: install
name: Database test
skipNodeEmails: true
baseUrl: https://raw.githubusercontent.com/jelastic-jps/mysql-cluster/master
globals:
logic_jps: ${baseUrl}/addons/auto-clustering/scripts/auto-cluster-logic.jps
MYSQL_USERNAME: username
MYSQL_PASSWORD: ${fn.password(20)}
nodes:
- image: mireiawen/debian-sql
count: 1
cloudlets: 8
nodeGroup: extra
displayName: SQL worker
env:
MYSQL_USERNAME: ${globals.MYSQL_USERNAME}
MYSQL_PASSWORD: ${globals.MYSQL_PASSWORD}
- nodeType: mariadb-dockerized
nodeGroup: sqldb
count: 2
cloudlets: 16
cluster: false
onInstall:
install:
jps: ${globals.logic_jps}
envName: ${env.envName}
nodeGroup: sqldb
settings:
path: ${baseUrl}
scheme: master
logic_jps: ${globals.logic_jps}
db_user: ${globals.MYSQL_USERNAME}
db_pass: ${globals.MYSQL_PASSWORD}
repl_user: repl-${fn.random}
repl_pass: ${fn.password(20)}
After environment is ready, you can test the connection by executing the following command in your docker image:
mysql -h proxy -u $MYSQL_USERNAME -p$MYSQL_PASSWORD