I'm attempting to put a ruby Cucumber test into Docker. I'm using a docker-compose.yml file to start a selenium hub container along with a chrome and firefox node. Then I'm building an alpine ruby based image with my tests.
I've gotten the process to work, however it involves finding the IP of the hub container each time it is built, and then hardcoding the IP into my env.rb file where I connect to the Selenium grid.
I've seen that containers that are linked can be connected using the name but haven't had much luck there. Is there any way I can easily pass the hub container IP to my test's container?
Here is my yml file:
version: "3"
services:
hub:
image: selenium/hub
ports:
- "4444:4444"
environment:
GRID_MAX_SESSION: 16
GRID_BROWSER_TIMEOUT: 3000
GRID_TIMEOUT: 3000
chrome:
image: selenium/node-chrome
container_name: web-automation_chrome
depends_on:
- hub
environment:
HUB_PORT_4444_TCP_ADDR: hub
HUB_PORT_4444_TCP_PORT: 4444
NODE_MAX_SESSION: 4
NODE_MAX_INSTANCES: 4
volumes:
- /dev/shm:/dev/shm
ports:
- "9001:5900"
links:
- hub
firefox:
image: selenium/node-firefox
container_name: web-automation_firefox
depends_on:
- hub
environment:
HUB_PORT_4444_TCP_ADDR: hub
HUB_PORT_4444_TCP_PORT: 4444
NODE_MAX_SESSION: 2
NODE_MAX_INSTANCES: 2
volumes:
- /dev/shm:/dev/shm
ports:
- "9002:5900"
links:
- hub
myapp:
build: .
image: justinpshields/myapp
depends_on:
- hub
environment:
URL: hub
links:
- hub
networks:
default:
links is useless. Every container in a docker-compose.yml share the same network unless stated otherwise.
You should also wait until the selenium hub start and attach its browsers containers.
For instance with that:
while ! curl -sSL "http://$SELENIUMHUBHOST:4444/status" 2>&1 | grep "\"ready\": true" >/dev/null; do
echo 'Waiting for the Grid'
sleep 1
done
while ! curl -sSL "http://$SELENIUMHUBHOST:4444/status" 2>&1 | grep "\"browserName\": \"$BROWSER\"" >/dev/null; do
echo "Waiting for the node $BROWSER"
sleep 1
done
Related
Let me start off by stating that I know this question has been asked on many forums. I have read them all.
I have two Docker containers that are built with docker-compose and contain a Laravel project each. They are both attached to a network and can ping one another successfully, however, when I make a request from Postman to the one backend that then makes a curl request to the other, I get the connection refused error shown below.
This is my docker-compose file for each project respectfully:
version: '3.8'
services:
bumblebee:
build:
context: .
dockerfile: Dockerfile
volumes:
- .:/app
networks:
- picknpack
ports:
- "8010:8000"
networks:
picknpack:
external: true
version: '3.8'
services:
optimus:
build:
context: .
dockerfile: Dockerfile
volumes:
- .:/app
ports:
- "8020:8000"
networks:
- picknpack
depends_on:
- optimus_db
optimus_db:
image: mysql:8.0.25
environment:
MYSQL_DATABASE: optimus
MYSQL_USER: test
MYSQL_PASSWORD: test1234
MYSQL_ROOT_PASSWORD: root
volumes:
- ./storage/dbdata:/var/lib/mysql
ports:
- "33020:3306"
networks:
picknpack:
external: true
Here you can see the successful ping:
I would love to keep messing with configuration files but I have a deadline to meet and nothing is working, any help would be appreciated.
EDIT
Please see inspection of network:
Within the docker network that I created, both containers are exposed on port 8000 as per their Dockerfiles. It was looking at me square in the face: 'Connection refused on port 80'. The HTTP client was using that as default rather than 8000. I updated the curl request to hit port 8000 and it works now. Thanks to #user3532758 for your help. Note that the containers are mapped to ports 8010 and 8020 in the external local network, not within the docker network. There they are both served on port 8000 with different IPs
I am new to Docker and I am trying to run Selenium Grid tests on Docker. For this purpose, I created a docker compose file and executed below command
docker-compose -f docker-compose.yaml up
Everything worked fine but after a few hours I restarted host machine and executed above command again. This time I get below error
ERROR: for selenium-hub Cannot create container for service selenium-hub: Conflict. The container name "/selenium-hub" is already in use by container "some-hash". You have to remove (or rename) that container to be able to reuse that name.
I tried docker-compose -f docker-compose.yaml run selenium-hub but this command does not start selenium nodes. So my questions are -
Do I need to remove the container everytime before I run the docker compose
again?
Is there any way I can use docker-compose like file, so that
everytime I restart docker, I can just run the file to start all containers together?
Below the Docker-Compose I used
version: "3"
services:
selenium-hub:
image: selenium/hub:3.141.59-20200525
container_name: selenium-hub
ports:
- "4444:4444"
chrome:
image: selenium/node-chrome:3.141.59-20200525
volumes:
- /dev/shm:/dev/shm
depends_on:
- selenium-hub
environment:
- HUB_HOST=selenium-hub
- HUB_PORT=4444
firefox:
image: selenium/node-firefox:3.141.59-20200525
volumes:
- /dev/shm:/dev/shm
depends_on:
- selenium-hub
environment:
- HUB_HOST=selenium-hub
- HUB_PORT=4444
opera:
image: selenium/node-opera:3.141.59-20200525
volumes:
- /dev/shm:/dev/shm
depends_on:
- selenium-hub
environment:
- HUB_HOST=selenium-hub
- HUB_PORT=4444
There are possible ways
docker system prune will clean up the cache and remove the dangling intermediate container and delete containers name that not actively running. This command have to be used carefully
docker container prune will delete only dead/stop containers and will free up names
docker rm -v $(docker ps -aq -f 'status=exited')
docker rmi $(docker images -aq -f 'dangling=true')
docker-compose rm --force emoves one-off containers created by docker-compose up or docker-compose run
Just use docker-compose up command to start the already created containers.
I have 3 dockerized services. Services A and B run inside same docker-compose file:
docker-compose.yml
version: '3.5'
services:
service_a:
container_name: service_a
networks:
- my_net
service_b:
container_name: service_b
networks:
- my_net
networks: #This is just because I wanted to change the network default name
my_net:
name: my_net
Service C needs make requests against services A and B, but it runs separately using docker without compose (that's because I'm passing --network option). So, I run service C linking A and B:
docker run --network my_net --link service_a --link service_b service_c_docker_image
This is working on MacOS, but not in Ubuntu!
If I run ping command, instead of default service_c_docker_image command:
docker run --network my_net --link service_a --link service_b service_c_docker_image ping service_a
on MacOS, the host is reached properly; on Ubuntu, I get: ping: service_a: Name or service not known. And same with service_b.
Both machines are using same version of docker and docker-compose.
What am I missing?
You may have a typo in your question as that compose file should not run at all, the service level network names my_net should match the top level network name which then can be renamed using name: intra_net. The network set in the docker run command should match what the network was renamed to in the top level networks section (and that network needs to already exist, so run the compose stack first).
working example:
docker-compose.yaml
version: '3.5'
services:
service_a:
image: odise/busybox-curl
command: ["curl", "-s", "service_b:5678"]
depends_on:
- service_b
networks:
- my_net
service_b:
image: hashicorp/http-echo
command: ["-text", "hello world"]
networks:
- my_net
networks:
my_net:
name: infra_network
Run the services docker-compose up -d and check the logs:
> docker-compose logs
Attaching to docker-compose-networks_service_a_1, docker-compose-networks_service_b_1
service_b_1 | 2019/01/06 05:53:55 Server is listening on :5678
service_b_1 | 2019/01/06 05:53:55 service_b:5678 172.19.0.3:46900 "GET / HTTP/1.1" 200 12 "curl/7.39.0" 106.6µs
service_a_1 | hello world
Then start the other container with docker
> docker run --network infra_network odise/busybox-curl curl -s service_b:5678
hello world
Silly me. Actually, my configuration is right but services A and B were not run because an application level error, so links were not working.
I have developed a Blockchain network with hyperledge composer over development environment as the documentation shows. I have tested it and works nice. So I want to build a production network. At this moment, my first objective is add more peers to development environment on the same server in order to learn. I have looked the startFabric.sh and I have edit the docker file and this sh but it doesn’t work. I have attached two files that I have edited from original code. The error that it fires me is that container of peer1 isn’t working. The database 2 is working.
I have searched on forums about how I can add more peer but I don’t find a good guide in order to understand how to do step by step.
So my question, what have I done bad? Do you know a good tutorial in order to learn how I add more peers to development environment?
Thank you
startFabric.sh
#!/bin/bash
# Exit on first error, print all commands.
set -ev
#Detect architecture
ARCH=`uname -m`
# Grab the current directory
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
#
cd "${DIR}"/composer
ARCH=$ARCH docker-compose -f "${DIR}"/composer/docker-compose.yml down
ARCH=$ARCH docker-compose -f "${DIR}"/composer/docker-compose.yml up -d
# wait for Hyperledger Fabric to start
# incase of errors when running later commands, issue export FABRIC_START_TIMEOUT=<larger number>
echo ${FABRIC_START_TIMEOUT}
sleep ${FABRIC_START_TIMEOUT}
# Create the channel
docker exec peer0.org1.example.com peer channel create -o orderer.example.com:7050 -c composerchannel -f /etc/hyperledger/configtx/composer-channel.tx
docker exec peer1.org1.example.com peer channel create -o orderer.example.com:7050 -c composerchannel -f /etc/hyperledger/configtx/composer-channel1.tx
# Join peer0.org1.example.com to the channel.
docker exec -e "CORE_PEER_MSPCONFIGPATH=/etc/hyperledger/msp/users/Admin#org1.example.com/msp" peer0.org1.example.com peer channel join -b composerchannel.block
docker exec -e "CORE_PEER_MSPCONFIGPATH=/etc/hyperledger/msp/users/Admin#org1.example.com/msp" peer1.org1.example.com peer channel join -b composerchannel.block
cd ../..
docker-composer.yml
version: '2'
services:
ca.org1.example.com:
image: hyperledger/fabric-ca:$ARCH-1.0.1
environment:
- FABRIC_CA_HOME=/etc/hyperledger/fabric-ca-server
- FABRIC_CA_SERVER_CA_NAME=ca.org1.example.com
# - FABRIC_CA_SERVER_CA_CERTFILE=/etc/hyperledger/fabric-ca-server-config/org1.example.com-cert.pem
# - FABRIC_CA_SERVER_CA_KEYFILE=/etc/hyperledger/fabric-ca-server-config/a22daf356b2aab5792ea53e35f66fccef1d7f1aa2b3a2b92dbfbf96a448ea26a_sk
ports:
- "7054:7054"
command: sh -c 'fabric-ca-server start --ca.certfile /etc/hyperledger/fabric-ca-server-config/ca.org1.example.com-cert.pem --ca.keyfile /etc/hyperledger/fabric-ca-server-config/19ab65a$
volumes:
- ./crypto-config/peerOrganizations/org1.example.com/ca/:/etc/hyperledger/fabric-ca-server-config
container_name: ca.org1.example.com
orderer.example.com:
container_name: orderer.example.com
image: hyperledger/fabric-orderer:$ARCH-1.0.1
environment:
- ORDERER_GENERAL_LOGLEVEL=debug
- ORDERER_GENERAL_LISTENADDRESS=0.0.0.0
- ORDERER_GENERAL_GENESISMETHOD=file
- ORDERER_GENERAL_GENESISFILE=/etc/hyperledger/configtx/composer-genesis.block
- ORDERER_GENERAL_LOCALMSPID=OrdererMSP
- ORDERER_GENERAL_LOCALMSPDIR=/etc/hyperledger/msp/orderer/msp
working_dir: /opt/gopath/src/github.com/hyperledger/fabric
command: orderer
ports:
- 7050:7050
volumes:
- ./:/etc/hyperledger/configtx
- ./crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/msp:/etc/hyperledger/msp/orderer/msp
peer0.org1.example.com:
container_name: peer0.org1.example.com
image: hyperledger/fabric-peer:$ARCH-1.0.1
environment:
- CORE_LOGGING_PEER=debug
- CORE_CHAINCODE_LOGGING_LEVEL=DEBUG
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
- CORE_PEER_ID=peer0.org1.example.com
- CORE_PEER_ADDRESS=peer0.org1.example.com:7051
- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=composer_default
- CORE_PEER_LOCALMSPID=Org1MSP
- CORE_PEER_MSPCONFIGPATH=/etc/hyperledger/peer/msp
- CORE_LEDGER_STATE_STATEDATABASE=CouchDB
- CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb:5984
working_dir: /opt/gopath/src/github.com/hyperledger/fabric
command: peer node start --peer-defaultchain=false
ports:
- 7051:7051
- 7053:7053
volumes:
- /var/run/:/host/var/run/
- ./:/etc/hyperledger/configtx
- ./crypto-config/peerOrganizations/org1.example.com/peers/peer0.org1.example.com/msp:/etc/hyperledger/peer/msp
- ./crypto-config/peerOrganizations/org1.example.com/users:/etc/hyperledger/msp/users
depends_on:
- orderer.example.com
- couchdb
couchdb:
container_name: couchdb
image: hyperledger/fabric-couchdb:$ARCH-1.0.1
ports:
- 5984:5984
environment:
DB_URL: http://localhost:5984/member_db
peer1.org1.example.com:
container_name: peer1.org1.example.com
image: hyperledger/fabric-peer:$ARCH-1.0.1
environment:
- CORE_LOGGING_PEER=debug
- CORE_CHAINCODE_LOGGING_LEVEL=DEBUG
- CORE_VM_ENDPOINT=unix:///host/var/run/docker.sock
- CORE_PEER_ID=peer1.org1.example.com
- CORE_PEER_ADDRESS=peer1.org1.example.com:7051
- CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE=composer_default
- CORE_PEER_LOCALMSPID=Org1MSP
- CORE_PEER_MSPCONFIGPATH=/etc/hyperledger/peer/msp
- CORE_LEDGER_STATE_STATEDATABASE=CouchDB
- CORE_LEDGER_STATE_COUCHDBCONFIG_COUCHDBADDRESS=couchdb2:5985
working_dir: /opt/gopath/src/github.com/hyperledger/fabric
command: peer node start --peer-defaultchain=false
ports:
- 7061:7061
- 7063:7063
volumes:
- /var/run/:/host/var/run/
- ./:/etc/hyperledger/configtx
- ./crypto-config/peerOrganizations/org1.example.com/peers/peer1.org1.example.com/msp:/etc/hyperledger/peer/msp
- ./crypto-config/peerOrganizations/org1.example.com/users:/etc/hyperledger/msp/users
depends_on:
- orderer.example.com
- couchdb2
couchdb2:
container_name: couchdb2
image: hyperledger/fabric-couchdb:$ARCH-1.0.1
ports:
- 5985:5985
environment:
DB_URL: http://localhost:5984/member_db
We provide a basic Hyperledger Fabric network for development purposes only and isn't meant to be an example to demostrate how to build one. Hyperledger Composer will work with any Hyperledger Fabric setup with the right connection profiles anf Hyperledger Fabric provide documentation and examples on how to build your own networks which I think is what you need
See https://hyperledger-fabric.readthedocs.io/en/latest/build_network.html
about how to build your own network and also see
https://hyperledger.github.io/composer/reference/connectionprofile.html
for information about composer connection profiles.
also see
Does composer support endorsement policy? How?
which provides some info about multi org networks and connection profiles
I am using the docker-compose 'recipe' below to bring up a container that runs a component of the storm stream processing framework. I am finding that on Mac's
when i enter the container (once it is up and running via docker exec -t -i <container-id> bash)
and I do ping storm-supervisor I get the error
'unknown host'. However, when i run the same docker-compose script on Linux
the host is recognized and ping succeeds.
The failure to resolve the host leads to problems with the Storm component... but what
that component is doing can be ignored for this question. I'm pretty sure if I figured out
how to get the Mac's docker-compose behavior to match Linux's then I would have no problem.
I think i am experiencing the issue mentioned in this post:
https://forums.docker.com/t/docker-compose-not-setting-hostname-when-network-mode-host/16728
version: '2'
services:
supervisor:
image: sunside/storm-supervisor
container_name: storm-supervisor
hostname: storm-supervisor
network_mode: host
ports:
- "8000:8000"
environment:
- "LOCAL_HOSTNAME=localhost"
- "NIMBUS_ADDRESS=localhost"
- "NIMBUS_THRIFT_PORT=49627"
- "DRPC_PORT=49772"
- "DRPCI_PORT=49773"
- "ZOOKEEPER_ADDRESS=localhost"
- "ZOOKEEPER_PORT=2181"
thanks in advance for any leads or tips !
"network_mode: host" will not work well on docker mac. I experienced the same issue where I had few of my containers in bridge network and the others in host network.
However, you can move all your containers to a custom bridge network. It solved for me.
You can edit your docker-compose.yml file to have a custom bridge network.
version: '2'
services:
supervisor:
image: sunside/storm-supervisor
container_name: storm-supervisor
hostname: storm-supervisor
ports:
- "8000:8000"
environment:
- "LOCAL_HOSTNAME=localhost"
- "NIMBUS_ADDRESS=localhost"
- "NIMBUS_THRIFT_PORT=49627"
- "DRPC_PORT=49772"
- "DRPCI_PORT=49773"
- "ZOOKEEPER_ADDRESS=localhost"
- "ZOOKEEPER_PORT=2181"
networks:
- storm
networks:
storm:
external: true
Also, execute the below command to create the custom network.
docker network create storm
You can verify it by
docker network ls
Hope it helped.