I have following stage in gitlab yml file.
integration-tests:
image: mydocker-hub-id/mvn-intergration-tests-image:latest
stage: test
services:
- postgres
script:
- export PGPASSWORD=$POSTGRES_PASSWORD
- psql -h "postgres" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "SELECT 'OK' AS status;"
- cd source_code
- pg_isready
- mvn verify -P test-ci -DskipUTs=true
When job is executed for command pg_isready I have response 5432 no response.
When tests are executed from each intergration test I have error:
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
Image used in this stage is built based on postgres image which has additionally installed maven and java 11.
What could be the issue that spring application cannot connect with postgres server. Command postgres --version displays proper version. Also command
"psql -h "postgres" -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "SELECT 'OK' AS status;"
returns the OK result.
Spring db connection in app.properties is following:
spring.datasource.url=jdbc:postgresql://localhost:5432/test
I found in gitlab documentation details about postgres service configuration
https://docs.gitlab.com/ee/ci/services/postgres.html?fbclid=IwAR1eDlOQ4ACn6zqw1auIetB07JRcCGi1Pjl-hpfGBM45ujvkImm6fHSSbgg
The db url should contain the name of host postgres instead of localhost.
I found gitlab-runner is running the postgresql docker image as postgres and this is working changing jdbc:postgresql://localhost:5432/dbname into jdbc:postgresql://postgres:5432/dbname.
Related
I am trying to set up automated integration tests against an Oracle database and am
planning on using https://www.testcontainers.org/ to
launch the containers in my docker-compose.yml.
I have succeeded in getting Dockerized Oracle running on my laptop
using these instructions:
https://github.com/oracle/docker-images/tree/master/OracleWebLogic/samples/12212-oradb-wlsstore
The steps are:
# see https://docs.oracle.com/cd/E37670_01/E75728/html/oracle-registry-server.html
docker login container-registry.oracle.com # supply your oracle credentials as described above
docker network create -d bridge SampleNET
cat > env.txt <<EOF
DB_SID=InfraDB
DB_PDB=InfraPDB1
DB_DOMAIN=us.oracle.com
DB_BUNDLE=basic
EOF
docker run -d --name InfraDB --network=SampleNET -p 1521:1521 -p 5500:5500 --env-file env.txt -it --shm-size="8g" store/oracle/database-enterprise:12.2.0.1
After a minute when I issue the command "sudo docker ps -a" I see the DB is up and healthy.
CONTAINER ID IMAGE COMMAND
6a276a311b2e store/oracle/database-enterprise:12.2.0.1 "/bin/sh -c '/bin/ba…"
CREATED STATUS PORTS
2 minutes ago Up 2 minutes (healthy) 0.0.0.0:1521->1521/tcp, 0.0.0.0:5500->5500/tcp
NAMES
InfraDB
And I can now run sqlplus (from a shell prompt on my laptop, not a Docker container):
sqlplus sys/Oradoc_db1#localhost:1521/InfraDB.us.oracle.com AS SYSDBA
Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
SQL>
My goal now is to create a Docker compose file that allows me to
the same thing as what was accomplished with the command line invocation of Dockerized
Oracle. Here is the docker-compose.yml file I tried:
version: '2'
services:
database:
image: store/oracle/database-enterprise:12.2.0.1
ports:
- 1521:1521
- 8080:8080
- 5500:5500
environment:
- DB_SID:InfraDB
- DB_PDB:InfraPDB1
- DB_DOMAIN:us.oracle.com
- DB_BUNDLE:basic
networks:
default:
external:
name: SampleNET
So, with this docker compose file I can launch a shell in the
container running my Oracle DB and run sqlplus:
container=`sudo docker ps -a | grep oracle | sed -e's/\s.*//'`
sudo docker exec -t -i $container bash
# SESSION INSIDE DOCKER CONTAINER:
[oracle#8f26f224db03 /]$ sqlplus sys/Oradoc_db1 AS SYSDBA
SQL*Plus: Release 12.2.0.1.0 Production on Tue Mar 3 02:44:03 2020
Copyright (c) 1982, 2016, Oracle. All rights reserved.
Now, the million dollar question:
What is the syntax of the sqlplus command I would specify
to connect FROM MY LAPTOP (i.e., not launching a shell in the
container running the DB server)?
Here are some of the many variants I tried (note: I tried with localhost, 127.0.0.1, and
the actual IP of my laptop):
sqlplus sys/Oradoc_db1#'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=localhost)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=InfraDB)))'
sqlplus sys/Oradoc_db1#'(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=tcp)(HOST=127.0.0.1)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=InfraDB.us.oracle.com)))'
Diagnostic Note:
I am running Docker on Mac.
After I launch Docker-ized Oracle I do see listeners on the ports that the docker compose configuration is forwarding, as shown below:
sudo lsof -i -P | grep -i "listen" | egrep "1521|8080|5500"
com.docke 6295 christopher.bedford 33u IPv6 0xfffffffffffffff 0t0 TCP *:1521 (LISTEN)
com.docke 6295 christopher.bedford 34u IPv6 0xfffffffffffffff 0t0 TCP *:5500 (LISTEN)
com.docke 6295 christopher.bedford 38u IPv6 0xfffffffffffffff 0t0 TCP *:8080 (LISTEN)
Problem Solved
Here is a summary of what I did to solve the problem thanks to the suggestion of my friend Matt (Changos Muertos below).
1). creation of the bridge network 'SampleNET' is not required.
2). seems like specifying environment variables inline like this (below) does not work
environment:
- DB_SID:InfraDB
- DB_PDB:InfraPDB1
- DB_DOMAIN:us.oracle.com
- DB_BUNDLE:basic
3). Instead it is better to define those environment settings in a file that is referenced by the 'env_file' configuration option as shown below. This file resides on /tmp/docker-compose.yml
version: '2'
services:
database:
image: store/oracle/database-enterprise:12.2.0.1
ports:
- 1521:1521
- 8080:8080
- 5500:5500
env_file:
- e2.txt
And the env file contents live in the same directory ( /tmp/e2.txt ), and has the content below:
DB_SID=InfraDB
DB_PDB=InfraPDB1
DB_DOMAIN=us.oracle.com
DB_BUNDLE=basic
To run i do
docker-compose -f /tmp/docker-compose.yml up
and then i can connect (from my laptop as was the original goal) via:
sqlplus sys/Oradoc_db1#0.0.0.0:1521/InfraDB.us.oracle.com as SYSDBA
This depends on a few things, like network type, and if the port is forwarded properly. Then verify this on the host (in linux 'netstat -tlpn' ), this will show you if it is listening on the port, and if so what interface and protocol. The tool may differ depending on the host OS.
Once you see it listening via tcp (not tcp6 only), and the firewall is open on the host, you can combine that info to make your connection string.
The issue is I get the curl: (7) Failed to connect to localhost port 8090: Connection refused GItLab CI error but this does not happen on my laptop where I get the source html of the webpage. The .gitlab-ci.yml below is a simple reproduction of the issue. I have spent numerous hours trying to figure this out - i'm sure someone else has also.
Aside: This isn't a similar question - since they don't offer a solution.
GitLab Repo: https://gitlab.com/mudassir-ahmed/wordpress-testing-with-gitlab-ci/tree/another-approach but the only file it contains is the .gitlab-ci.yml shown below...
image: docker:stable
variables:
# When using dind service we need to instruct docker, to talk with the
# daemon started inside of the service. The daemon is available with
# a network connection instead of the default /var/run/docker.sock socket.
#
# The 'docker' hostname is the alias of the service container as described at
# https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#accessing-the-services
#
# Note that if you're using the Kubernetes executor, the variable should be set to
# tcp://localhost:2375/ because of how the Kubernetes executor connects services
# to the job container
# DOCKER_HOST: tcp://localhost:2375/
#
# For non-Kubernetes executors, we use tcp://docker:2375/
DOCKER_HOST: tcp://docker:2375/
# When using dind, it's wise to use the overlayfs driver for
# improved performance.
DOCKER_DRIVER: overlay2
services:
- docker:dind
before_script:
- docker info
build:
stage: build
script:
- apk update
- apk add curl
#- hostname -i
- docker container ls
- docker run -d -p 8090:80 --name nginx-server kitematic/hello-world-nginx
- curl localhost:8090 # This works on my laptop but not on a GitLab runner.
Referring to the answer from here : gitlab-ci.yml & docker-in-docker (dind) & curl returns connection refused on shared runner
There are two ways to fix this :
Option 1: Replace localhost in curl localhost:8090 with docker like this curl docker:8090
Option 2:
services:
- name: docker:dind
alias: localhost
docker run -d -p 8090:80 --name nginx-server kitematic/hello-world-nginx
curl localhost:8090 # This works on my laptop but not on a GitLab runner.
Assuming that is your code i think that you should somehow add some timeout between docker run and curl.
I have similar issues some time ago after starting docker container on gitlab runner machine i wasnt able to accces my url to. When i added command which check if container is running for " about one minute " it resolved my problem.
"docker inspect -f {{.State.Running}} " + containerName" but in order to do that check, you should add some additional script
For a while I'm searching for a good way to run Oracle in Docker. It was always ending up in huge images, and slow starting containers.
But today I saw the link: https://store.docker.com/images/oracle-database-enterprise-edition
Is this just an official Oracle12c we can use (I was able to pull and start it) or is this only for enterprise users?
Can I customize this image with my own .sql scripts? (With some entrypoint)?
You can follow the following steps:
docker login
docker pull store/oracle/database-enterprise:12.2.0.1
docker run -d -it --name oracledb12c -P store/oracle/database-enterprise:12.2.0.1
docker port containerID 1521/tcp (you will can connect to the datbase outside of the container using this port)
$ docker ps #check while the container status is not healthy)
Connect with sqlplus
docker exec -it containerID bash -c "source /home/oracle/.bashrc; sqlplus /nolog"
Connect with jdbc:
USER: sys PASSWD: Oradoc_db1
jdbc:oracle:thin:#(description=(address=(host=127.0.0.1)(protocol=tcp)(port=portNumner))
(connect_data=(service_name=ORCLPDB1.localdomain)))
I'm trying a simple setup with a docker postgres image, publishing a port for me to connect from the localhost.
docker run -p 5432:5432 --name postgres -e POSTGRES_PASSWORD=password -e POSTGRES_USER=postgres -d postgres
I'm able to connect just fine if I specify the docker external IP:
psql -h 192.1469.99.100 -p 5432 -U postgres -d postgres --password
However I get a password authentication failure when trying against localhost:
psql -h localhost -p 5432 -U postgres -d postgres --password
psql: FATAL: password authentication failed for user "postgres"
Do I need to set up some manual port forwarding manually? The weird thing is that it seems to connect to the postgres server just fine, it's just bizarrely telling me the password fails. I've done something wrong with the docker config perhaps?
The pg_hba.conf looks like:
local all all trust
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
The host machine is Windows 7.
Did you try another port than 5432? Try some free port so that there is no conflict between your local postgre and your docker postgre. To me it looks like your local postgre is having higher prio on localhost:5432, takes the traffic and so you are trying to login to your local postgre instead of your docker postgre, but obv. just a guess.
Did you check if you have this port allowed in the virtual machine or whatever you use for virtualization?
The container has its own network and its loopback address does not refer to the host and vice versa.
The latest release of Docker doesn't use a virtual machine anymore, instead using a hypervisor to connect to the containers. This means I can no longer login to postgres with psql:
➜ postgres git:(master) ✗ docker run -d -p 5433:5432 db postgres
<sha>
➜ postgres git:(master) ✗ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
111f3bed4c52 db "/docker-entrypoint.s" 17 minutes ago Up 17 minutes 0.0.0.0:5433->5432/tcp zen_hugle
➜ postgres git:(master) ✗ psql -p 5433 -U postgres
psql: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5433"?
I have also tried specifying localhost as the host, but that results in a strange output:
➜ postgres git:(master) ✗ psql -h localhost -p 5433 -U postgres
psql: %
Does anyone know what to do in this case? Thank you.
With follwing command,
docker run -d -p 5433:5432 db postgres
You are exporting your docker's 5432 port to docker-engine's 5433 port. Not your host machine's 5433.
Fetch your docker-machine's IP address with following command (assuming your docker vm name is default)
docker-machine env default
This should give you result similar to following lines
> export DOCKER_TLS_VERIFY="1"
> export DOCKER_HOST="tcp://192.168.99.100:2376"
> export DOCKER_CERT_PATH="/Users/<your-user>/.docker/machine/machines/default"
> export DOCKER_MACHINE_NAME="default"
Use your docker-machine's IP address to connect to Postgres running in container
>psql -h 192.168.99.100 -p 5433 -U postgres
psql (9.5.0, server 9.5.5)
Type "help" for help.
postgres=#
You can connect over tcp by using an IP like psql -h 0.0.0.0 -p 5433 -U postgres (or 127.0.0.1, etc.).
Using the default or "localhost" will try using the local domain socket (although the version of Docker doesn't change this behavior, you will generally need to connect to a containerized db via tcp).