Remote postgres connection on circleci build to run laravel phpunit tests - laravel

We are using laravel 5.6, postgresql and circleci in our api production environment and still trying to implement some key unit tests to run before a commit is merged to master.
When trying to configure the remote postgresql database access on circle, there's the following problem:
Our .circleci/config.yml was supposed to pull a custom built image (edunicastro/docker:latest) and run phpunit tests in the "build" step
But we are getting the following error message:
PDOException: SQLSTATE[08006] [7] could not connect to server: Connection refused
Is the server running on host "127.0.0.1" and accepting
TCP/IP connections on port 5432?
The problem is this was supposed to connect to our remote database, but in our production environment we have the connection set up using .env and laravel.
I have tried copying the "DB_PGSQL_HOST" key to our config.yml but nothing changed, it kept trying to connect to 127.0.0.1.
Using the key "PGHOST" instead also had no effect.
This is the relevant, "build" part of our config.yml:
version: 2
jobs:
build:
docker:
- image: edunicastro/docker:latest
environment:
DB_PGSQL_CONNECTION: <prod_laravel_connection_name>
DB_PGSQL_HOST: <prod_db_host>
DB_PGSQL_PORT: 5432
DB_PGSQL_DATABASE: <prod_db_name>
working_directory: ~/repo
steps:
- checkout
- restore_cache:
keys:
- v1-dependencies-{{ checksum "composer.json" }}
- v1-dependencies-
- run: composer install -n --prefer-dist
- run: ./vendor/bin/phpunit
- save_cache:
paths:
- ./vendor
key: v1-dependencies-{{ checksum "composer.json" }}

Okay, I was missing the command to copy the .env over, right under - checkout:
- checkout
- run: cp .env.test .env
Laravel was already configured and set to use it, so I didn't need to change anything else.

Related

sh: 1: nest: Permission denied in GitHub Action

For some reason the build step for my NestJS project in my GitHub Action fails for a few days now. I use Turborepo with pnpm in a monorepo and try to run the build with turbo run build. This works flawlessly on my local machine, but somehow in GitHub it fails with sh: 1: nest: Permission denied. ELIFECYCLE  Command failed with exit code 126. I'm not sure how this is possible, since I couldn't find any meaningful change I made to the code in the meantime. It just stopped working unexpectedly. I actually think it is an issue with GH Actions, since it actually works in my local Docker build as well.
Has anyone else encountered this issue with NestJS in GH Actions?
This is my action yml:
name: Test, lint and build
on:
push:
jobs:
test-lint-build:
runs-on: ubuntu-latest
services:
postgres:
# Docker Hub image
image: postgres
# Provide the password for postgres
env:
POSTGRES_HOST: localhost
POSTGRES_USER: test
POSTGRES_PASSWORD: docker
POSTGRES_DB: financing-database
ports:
# Maps tcp port 5432 on service container to the host
- 2345:5432
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- name: Checkout
uses: actions/checkout#v3
- name: Install pnpm
uses: pnpm/action-setup#v2.2.2
with:
version: latest
- name: Install
run: pnpm i
- name: Lint
run: pnpm run lint
- name: Test
run: pnpm run test
- name: Build
run: pnpm run build
env:
VITE_SERVER_ENDPOINT: http://localhost:8000/api
- name: Test financing-server (e2e)
run: pnpm --filter #project/financing-server run test:e2e
I found out what was causing the problem. I was using node-linker = hoisted to mitigate some issues the pnpm way of linking modules was causing with my jest tests. Removing this from my project suddenly made the action work again.
I still don't know why this only broke the build recently, since I've had this option activated for some time now.

Getting a SQLSTATE[HY000] [2002] Connection refused error in github action?

This is what my actions look like, when I get to the point of php artisan test I get the error:
name: Continuous Integration
on:
push
jobs:
laravel-tests:
runs-on: ubuntu-20.04
- name: create migration
env:
DB_CONNECTION: sqlite
DB_DATABASE: database/database.sqlite
run: php artisan migrate --seed
- name: run tests
run: php artisan test
According to the documentation on steps:
Each step runs in its own process in the runner environment and has access to the workspace and filesystem. Because steps run in their own process, changes to environment variables are not preserved between steps.
So you'll need to provide the environment variables for the entire workflow, as described in the documentation for environment variables and the jobs.<job_id>.env keyword:
name: Continuous Integration
on:
push
jobs:
laravel-tests:
env:
DB_CONNECTION: sqlite
DB_DATABASE: database/database.sqlite
runs-on: ubuntu-20.04
steps:
...

docker-compose pull Error: "error creating temporary lease: read-only file system"

I'm trying to run docker-compose pull but I get some errors that I don't know what to do with.
My docker-compose.yaml file:
version: '3'
services:
strapi:
image: strapi/strapi
environment:
DATABASE_CLIENT: postgres
DATABASE_NAME: strapi
DATABASE_HOST: postgres
DATABASE_PORT: 5432
DATABASE_USERNAME: strapi
DATABASE_PASSWORD: strapi
volumes:
- ./app:/srv/app
ports:
- '1337:1337'
depends_on:
- postgres
postgres:
image: postgres
environment:
POSTGRES_DB: strapi
POSTGRES_USER: strapi
POSTGRES_PASSWORD: strapi
volumes:
- ./data:/var/lib/postgresql/data
The error message:
Pulling postgres ... error
Pulling strapi ... error
ERROR: for strapi error creating temporary lease: write /var/lib/desktop-containerd/daemon/io.containerd.metadata.v1.bolt/meta.db: read-only file system: unknown
ERROR: for postgres error creating temporary lease: write /var/lib/desktop-containerd/daemon/io.containerd.metadata.v1.bolt/meta.db: read-only file system: unknown
ERROR: error creating temporary lease: write /var/lib/desktop-containerd/daemon/io.containerd.metadata.v1.bolt/meta.db: read-only file system: unknown
error creating temporary lease: write /var/lib/desktop-containerd/daemon/io.containerd.metadata.v1.bolt/meta.db: read-only file system: unknown
I tried a multitude of things so YMMV, but here are all of the steps I did that ultimately got it working.
I am using Windows 10 with WSL2 backend on Ubuntu, so again YMMV as I see MacOS is tagged. This is one of the few questions I see related to mine, so I thought it would be valuable.
Steps for success:
Update WSL (wsl --update -- unrelated to the GitHub issue below)
stop Docker Desktop
stop WSL (wsl --shutdown)
unregister the docker-desktop distro (which contains binaries, but no data)
wsl --unregister docker-desktop
restart Docker Desktop (try running as admin)
Enable use of docker compose V2 (settings -> general -> Use Docker Compose V2)
Associated GitHub issue link
Extra Info:
I ended up using V2 of docker compose when it worked... it works either way now that the image has pulled properly, though.
I unsuccessfully restarted, reinstalled, and factory reset Docker Desktop many times.

Unable to connect to a target server via SSH from a GitLab pipeline?

I have set up .gitlab-ci.yml. I am unable to login to the production server from gitlab. I have set the private and public key variables of my server in GITLAB but still getting timeout error in pipeline.
job1:
stage: build1
script:
- mvn package
variables:
SSH_PUBLIC_key: "$SSH_PUBLIC_key"
SSH_PRIVATE_KEY: "$SSH_PRIVATE_KEY"
artifacts:
paths:
- server
script:
- scp "myjar" root#"myIP":/tmp
job1:
stage: build1
script:
- mvn package
variables:
SSH_PUBLIC_key: "$SSH_PUBLIC_key"
SSH_PRIVATE_KEY: "$SSH_PRIVATE_KEY"
artifacts:
paths:
- server
script:
- scp "myjar" root#"myIP":/tmp
timeout error comes, when the instance (in your case the production instance) is not reachable from GitLab (can be hosted on VM, Kubernetes, etc). Please check if you are able to perform telnet/ssh manually from the GitLab hosted VM
Replace myIP with proper values and see if that helps.
telnet <myIP> 22
ssh <myIP>

Unable to start MySQL service in docker during gitlab-ci

I have the following .gitlab-ci taken from the example of Laravel Dusk CI:
stages:
- build
- test
# Variables
variables:
MYSQL_ROOT_PASSWORD: root
MYSQL_USER: root
MYSQL_PASSWORD: secret
MYSQL_DATABASE: test
DB_HOST: mysql
DB_CONNECTION: mysql
build:
stage: build
services:
- mysql:5.7
image: chilio/laravel-dusk-ci:stable
script:
- composer install --prefer-dist --no-ansi --no-interaction --no-progress --no-scripts
# - npm install # if you need to install additional modules from your projects package.json
# - npm run dev # if you need to run dev scripts for example laravel mix
cache:
key: ${CI_COMMIT_REF_NAME}
paths:
# these are only examples, you should modify them according to your project,
# or remove cache routines entirely, if they are causing any problems on your next builds..
# below are 2 safe ones if you use composer install and npm install in your stage script
- vendor
- node_modules
# - /resources/assets/vendors # for example if you put your vendor node-libraries there
test:
stage: test
cache:
key: ${CI_COMMIT_REF_NAME}
paths:
- vendor
- node_modules
policy: pull
services:
- mysql:5.7
image: chilio/laravel-dusk-ci:stable
script:
- cp .env.example .env
# - cp phpunit.xml.ci phpunit.xml # if you are using custom config for your phpunit tests in CI
- configure-laravel
- start-nginx-ci-project
- ./vendor/phpunit/phpunit/phpunit -v --coverage-text --colors --stderr
# - phpunit -v --coverage-text --colors --stderr # if you want to use preinstalled phpunit
- php artisan dusk --colors --debug
artifacts:
paths:
- ./storage/logs # for debugging
- ./tests/Browser/screenshots
- ./tests/Browser/console
expire_in: 7 days
when: always
However, when the runner executes the job, I keep getting the following warning:
Using Docker executor with image chilio/laravel-dusk-ci:stable ...
Starting service mysql:5.7 ...
Pulling docker image mysql:5.7 ...
Using docker image sha256:66bc0f66b7af6ba3ea96582685d3afcd6dff93c2f8999da0ffadd67b280db548 for mysql:5.7 ...
Waiting for services to be up and running...
*** WARNING: Service runner-237f18d2-project-23-concurrent-0-mysql-0 probably didn't start properly.
Health check error:
ContainerStart: Error response from daemon: Cannot link to a non running container: /runner-237f18d2-project-23-concurrent-0-mysql-0 AS /runner-237f18d2-project-23-concurrent-0-mysql-0-wait-for-service/service
Service container logs:
2018-07-11T19:49:03.214991318Z
2018-07-11T19:49:03.215062485Z ERROR: mysqld failed while attempting to check config
2018-07-11T19:49:03.215067480Z command was: "mysqld --verbose --help"
2018-07-11T19:49:03.215070774Z
2018-07-11T19:49:03.215073778Z mysqld: error while loading shared libraries: libpthread.so.0: cannot stat shared object: Permission denied
I've tried to set the runner to privileged in the config.toml:
privileged = true
To solve the Question:
mysqld: error while loading shared libraries: libpthread.so.0: cannot stat shared object: Permission denied
Step1: update your software and kernel(maybe):
apt-get update && apt-get upgrade
Step2: install the docker dependency package:
(ubuntu/debian): apt-get install apt-transport-https ca-certificates curl gnupg2 software properties-common
(centos/redhat):yum-utils device-mapper-persistent-data lvm2
Step3: reboot your server & restart your docker-ce:
reboot
systemctl restart docker-ce

Resources