CircleCI YAML config fails - laravel

I have created a CircleCI config which will run my PHPUnit tests against my laravel application and that is working 100% however I am now trying to add a workflow to then SSH and deploy my app to an AWS EC2 server and I am getting the following errors:
Your config file has errors and may not run correctly:
2 schema violations found
required key [jobs] not found
required key [version] not found
However I cannot see an issue with my CircleCI config file, have I made a mistake somewhere?
version: 2
jobs:
build:
docker:
- image: circleci/php:7.1-browsers
working_directory: ~/laravel
steps:
- checkout
- run:
name: Download NodeJS v6
command: curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
- run:
name: Install SQLite and NodeJS 6
command: sudo apt-get install -y libsqlite3-dev nodejs
- run:
name: Setup Laravel testing environment variables for CircleCI test
command: cp .env.circleci .env
- run:
name: Update composer to latest version
command: composer self-update
- restore_cache:
keys:
- composer-v1-{{ checksum "composer.json" }}
- composer-v1-
- run: composer install -n --prefer-dist --ignore-platform-reqs
- save_cache:
key: composer-v1-{{ checksum "composer.json" }}
paths:
- vendor
- restore_cache:
key: dependency-cache-{{ checksum "package.json" }}
- run:
name: Install NodeJS Packages
command: npm install
- save_cache:
key: dependency-cache-{{ checksum "package.json" }}
paths:
- ./node_modules
- run:
name: Create SQLite Database
command: touch database/database.sqlite
- run:
name: Migrate Laravel Database
command: php artisan migrate --database=sqlite --force
- run:
name: Run NPM
command: npm run production
# Run Laravel Server for front-end tests
- run:
name: Run Laravel Server
command: php artisan serve
background: true
- run:
name: Run PHPUnit Tests
command: vendor/bin/phpunit
deploy:
machine:
enabled: true
steps:
- run:
name: Deploy Over SSH
command: |
ssh $SSH_USER#$SSH_HOST "cd /var/www/html"
workflows:
version: 2
build-and-deploy:
jobs:
- build
- deploy:
requires:
- build
filters:
branches:
only: master
Any help is appreciated, thank you!

CircleCI has documentation for AWS deployment. Look here https://circleci.com/docs/1.0/continuous-deployment-with-aws-codedeploy/
I think your problem is with SSH authorization for AWS. You can try it locally and make sure that your authorization is successfully, and then do the same thing with your AWS.

Related

Connection refused for local server in github actions workflow

I'm trying to run a local server for my project CI/CD Pipeline. When I start the server I got a "Connection refused" on it.
My project is a fastAPI application and I'm trying to run a integration tests on PR to validate the app before merge the code. I tried to start my app directly (gunicorn), building a docker image and starting it... I tried a lot of things. Then, I tried to run a simple server instead of my app and... got the same error!
This is my simple server workflow:
on:
push:
branches:
- "develop"
jobs:
mylocalserver:
name: Run server
runs-on: ubuntu-latest
steps:
- name: Setup python
uses: actions/setup-python#v3
with:
python-version: 3.9
- name: Run server in background
run: |
python -V
python -m http.server 3000 &> /dev/null &
sudo lsof -i -P -n | grep LISTEN
curl http://localhost:3000
- name: Run server with console
run: |
python -m http.server 3000
Output:
If I run my app with console (no daemon mode in gunicorn), the server start and log to console in workflow with success:
But this way I cannot run nothing after this (and I have to cancel workflow). Some idea? Thank you!
Maybe not the best answer, but for now runnig the job into a container works (only add a container label in question example). Example for my fastAPI app:
on:
pull_request:
branches:
- 'main'
- 'develop'
jobs:
run-on-pr:
runs-on: ubuntu-latest
container: ubuntu
services:
mongodb:
image: mongo
ports:
- 27017:27017
steps:
- name: Setup git
run: |
apt-get update; apt-get install -y git
- name: Git checkout
uses: actions/checkout#v3
with:
path: api
- name: Setup python
uses: actions/setup-python#v4
with:
python-version: 3.9
- name: Install pip
run: |
apt-get update; apt-get install -y python3-pip
- name: Build and ENV
run: |
cd api
cp .env_example_docker .env
pip3 install -r requirements.txt
- name: Run fastAPI
run: |
cd api
gunicorn -D -k uvicorn.workers.UvicornWorker -c ./gunicorn_conf.py app.main:app
env:
MONGO_URL: "mongodb://mongodb:27017"
- name: Install curl
run: |
apt-get update; apt-get install -y curl
- name: Run curl
run: |
curl http://localhost:3000
This works, but I have to install all in container (git, pip). I will try a solution without using the container label and if I found anything I can post here.

Gitlab CI/CD env var availabel only on master

If I push to master, it works perfectly, all environment variables available and I have a successful deploy to Heroku.
Problem: If I push to the dev branch, it can't see the environment variables for the deploy.
$ dpl --provider=heroku --app=$HEROKU_DEV_APP --api-key=$HEROKU_API_KEY
invalid option "--api-key="
ERROR: Job failed: exit code 1
Environment settings:
.gitlab-ci.yml:
stages:
- build
- test
- deploy
build:
stage: build
image: maven:3.6.3-jdk-14
script:
- mvn clean package
tags:
- docker
test:
stage: test
image: maven:3.6.3-jdk-14
script:
- mvn test
tags:
- docker
deploy_dev:
stage: deploy
image: ruby:2.3
script:
- apt-get update -qy
- apt-get install -y ruby-dev
- gem install dpl
- dpl --provider=heroku --app=$HEROKU_DEV_APP --api-key=$HEROKU_API_KEY
environment:
name: prod
url: https://.....herokuapp.com/
only:
- dev
tags:
- docker
deploy_prod:
stage: deploy
image: ruby:2.3
script:
- apt-get update -qy
- apt-get install -y ruby-dev
- gem install dpl
- dpl --provider=heroku --app=$HEROKU_PROD_APP --api-key=$HEROKU_API_KEY
environment:
name: prod
url: https://.....herokuapp.com/
when: manual
only:
- master
tags:
- docker
This is because your Heroku api key variable is set as protected.
Protected variables are visible only by protected branches and protected tags. That is why it works for you on master but not on dev.
More information: https://gitlab.com/help/ci/variables/README#protect-a-custom-variable and https://gitlab.com/help/user/project/protected_branches.md
Your options are: either remove protected flag, or introduce another unprotected variable with another api key for your non-protected branches which would be less sensitive.

Can not trigger build with indication of "Not Run" in CircleCI

I am planning to migrate my existing CI to CircleCI 2 in favor of the docker. Once I tried to trigger a build it's shown something like this:
I also checked the steps as indicated in the below alert here https://circleci.com/sunset1-0/
Service alert: Your project references CircleCI 1.0 or it has no
configuration. CircleCI 1.0 and projects without configuration files
are no longer supported. You must update your project to use CircleCI
2.0 configuration to continue. Learn more.
Is there anything I missed out?
Below is my .circleci/config.yml
version: 2 # use CircleCI 2.0
jobs: # a collection of steps
build: # runs not using Workflows must have a `build` job as entry point
parallelism: 3 # run three instances of this job in parallel
docker: # run the steps with Docker
- image: circleci/ruby:2.4.2-jessie-node # ...with this image as the primary container; this is where all `steps` will run
environment: # environment variables for primary container
BUNDLE_JOBS: 3
BUNDLE_RETRY: 3
BUNDLE_PATH: vendor/bundle
RAILS_ENV: test
- image: mysql:5.7 # database image
environment: # environment variables for database
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
steps: # a collection of executable commands
- checkout # special step to check out source code to working directory
- run:
name: setup
command: |
curl -sL https://deb.nodesource.com/setup_10.x | bash \
&& curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
&& echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
- run:
name: Dependencies
command: |
apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y \
build-essential mysql-client nodejs yarn && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Which version of bundler?
- run:
name: Which bundler?
command: bundle -v
# Restore bundle cache
# Read about caching dependencies: https://circleci.com/docs/2.0/caching/
- restore_cache:
keys:
- rails-demo-bundle-v2-{{ checksum "Gemfile.lock" }}
- rails-demo-bundle-v2-
- run: # Install Ruby dependencies
name: Bundle Install
command: bundle check --path vendor/bundle || bundle install --deployment
# Store bundle cache for Ruby dependencies
- save_cache:
key: rails-demo-bundle-v2-{{ checksum "Gemfile.lock" }}
paths:
- vendor/bundle
# Only necessary if app uses webpacker or yarn in some other way
- restore_cache:
keys:
- rails-demo-yarn-{{ checksum "yarn.lock" }}
- rails-demo-yarn-
- run:
name: Yarn Install
command: yarn install --cache-folder ~/.cache/yarn
# Store yarn / webpacker cache
- save_cache:
key: rails-demo-yarn-{{ checksum "yarn.lock" }}
paths:
- ~/.cache/yarn
- run:
name: Wait for DB
command: dockerize -wait tcp://localhost:5432 -timeout 1m
- run:
name: Database setup
command: bin/rails db:schema:load --trace
- run:
name: Run rspec in parallel
command: |
bundle exec rspec --profile 10 \
--format RspecJunitFormatter \
--out test_results/rspec.xml \
--format progress \
$(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)
# Save test results for timing analysis
- store_test_results: # Upload test results for display in Test Summary: https://circleci.com/docs/2.0/collect-test-data/
path: test_results
# See https://circleci.com/docs/2.0/deployment-integrations/ for example deploy configs
In order for the build work, your config.yml file must live inside a .circleci folder at the root of your project directory.
You post a very good description. Thanks for the code. The version config looks fine. However, you wrote that your config currently lives at .circle/config.yml (Notice the missing ci at end of circle)
Move the config.yml to .circleci/config.yml and the build should proceed further. (right now the build is failing because it has no configuration.
E

What is the npm command to install .circleci/config.yml for cypress test

We are having CircleCI CI/CD pipeline for our project. I would need to setup cypress test to run on CircleCi pipeline. Could anyone please advise about the npm install command to create circle.yml or .circleci/config.yml file under the root folder:
Here's an example of my circle.yml, which is located in my project root:
version: 2.1
jobs:
test:
docker:
- image: cypress/base:10
steps:
- checkout
- restore_cache:
keys:
- cache-{{ arch }}-{{ .Branch }}-{{ checksum "package.json" }}
- run:
name: Yarn install
command: yarn install --frozen-lockfile
- save_cache:
key: cache-{{ arch }}-{{ .Branch }}-{{ checksum "package.json" }}
paths:
- ~/.cache
- run:
command: yarn lint
- run:
command: yarn test
- run:
command: yarn test-e2e
- run:
command: yarn run semantic-release
workflows:
build:
jobs:
- test
version: 2
You can replace yarn test-e2e with your cypress cli command, for example npm run cypress:run

No version of Cypress is installed in Ci (TravisCI and CircleCI)

I'm getting this error in CI when updating to Cypress 3.0, saying that Cypress is not installed, but I am running npm install before my cypress run command. The error:
No version of Cypress is installed in:
/home/ubuntu/.cache/Cypress/3.0.1/Cypress
Please reinstall Cypress by running: cypress install
----------
Why is Cypress not finding the Cypress executable?
Here's my circle.yml:
build:
<<: *defaults
steps:
- checkout
- restore_cache:
keys:
- v1-npm-deps
- run: npm install
- save_cache:
key: v1-npm-deps-{{ checksum "package.json" }}
paths:
- node_modules
- ~/.cache
- ~/.npm
- run: npm test
- run: npm run build
- persist_to_workspace:
root: /tmp/workspace
paths:
- .circleci/deploy.sh
- .circleci/e2e-test.sh
- package.json
- cypress.json
- node_modules/
- build/*
- cypress/*
This a small problem with caching node_modules- the post-install script that installs the Cypress binary won't be run since node_modules/cypress exists.
To fix this you can flush the cache of the CI build and everything should be solved.
This is why I recommend using npm ci, since node_modules will get wiped every time the command is run
Also:
- in Circle CI 2.0, caching works differently than in 1.0 or TravisCI because the cache is immutable. You can only create another cache, never destroy and rewrite one. So, you should do caching like this:
- restore_cache:
keys:
- v1-deps-{ .Branch }-{ checksum "package.json" }
- v1-deps-{ .Branch }
- v1-deps
- run:
- npm ci
- save_cache:
key: v1-deps-{ .Branch }-{ checksum "package.json" }
paths:
- ~/.cache
- ~/.npm

Resources