I have built a pipeline with four steps: build, test, lint and deploy. However, I have to run npm install in three individual steps which I think could be done in a cleaner way. Could someone point me to how I could maybe do npm install globally instead?
This is the config.yml file:
version: 2.1
orbs:
node: circleci/node#4.1.0
heroku: circleci/heroku#0.0.10
eslint: arrai/eslint#2.0.0
jobs:
build:
executor:
name: node/default
steps:
- checkout
- run: npm install
test:
executor:
name: node/default
steps:
- checkout
- run: npm install
- run: npm run test
lint:
executor:
name: node/default
steps:
- checkout
- run: npm install
- run: npm run lint
deploy:
executor:
name: heroku/default
steps:
- checkout
- heroku/deploy-via-git
workflows:
main:
jobs:
- build
- test:
requires:
- build
- lint:
requires:
- test
- deploy:
requires:
- lint
You should save installed packages in cache then restore them each time.
Ref:
https://circleci.com/blog/config-best-practices-dependency-caching/
Related
If I push to master, it works perfectly, all environment variables available and I have a successful deploy to Heroku.
Problem: If I push to the dev branch, it can't see the environment variables for the deploy.
$ dpl --provider=heroku --app=$HEROKU_DEV_APP --api-key=$HEROKU_API_KEY
invalid option "--api-key="
ERROR: Job failed: exit code 1
Environment settings:
.gitlab-ci.yml:
stages:
- build
- test
- deploy
build:
stage: build
image: maven:3.6.3-jdk-14
script:
- mvn clean package
tags:
- docker
test:
stage: test
image: maven:3.6.3-jdk-14
script:
- mvn test
tags:
- docker
deploy_dev:
stage: deploy
image: ruby:2.3
script:
- apt-get update -qy
- apt-get install -y ruby-dev
- gem install dpl
- dpl --provider=heroku --app=$HEROKU_DEV_APP --api-key=$HEROKU_API_KEY
environment:
name: prod
url: https://.....herokuapp.com/
only:
- dev
tags:
- docker
deploy_prod:
stage: deploy
image: ruby:2.3
script:
- apt-get update -qy
- apt-get install -y ruby-dev
- gem install dpl
- dpl --provider=heroku --app=$HEROKU_PROD_APP --api-key=$HEROKU_API_KEY
environment:
name: prod
url: https://.....herokuapp.com/
when: manual
only:
- master
tags:
- docker
This is because your Heroku api key variable is set as protected.
Protected variables are visible only by protected branches and protected tags. That is why it works for you on master but not on dev.
More information: https://gitlab.com/help/ci/variables/README#protect-a-custom-variable and https://gitlab.com/help/user/project/protected_branches.md
Your options are: either remove protected flag, or introduce another unprotected variable with another api key for your non-protected branches which would be less sensitive.
I’m a little confused about how Postman (Newman) tests would execute against a build unless that build is running somewhere. Wouldn’t I need to deploy it somewhere and THEN execute Travis CI?
I connected Github to Travis & Heroku, I think I need to link them in the .travis.yml file.
.travis.yml
language: node_js
node_js:
- "12.14.1"
install:
- npm install newman
- npm install jest
before_script:
- node --version
- npm --version
- yarn --version
- node_modules/.bin/newman --version
- node_modules/.bin/jest --version
deploy:
provider: heroku
api_key:
secure: <HEROKU_API_KEY>
app: <HEROKU_APP_NAME>
on:
repo: <GITHUB_REPOSITORY>
script:
- node_modules/.bin/newman run <COLLECTION_LINK> --environment <ENV_LINK>
- yarn test
What should I specify to run tests after the build & deploy? Am I missing a step?
What you are looking for is build stages, see docs https://docs.travis-ci.com/user/build-stages/.
The syntax is pretty straightforward.
jobs:
include:
- stage: install
script: npm run install
- stage: build
script: npm run build
- stage: deploy
deploy:
provider: heroku
api_key:
secure: <HEROKU_API_KEY>
app: <HEROKU_APP_NAME>
on:
repo: <GITHUB_REPOSITORY>
- stage: test
script: npm run tests
We are having CircleCI CI/CD pipeline for our project. I would need to setup cypress test to run on CircleCi pipeline. Could anyone please advise about the npm install command to create circle.yml or .circleci/config.yml file under the root folder:
Here's an example of my circle.yml, which is located in my project root:
version: 2.1
jobs:
test:
docker:
- image: cypress/base:10
steps:
- checkout
- restore_cache:
keys:
- cache-{{ arch }}-{{ .Branch }}-{{ checksum "package.json" }}
- run:
name: Yarn install
command: yarn install --frozen-lockfile
- save_cache:
key: cache-{{ arch }}-{{ .Branch }}-{{ checksum "package.json" }}
paths:
- ~/.cache
- run:
command: yarn lint
- run:
command: yarn test
- run:
command: yarn test-e2e
- run:
command: yarn run semantic-release
workflows:
build:
jobs:
- test
version: 2
You can replace yarn test-e2e with your cypress cli command, for example npm run cypress:run
I have created a CircleCI config which will run my PHPUnit tests against my laravel application and that is working 100% however I am now trying to add a workflow to then SSH and deploy my app to an AWS EC2 server and I am getting the following errors:
Your config file has errors and may not run correctly:
2 schema violations found
required key [jobs] not found
required key [version] not found
However I cannot see an issue with my CircleCI config file, have I made a mistake somewhere?
version: 2
jobs:
build:
docker:
- image: circleci/php:7.1-browsers
working_directory: ~/laravel
steps:
- checkout
- run:
name: Download NodeJS v6
command: curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
- run:
name: Install SQLite and NodeJS 6
command: sudo apt-get install -y libsqlite3-dev nodejs
- run:
name: Setup Laravel testing environment variables for CircleCI test
command: cp .env.circleci .env
- run:
name: Update composer to latest version
command: composer self-update
- restore_cache:
keys:
- composer-v1-{{ checksum "composer.json" }}
- composer-v1-
- run: composer install -n --prefer-dist --ignore-platform-reqs
- save_cache:
key: composer-v1-{{ checksum "composer.json" }}
paths:
- vendor
- restore_cache:
key: dependency-cache-{{ checksum "package.json" }}
- run:
name: Install NodeJS Packages
command: npm install
- save_cache:
key: dependency-cache-{{ checksum "package.json" }}
paths:
- ./node_modules
- run:
name: Create SQLite Database
command: touch database/database.sqlite
- run:
name: Migrate Laravel Database
command: php artisan migrate --database=sqlite --force
- run:
name: Run NPM
command: npm run production
# Run Laravel Server for front-end tests
- run:
name: Run Laravel Server
command: php artisan serve
background: true
- run:
name: Run PHPUnit Tests
command: vendor/bin/phpunit
deploy:
machine:
enabled: true
steps:
- run:
name: Deploy Over SSH
command: |
ssh $SSH_USER#$SSH_HOST "cd /var/www/html"
workflows:
version: 2
build-and-deploy:
jobs:
- build
- deploy:
requires:
- build
filters:
branches:
only: master
Any help is appreciated, thank you!
CircleCI has documentation for AWS deployment. Look here https://circleci.com/docs/1.0/continuous-deployment-with-aws-codedeploy/
I think your problem is with SSH authorization for AWS. You can try it locally and make sure that your authorization is successfully, and then do the same thing with your AWS.
I'm getting this error in CI when updating to Cypress 3.0, saying that Cypress is not installed, but I am running npm install before my cypress run command. The error:
No version of Cypress is installed in:
/home/ubuntu/.cache/Cypress/3.0.1/Cypress
Please reinstall Cypress by running: cypress install
----------
Why is Cypress not finding the Cypress executable?
Here's my circle.yml:
build:
<<: *defaults
steps:
- checkout
- restore_cache:
keys:
- v1-npm-deps
- run: npm install
- save_cache:
key: v1-npm-deps-{{ checksum "package.json" }}
paths:
- node_modules
- ~/.cache
- ~/.npm
- run: npm test
- run: npm run build
- persist_to_workspace:
root: /tmp/workspace
paths:
- .circleci/deploy.sh
- .circleci/e2e-test.sh
- package.json
- cypress.json
- node_modules/
- build/*
- cypress/*
This a small problem with caching node_modules- the post-install script that installs the Cypress binary won't be run since node_modules/cypress exists.
To fix this you can flush the cache of the CI build and everything should be solved.
This is why I recommend using npm ci, since node_modules will get wiped every time the command is run
Also:
- in Circle CI 2.0, caching works differently than in 1.0 or TravisCI because the cache is immutable. You can only create another cache, never destroy and rewrite one. So, you should do caching like this:
- restore_cache:
keys:
- v1-deps-{ .Branch }-{ checksum "package.json" }
- v1-deps-{ .Branch }
- v1-deps
- run:
- npm ci
- save_cache:
key: v1-deps-{ .Branch }-{ checksum "package.json" }
paths:
- ~/.cache
- ~/.npm