I'm trying to set up a workflow in CircleCI for my React project.
What I want to achieve is to get a job to build the stuff and another one to deploy the master branch to Firebase hosting.
This is what I have so far after several configurations:
witmy: &witmy
docker:
- image: circleci/node:7.10
version: 2
jobs:
build:
<<: *witmy
steps:
- checkout
- restore_cache:
keys:
- v1-dependencies-{{ checksum "package.json" }}
- v1-dependencies-
- run: yarn install
- save_cache:
paths:
- node_modules
key: v1-dependencies-{{ checksum "package.json" }}
- run:
name: Build app in production mode
command: |
yarn build
- persist_to_workspace:
root: .
deploy:
<<: *witmy
steps:
- attach_workspace:
at: .
- run:
name: Deploy Master to Firebase
command: ./node_modules/.bin/firebase deploy --token=MY_TOKEN
workflows:
version: 2
build-and-deploy:
jobs:
- build
- deploy:
requires:
- build
filters:
branches:
only: master
The build job always success, but with the deploy I have this error:
#!/bin/bash -eo pipefail
./node_modules/.bin/firebase deploy --token=MYTOKEN
/bin/bash: ./node_modules/.bin/firebase: No such file or directory
Exited with code 1
So, what I understand is that the deploy job is not running in the same place the build was, right?
I'm not sure how to fix that. I've read some examples they provide and tried several things, but it doesn't work. I've also read the documentation but I think it's not very clear how to configure everything... maybe I'm too dumb.
I hope you guys can help me out on this one.
Cheers!!
EDITED TO ADD MY CURRENT CONFIG USING WORKSPACES
I've added Workspaces... but still I'm not able to get it working, after a loooot of tries I'm getting this error:
Persisting to Workspace
The specified paths did not match any files in /home/circleci/project
And also it's a real pain to commit and push to CircleCI every single change to the config file when I want to test it... :/
Thanks!
disclaimer: I'm a CircleCI Developer Advocate
Each job is its own running Docker container (or VM). So the problem here is that nothing in node_modules exists in your deploy job. There's 2 ways to solve this:
Install Firebase and anything else you might need, on the fly, just like you do in the build job.
Utilize CircleCI Workspaces to carry over your node_modules directory from the build job to the deploy job.
In my opinion, option 2 is likely your best bet because it's more efficient.
Related
I'm trying to integrate cypress testing into gitlab pipeline.
I've tried about 10 different configurations which all fail.. I've included what I think are the relevant portions of of the gitlab.yml file, as well as the screenshot of the error on gitlab.
Thanks for any help
variables:
GIT_SUBMODULE_STRATEGY: recursive
cache:
paths:
- src/ui/node_modules/
- /root/.cache/Cypress/ //added this, also have tried src/ui/cypress/
build_ui:
image: node:16.14.2
stage: build
script:
- cd src/ui
- yarn install --pure-lockfile --prefer-offline --cache-folder .yarn
ui_test:
image: node:16.14.2
stage: test
needs: [build_ui]
script:
- cd src/ui
- yarn run runCypressHeadless
Each job gets its own separate environment. Therefore, you need to install your dependencies in each job. Add your yarn install command to the ui_test job.
The reason why your cache: did not restore to the job from the previous stage is because caches are per job by default (e.g. caches are restored from previous pipelines that ran the same job). If you want subsequent jobs in the same pipeline to use the cache, set the cache:key: to something like $CI_COMMIT_SHA or use cache:key:files: to use a file key, like your lockfile(s).
Also, you can only cache paths in the workspace. So you won't be able to cache/restore /root/.cache/... -- instead you should change the cache location to somewhere in the workspace.
For additional reference, see: caching in GitLab CI and caching NodeJS dependencies.
Hello I am using kedro (a pipeline tool) and want to use github actions to trigger a kedro command (kedro run) whenever I make a push to my github repo.
Since I have all the data in my local repo, I thought it would make sense to run the kedro command on my local machine.
So my question is, is there a way to trigger a local action using github actions? Using self-hosted runners perhaps?
You can run the kedro pipeline directly in gh provided runner using the steps below. I added a script I used previously in here to run kedro lint with every push.
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Set up Python 3.7.9
uses: actions/setup-python#v2
with:
python-version: 3.7.9
- uses: actions/cache#v2
with:
path: ${{ env.pythonLocation }}
key: ${{ env.pythonLocation }}-${{ hashFiles('src/requirements.txt') }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r src/requirements.txt
- name: Run Kedro Pipeline
run: |
kedro run
That said, I'm also wondering why you would need to run it on every push. Given compute provided by github actions is likely resource constrained running there might not be the best place. You would also need to keep your data within your repo for this approach to work.
Hi #magical_unicorn I think #avan-sh's answer is correct - what I would also add is that we encourage you to not commit data to VCS / Git. There are some technical limitations such as filesize, but more importantly it's not great security practice when working with teams.
Whilst not necessary on all projects - it might be good practice to explore using some sort of cloud storage for your data, decoupling it from the version control system.
I would like to use CircleCi to run my Cypress and react-testing-library tests because I want to test my react app.
On local env I would run (which work fine):
yarn run test to execute my react-testing-library tests
yarn cypress run to execute Cypress test
Now, I have found resources on how to make circleci config.yaml however nothing have worked. For reference link1, link2, link3, link4, link5
Some of the tests failed due to: error cypress#7.1.0: The engine "node" is incompatible with this module. Expected version ">=12.0.0". Got "10.24.1" or wrong cashing or something else. After 20 runs I am clueless, can someone help me, please?
As I was browsing resources I thought this should work for Cypress tests but it did not.
version: 2.1
orbs:
cypress: cypress-io/cypress#1
workflows:
build:
jobs:
- cypress/install:
build: yarn run build # run a custom app build step
yarn: true
- cypress/run:
requires:
- cypress/install
parallel: true # split all specs across machines
parallelism: 4 # use 4 CircleCI machines to finish quickly
yarn: true
group: 'all tests' # name this group "all tests" on the dashboard
start: yarn start # start server before running tests
For those who will search this issue later. I overcome errors:
error cypress#7.1.0: The engine "node" is incompatible with this module. Expected version ">=12.0.0". Got "10.24.1" by not using orb and instead use workflow -> jobs -> steps
fsevents not accessible from jest-haste-map by using yarn instead of npm
Lastly, some of your errors may come from your app (at least in my case react app) taking configuration from .env file that is not uploaded to github and therefore is not checkout to CircleCI docker and therefore during the test of the app will not work.
The working solution that I am using is:
version: 2.1
jobs:
run_tests:
docker:
- image: cypress/base:12
environment:
# this enables colors in the output
TERM: xterm
working_directory: ~/portalo
steps:
- checkout
- run:
name: Install project dependencies
command: yarn install --frozen-lockfile
- run:
name: Compile and start development server on port 3000
command: yarn startOnPort3000Linux
background: true
- run:
name: Wait for development server to start
command: 'yarn wait-on http://localhost:3000'
- run:
name: Run routing tests with react-testing-library via yarn test
command: 'yarn test ~/portalo/src/tests/react-testing-library/routing.test.tsx'
- run:
name: Run e2e tests with Cypruss via cypress run
command: $(yarn bin)/cypress run
workflows:
version: 2.1
build_and_test:
jobs:
- run_tests
Note: wait-on had to be added. In my case by yarn add wait-on
Note2: All steps have to be in a single to have present all installed packages. It could be tweet by using save/restore cache.
My CI/CD is: git > cicleci > heroku
I'm using circleci's suggested configuration (attached below)
I have the proper environment variable in circleci.
The first part is working well, however, when I deploy to heroku I can see warnings from devs dependencies like:
./src/hooks/myHook.js
Line 11: React Hook useEffect has a missing dependency: 'x'. Either include it or remove the dependency array react-hooks/exhaustive-deps
which makes me think that React dev is running.
Can anyone help me?
version: 2
jobs:
build:
...
deploy:
docker:
- image: buildpack-deps:trusty
steps:
- checkout
- run:
name: Deploy Master to Heroku
command: |
git push https://heroku:$HEROKU_API_KEY#git.heroku.com/$HEROKU_APP_NAME.git master
workflows:
version: 2
build-deploy:
jobs:
- build
- deploy:
requires:
- build
filters:
branches:
only: master
I'm not sure if you are already but I highly recommend using this Heroku buildpack rather than the standard Heroku Node buildpack: https://github.com/mars/create-react-app-buildpack. This buildpack starts in production by default.
In Heroku go to settings > Buildpacks and click on the Add buildpack, then enter the following url: https://github.com/mars/create-react-app-buildpack.git and save.
However if there is a hook error as above I would fix that issue by adding it into the dependency array in the hook.
I have ~5-10 builds in my .yml file for Gitlab CI. To save time, I'm wondering if there is a way to NOT re-clone the repo between every job. Ideally, the repo will be cloned once and then all 3 jobs run. I also don't want to combine the jobs into a single build because I'd like to see the results of each individually (when they are combined, gitlab's "pass/fail" is just the result of the last job).
I don't want to simply do git fetch because I want a fresh clone at the start.
stages:
- run
job1:
stage: run
script:
- pwd
- make all TEST=job1
job2:
stage: run
script:
- pwd
- make all TEST=job2
job3:
stage: run
script:
- pwd
- make all TEST=job3
...
I'm also fiddling around with this topic.
Actually, I go by doing a checkout-stage at first (with GIT_STRATEGY: clone) and then the build-stage with multiple jobs and GIT_STRATEGY: fetch.
This ensures that the repo is really full cloned at first and only fetched for every buildstep. Maybe this helps you too.
stages:
- checkout
- build
checkout:
variables:
GIT_STRATEGY: clone
GIT_SUBMODULE_STRATEGY: recursive
stage: checkout
script: '#echo Checking out...'
build:commander:
stage: build
variables:
GIT_STRATEGY: fetch
script:
- _Publish.bat commander
artifacts:
paths:
- BuildArtifacts\Commander\**
build:login:
stage: build
variables:
GIT_STRATEGY: fetch
script:
- _Publish.bat login
artifacts:
paths:
- BuildArtifacts\Login\**
build:cli:
stage: build
variables:
GIT_STRATEGY: fetch
script:
- _Publish.bat cli
artifacts:
paths:
- BuildArtifacts\Cli\**
This might be helpful, assuming you are using a new enough version of gitlab and the runner: https://docs.gitlab.com/ce/ci/yaml/README.html#git-strategy
You can set your git-strategy to none, and manually clone the repo in your before_script section.
This will have some difficulties still - because different runners can service different jobs, if you don't have a dedicated runner for this project, all runners would need access to the repo location.