I'm trying to set up a GitLab runner for a repository. I've installed and registered a runner on a remote machine. The runner in the Gitlab setting page is in the green light. Run untagged jobs has been set to true.
The pipeline is running, but all the jobs inside this pipeline are all pending.
The status of the GitLab runner on the remote machine:
And here is my simple script of .gitlab-ci.yml:
stages:
- build
- test
image: "ruby:2.6.5"
build-ruby-bundler:
stage: build
script:
- apt-get update -qq
- ruby -v
- which ruby
- gem install bundler --no-document
- bundle install --jobs $(nproc) "${FLAGS[#]}"
rubocop:
stage: test
script:
- bundle exec rubocop
I just realized, I registered the GitLab runner in user mode, which I need to register in system mode using sudo gitlab-runner register.
Related
If I push to master, it works perfectly, all environment variables available and I have a successful deploy to Heroku.
Problem: If I push to the dev branch, it can't see the environment variables for the deploy.
$ dpl --provider=heroku --app=$HEROKU_DEV_APP --api-key=$HEROKU_API_KEY
invalid option "--api-key="
ERROR: Job failed: exit code 1
Environment settings:
.gitlab-ci.yml:
stages:
- build
- test
- deploy
build:
stage: build
image: maven:3.6.3-jdk-14
script:
- mvn clean package
tags:
- docker
test:
stage: test
image: maven:3.6.3-jdk-14
script:
- mvn test
tags:
- docker
deploy_dev:
stage: deploy
image: ruby:2.3
script:
- apt-get update -qy
- apt-get install -y ruby-dev
- gem install dpl
- dpl --provider=heroku --app=$HEROKU_DEV_APP --api-key=$HEROKU_API_KEY
environment:
name: prod
url: https://.....herokuapp.com/
only:
- dev
tags:
- docker
deploy_prod:
stage: deploy
image: ruby:2.3
script:
- apt-get update -qy
- apt-get install -y ruby-dev
- gem install dpl
- dpl --provider=heroku --app=$HEROKU_PROD_APP --api-key=$HEROKU_API_KEY
environment:
name: prod
url: https://.....herokuapp.com/
when: manual
only:
- master
tags:
- docker
This is because your Heroku api key variable is set as protected.
Protected variables are visible only by protected branches and protected tags. That is why it works for you on master but not on dev.
More information: https://gitlab.com/help/ci/variables/README#protect-a-custom-variable and https://gitlab.com/help/user/project/protected_branches.md
Your options are: either remove protected flag, or introduce another unprotected variable with another api key for your non-protected branches which would be less sensitive.
I’m a little confused about how Postman (Newman) tests would execute against a build unless that build is running somewhere. Wouldn’t I need to deploy it somewhere and THEN execute Travis CI?
I connected Github to Travis & Heroku, I think I need to link them in the .travis.yml file.
.travis.yml
language: node_js
node_js:
- "12.14.1"
install:
- npm install newman
- npm install jest
before_script:
- node --version
- npm --version
- yarn --version
- node_modules/.bin/newman --version
- node_modules/.bin/jest --version
deploy:
provider: heroku
api_key:
secure: <HEROKU_API_KEY>
app: <HEROKU_APP_NAME>
on:
repo: <GITHUB_REPOSITORY>
script:
- node_modules/.bin/newman run <COLLECTION_LINK> --environment <ENV_LINK>
- yarn test
What should I specify to run tests after the build & deploy? Am I missing a step?
What you are looking for is build stages, see docs https://docs.travis-ci.com/user/build-stages/.
The syntax is pretty straightforward.
jobs:
include:
- stage: install
script: npm run install
- stage: build
script: npm run build
- stage: deploy
deploy:
provider: heroku
api_key:
secure: <HEROKU_API_KEY>
app: <HEROKU_APP_NAME>
on:
repo: <GITHUB_REPOSITORY>
- stage: test
script: npm run tests
I am planning to migrate my existing CI to CircleCI 2 in favor of the docker. Once I tried to trigger a build it's shown something like this:
I also checked the steps as indicated in the below alert here https://circleci.com/sunset1-0/
Service alert: Your project references CircleCI 1.0 or it has no
configuration. CircleCI 1.0 and projects without configuration files
are no longer supported. You must update your project to use CircleCI
2.0 configuration to continue. Learn more.
Is there anything I missed out?
Below is my .circleci/config.yml
version: 2 # use CircleCI 2.0
jobs: # a collection of steps
build: # runs not using Workflows must have a `build` job as entry point
parallelism: 3 # run three instances of this job in parallel
docker: # run the steps with Docker
- image: circleci/ruby:2.4.2-jessie-node # ...with this image as the primary container; this is where all `steps` will run
environment: # environment variables for primary container
BUNDLE_JOBS: 3
BUNDLE_RETRY: 3
BUNDLE_PATH: vendor/bundle
RAILS_ENV: test
- image: mysql:5.7 # database image
environment: # environment variables for database
MYSQL_ALLOW_EMPTY_PASSWORD: 'yes'
steps: # a collection of executable commands
- checkout # special step to check out source code to working directory
- run:
name: setup
command: |
curl -sL https://deb.nodesource.com/setup_10.x | bash \
&& curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - \
&& echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list
- run:
name: Dependencies
command: |
apt-get update && \
DEBIAN_FRONTEND=noninteractive apt-get install -y \
build-essential mysql-client nodejs yarn && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
# Which version of bundler?
- run:
name: Which bundler?
command: bundle -v
# Restore bundle cache
# Read about caching dependencies: https://circleci.com/docs/2.0/caching/
- restore_cache:
keys:
- rails-demo-bundle-v2-{{ checksum "Gemfile.lock" }}
- rails-demo-bundle-v2-
- run: # Install Ruby dependencies
name: Bundle Install
command: bundle check --path vendor/bundle || bundle install --deployment
# Store bundle cache for Ruby dependencies
- save_cache:
key: rails-demo-bundle-v2-{{ checksum "Gemfile.lock" }}
paths:
- vendor/bundle
# Only necessary if app uses webpacker or yarn in some other way
- restore_cache:
keys:
- rails-demo-yarn-{{ checksum "yarn.lock" }}
- rails-demo-yarn-
- run:
name: Yarn Install
command: yarn install --cache-folder ~/.cache/yarn
# Store yarn / webpacker cache
- save_cache:
key: rails-demo-yarn-{{ checksum "yarn.lock" }}
paths:
- ~/.cache/yarn
- run:
name: Wait for DB
command: dockerize -wait tcp://localhost:5432 -timeout 1m
- run:
name: Database setup
command: bin/rails db:schema:load --trace
- run:
name: Run rspec in parallel
command: |
bundle exec rspec --profile 10 \
--format RspecJunitFormatter \
--out test_results/rspec.xml \
--format progress \
$(circleci tests glob "spec/**/*_spec.rb" | circleci tests split --split-by=timings)
# Save test results for timing analysis
- store_test_results: # Upload test results for display in Test Summary: https://circleci.com/docs/2.0/collect-test-data/
path: test_results
# See https://circleci.com/docs/2.0/deployment-integrations/ for example deploy configs
In order for the build work, your config.yml file must live inside a .circleci folder at the root of your project directory.
You post a very good description. Thanks for the code. The version config looks fine. However, you wrote that your config currently lives at .circle/config.yml (Notice the missing ci at end of circle)
Move the config.yml to .circleci/config.yml and the build should proceed further. (right now the build is failing because it has no configuration.
E
I have the following yml configuration file with 3 different jobs:
stages:
- build
- test
- analyze
build:
stage: build
script:
- apt-get update -qy
- apt-get install -y python3-dev python3-pip
- pip3 install -r requirements.txt
test:
stage: test
script:
- coverage run --source='.' manage.py test
cache:
paths:
- .coverage
analyze:
stage: analyze
script:
- flake8
- coverage report
In the first job I install the requirements, among which are coverage or flake8. But these tools are used in the following jobs. I have tried using 'dependencies' or 'cache', but it didn't work: only files/dirs under the project root directory can be shared, not the binaries under /user/local/bin.
I have tried to indicate another directory for pip install, but the binary is installed in /user/local/bin.
The workaround I have found is to install the dependencies in each job, but I think that this is the less optimal solution.
I think that there must be a better solution for that.
Thanks.
I just found a solution, at least for python3 (enough for me):
python3 has a built-in tool for managing virtual envs: venv
Using venv, we can create the virtual env in the project root dir, cache this dir, and enable our virtual env in each job.
variables:
VENV_NAME: "env"
cache:
paths:
- $VENV_NAME
first_job:
script:
- apt-get update -qy
- apt-get install -y python3-dev python3-pip python3-venv
- python3 -m venv $VENV_NAME
- source $VENV_NAME/bin/activate
- pip3 install -r requirements.txt
next_jobs:
script:
- source $VENV_NAME/bin/activate
- echo "hello world!"
PD: don't forget to exclude virtual env dir from coverage or other analysis tools
The pipeline .gitlab-ci.yml code successfully works till yesterday, but today i got the error which says “dpl command not found”
the below is my .gitlab-ci.yml file
image: node:8.9.3
stages:
- job1
- test
- production
job1:
stage: job1
script: "ls -l"
test:
stage: test
script:
- npm install
production:
type: deploy
stage: production
image: ruby:latest
script:
- apt-get update -qy
- apt-get install -y ruby-dev
- gem install dpl
- dpl --provider=heroku --app=quailapp --api-key=$HEROKU_PRODUCTION_API_KEY
only:
- master
This is the log Generated,
Setting up rake (10.5.0-2) ...
Setting up libruby2.3:amd64 (2.3.3-1+deb9u2) ...
Setting up ruby2.3 (2.3.3-1+deb9u2) ...
Setting up ruby2.3-dev:amd64 (2.3.3-1+deb9u2) ...
Setting up ruby-dev:amd64 (1:2.3.3) ...
Setting up ruby (1:2.3.3) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
$ gem install dpl
Successfully installed dpl-1.9.6
1 gem installed
$ dpl --provider=heroku --app=quailapp --api-key=$HEROKU_PRODUCTION_API_KEY
/bin/bash: line 68: dpl: command not found
ERROR: Job failed: exit code 1
please help me for finding the solution.
Same here, Issuing the command to install dpl with verbosity: gem install dpl --verbose I've been able to see something weird:
/usr/local/bundle/bin/dpl
Successfully installed dpl-1.9.6
1 gem installed
I don't know why but it is installed in a non-default path. As a workaround I've added the /usr/local/bundle/bin in $PATH environment variable issuing the following command:
export PATH=$PATH:/usr/local/bundle/bin
It works for me and my gitlab ci pipelines are now working again.
BTW, It would be great to know why it has changed suddenly...
Same problem here. I think, it's a problem in docker image. See https://github.com/docker-library/ruby/pull/209
They made some changes and broke path for gems binaries. We have to wait until they merge fix.
UPDATE:
It's already merged and their fix works for me.