Can you please help on my below issue.
As i am doing sonar scanner using cloud build with an advantage of secret manger but facing issue.
And followed same steps of https://cloud.google.com/cloud-build/docs/securing-builds/use-secrets
here is my code
steps:
- name: 'gcr.io/$_PROJECT_ID/sonar-scanner:latest'
entrypoint: 'bash'
args:
- '-c'
- '-Dsonar.host.url=http://sonar:9000/'
- '-Dsonar.login=$$USERNAME'
- '-Dsonar.password=$$PASSWORD'
- '-Dsonar.projectKey=$_BRANCH-analytics'
- '-Dsonar.sources=.'
secretEnv: ['USERNAME', 'PASSWORD']
dir: 'analytics'
availableSecrets:
secretManager:
- versionName: projects/project-id/secrets/sonar_pass/versions/1
env: 'PASSWORD'
- versionName: projects/project-id/secrets/sonar_user/versions/2
env: 'USERNAME'
tags: ['cloud-builders-community']
and the issue i am facing is:
bash: line 0: bash: -Dsonar.login=$USERNAME: invalid option name
ERROR
ERROR: build step 0 "gcr.io/project-id/sonar-scanner:latest" failed: step exited with non-zero status: 2
tried with different items but can't find a solution.
I am grateful if you guys help me on this.
Thank you
I actually had the same problem as you. It is indeed quite important that you use entrypoint: 'bash' and '-c', otherwise Cloud Build doesn't recognise the variables from the secret manager.
My cloudbuild.yaml step looks like this:
steps:
id: 'sonarQube'
name: 'gcr.io/$PROJECT_ID/sonar-scanner:latest'
entrypoint: 'bash'
args:
- '-c'
- |
sonar-scanner -Dsonar.host.url=<url> -Dsonar.login=$$SONARQUBE_TOKEN -Dsonar.projectKey=<project-key> -Dsonar.sources=.
secretEnv: ['SONARQUBE_TOKEN']
availableSecrets:
secretManager:
- versionName: projects/<project-id>/secrets/sonarqube-token/versions/latest
env: 'SONARQUBE_TOKEN'
I had some problems with the latest sonar-scanner image, because it used alpine. I got the next error: jre-bin-java-not-found even though the image has Java. Based on this, I created thus my own Docker image based on Ubuntu instead of Alpine. You can find the image in a pull request.
I found this example of using sonar-scanner in Cloud Build. It seems that sonar-scanner should be used without bash
I think that you should remove entrypoint: 'bash' and '-c'.
The similar approach is in this SO question. It should solve this error.
Related
I'm using a standard gcr.io/k8s-skaffold/pack build function to build my app for google cloud run using google cloud build.
In my cloudbuild.yaml I load 2 secrets from google secrets manager and pass it to the build function. The google cloud build has access to those secrets, otherwise I would get an error message for this (I got this kind of error at the beginning when setting up the build, now it seems to have access).
However, it seems like the environment variables don't get set.
I think that it might be a syntactical problem of how I try to pass the variables.
This is the stripped down cloudbuild.yaml
steps:
- name: gcr.io/k8s-skaffold/pack
args:
- build
- '$_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA'
- '--builder=gcr.io/buildpacks/builder:v1'
- '--network=cloudbuild'
- '--path=.'
- '--env=SEC_A=$$SEC_A'
- '--env=SEC_B=$$SEC_B'
secretEnv: ['SEC_A', 'SEC_B']
id: Buildpack
entrypoint: pack
availableSecrets:
secretManager:
- versionName: projects/<pid>/secrets/SEC_A/versions/latest
env: SEC_A
- versionName: projects/<pid>/secrets/SEC_B/versions/latest
env: SEC_B
An Error message that I hacked into the build for checking shows me that the env var is empty during this build step.
I tried using $, $$ (as seen above), &&, ${...}, for substitution. But maybe the problem lies somewhere else.
Yes, it's a common issue and a trap on Cloud Build. In fact, your secrets can't be read if you use the args[] arrays to pass argument. you have to use the script mode, like that
steps:
- name: gcr.io/k8s-skaffold/pack
entrypoint: bash
args:
- -c
- |
pack build $_GCR_HOSTNAME/$PROJECT_ID/$REPO_NAME/$_SERVICE_NAME:$COMMIT_SHA --builder=gcr.io/buildpacks/builder:v1 --network=cloudbuild --path=. --env=SEC_A=$$SEC_A --env=SEC_B=$$SEC_B
secretEnv: ['SEC_A', 'SEC_B']
id: Buildpack
My spring boot application was working fine in cloud build & deployed without any issue till September.
Now my trigger fails in gcloud app deploy.
Step #4: ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: Deployment cannot use a pre-built image. Pre-built images are only allowed in the App Engine Flexible Environment.
app.yaml
runtime: java11
env: standard
service: service
handlers:
- url: /.*
script: this field is required, but ignored
cloudbuild.yaml
steps:
# backend deployment
# Step 1:
- name: maven:3-jdk-14
entrypoint: mvn
dir: 'service'
args: ["test"]
# Step 2:
- name: maven:3-jdk-14
entrypoint: mvn
dir: 'service'
args: ["clean", "install", "-Dmaven.test.skip=true"]
# Step 3:
- name: docker
dir: 'service'
args: ["build", "-t", "gcr.io/service-base/base", "."]
# Step 4:
- name: "gcr.io/cloud-builders/docker"
args: ["push", "gcr.io/service-base/base"]
# Step 5:
- name: 'gcr.io/cloud-builders/gcloud'
dir: 'service/src/main/appengine'
args: ['app', 'deploy', "--image-url=gcr.io/service-base/base"]
timeout: "30m0s"
# Step 6:
# dispatch.yaml deployment
- name: "gcr.io/cloud-builders/gcloud"
dir: 'service/src/main/appengine'
args: ["app", "deploy", "dispatch.yaml"]
timeout: "30m0s"
timeout: "100m0s"
images: ["gcr.io/service-base/base"]
Cloud build error
Thanks in advance. Im confused how my build was working fine before & what am i doing wrong now.
You can't deploy custom container on App Engine standard. You have to provide your code and the environment runtime. Then Buildpack is used to create a standard container on Google Side (for information, a new Cloud Build job is ran for this) and deployed on App Engine.
I recommend you to have a look to Cloud Run to use your custom container. It's very close to App Engine (and even better on many points!) and very customizable.
What your cloudbuild.yaml comment's refer to as Step 5 corresponds to the Step #4 in the error because system begins numbering steps from 0.
The error message is accurate; App Engine standard (!) differs from App Engine flexible in that the latter (flexible) permits container image deployments. App Engine standard deploys from sources.
See Google's example.
It's possible that something has changed Google's side that's causing the issue but, the env: standard in your app.yaml suggests the build file has changed.
I'm currently attempting to use the commands feature available in CircleCI version 2.1, so that I can reuse some common commands. I'm testing using the CLI command:
circleci config process ./.circleci/config.latest.yaml > ./.circleci/config.yml
But I recieve the following error:
Error: Error calling workflow: 'main'
Error calling job: 'build'
Error calling command: 'build_source'
Cannot find a definition for command named restore-cache
It seems that restore-cache works just fine in a straight-up version 2 config file, but when I try and process a 2.1 file using process it kicks up a fuss.
Below is an edited version of my config.yaml file which should hopefully be of some use. Please let me know if there is any additional information that would be useful.
version: 2.1
defaults: &defaults
/**
* Unimportant stuff
*/
aliases:
- &restore-root-cache
keys:
- v1-deps-{{ .Branch }}-{{ checksum "package.json" }}
- v1-deps-{{ .Branch }}
- v1-deps
commands:
build_source:
description: 'Installs dependencies, then builds src, builds documentation, and runs tests'
steps:
- restore-cache: *restore-root-cache
- other-commands...
jobs:
build:
<<: *defaults
steps:
- checkout
- build_source
workflows:
version: 2.1
main:
jobs:
- build:
filters:
branches:
ignore: develop
The command is restore_cache (with an underscore), not restore-cache (with a dash) https://circleci.com/docs/2.0/configuration-reference/#restore_cache
It should work in commands.
restore cache is a special step that needs to be under a job. Not another command.
I'm running the cypress-example-kitchen sink app on CircleCI.
This is my yaml config script:
version: 2.1
orbs:
cypress: cypress-io/cypress#1.0.1
workflows:
build:
jobs:
- cypress/install:
build: 'npm run build'
- cypress/run:
requires:
- cypress/install
start: 'npm start'
This kicks off and passes just fine when I make a commit to my fork of the repo above.
However, when I try to execute a CircleCI build programmatically, using
curl -X POST https://circleci.com/api/v1.1/project/github/Atticus29/cypress-example-kitchensink?circle-token=myApiToken, the build fails and the jobs dashboard on CircleCI tells me that something is wrong with my config file:
6 schema violations found required key [jobs] not found workflows:
5 schema violations found
workflows: minimum size: [2], found: 1
workflows: build: jobs: 4 schema violations found
workflows: build: jobs: 0: 0 subschemas matched instead of one
workflows: build: jobs: 0: expected type: String, found: Mapping
workflows: build: jobs: 0: install: extraneous key [build] is not permitted
workflows: build: jobs: 1: 0 subschemas matched instead of one
workflows: build: jobs: 1: expected type: String, found: Mapping
workflows: build: jobs: 1: run: extraneous key [start] is not permitted
And that something went wrong with my build:
Build-agent version 0.1.1216-48f80d08 (2018-12-07T16:01:40+0000)
Configuration errors: 2 errors occurred:
Configuration version 2.1 requires the "Enable Build Processing" project setting. Enable Build Processing under Project Settings ->
Advanced Settings. In order to retrigger build processing, you must
push a new commit.
Cannot find a job named build to run in the jobs: section of your configuration file. If you expected a workflow to run, check your
config contains a top-level key called 'workflows:'
I can confirm that Enable Build Processing is on.
None of these were problems when I ran the build in the usual way. Any advice?
Circle CI for some reason keeps on assuming that the projects are not set up for v2.0 despite config.yml being called the right thing and living in the right place in the repo. After a few commits, this issue seems to go away?
I ended up running a build programmatically with the following script:
#!/bin/bash
PERSONAL_TOKEN=myPersonalTokenHere
MOST_RECENT_BUILD=`curl -s "https://circleci.com/api/v1.1/recent-builds?circle-token=$PERSONAL_TOKEN&limit=1"| grep 'build_num'|grep -o '\d.'|sed 's/,//g'|sort -r -n|head -n1`
curl -X POST "https://circleci.com/api/v1.1/project/github/holmbergius/wildMeCypress/$MOST_RECENT_BUILD/retry?circle-token=$PERSONAL_TOKEN"
I'm working with the following .gitlab-ci.yml config:
image: node:latest
stages:
- build
- test
build:
stage: build
script:
- npm install
test:
stage: test
script:
- ./testing/test.js
When pushing to GitLab, i get a 'Your pipeline as failed' error and when i look at what failed, its a yaml error:
Status: syntax is incorrect
Error: (<unknown>): did not find expected key while parsing a block mapping
at line 1 column 1
As far as i can see, the image: node:latest is correct. Any help with this would be great.
The problem is the indentation of you test job. That problem is making your whole yml to break therefore raising the error on line 1. Just remove the excessive whitespaces like the code below and it will be fine.
image: node:latest
stages:
- build
- test
build:
stage: build
script:
- npm install
test:
stage: test
script:
- ./testing/test.js
Note that in YAML the indentation is used to denote the structure. So it is important to pay attention to it.