When executing a workflow there is a unique Execution ID. Is it possible to access this value from within the workflow. For example, if I was to use the Execution ID as the filename in a step:
url: https://storage.googleapis.com/upload/storage/v1/b/bucketname/o
headers:
Content-Type: application/json
query:
uploadType: media
name: ${string(EXECUTION_ID) + ".json"}
```
As of now, it's not possible to get the Workflow execution id.
The only Environment variables that you can access are the following:
GOOGLE_CLOUD_PROJECT_NUMBER: The workflow project's number.
GOOGLE_CLOUD_PROJECT_ID: The workflow project's identifier.
GOOGLE_CLOUD_LOCATION: The workflow's location.
GOOGLE_CLOUD_WORKFLOW_ID: The workflow's identifier.
GOOGLE_CLOUD_WORKFLOW_REVISION_ID: The workflow's revision
identifier.
You can access them within the workflow with sys.get_env(). For example:
- getProjectID:
assign:
- projectID: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
See source
Related
Is it possible to reference OpenAPI operation description from an external file?
Here is my sample code. I want to keep the description "This API is used to get user details" in a separate file and use it here like a variable or template or as a reference. Is there any way to do this?
get:
tags:
- User
summary: Get user details
description: This API is used to get user details
operationId: updateUser
parameters:
- name: userid
in: path
description: The id that needs to be pulled
required: true
schema:
type: string
If you use Redocly CLI to bundle, then you can put it in a separate file like this:
get:
tags:
- User
summary: Get user details
description:
$ref: ./updateUser-description.md
operationId: updateUser
parameters:
- name: userid
in: path
description: The id that needs to be pulled
required: true
schema:
type: string
Then, in a separate file named updateUser-description.md (note, you could change the name too):
This API is used to get user details
Then, when you run the bundle command it resolves the $ref and replaces the description with the contents in that corresponding Markdown file.
npx #redocly/cli bundle my-openapi.yaml
I would like to use the Secret Manager to store a credential to our artifactory, within a cloud build step. I have it working using a build similar to:
steps:
- name: 'busybox:glibc'
entrypoint: 'sh'
args: ['-c', 'env']
secretEnv: ['SECRET_VALUE']
availableSecrets:
secretManager:
- versionName: "projects/PROJECT_ID/secrets/TEST-SECRET/versions/1"
env: 'SECRET_VALUE'
All great, no problems - I then try and slightly improve it to:
steps:
- name: 'busybox:glibc'
entrypoint: 'sh'
args: ['-c', 'env']
secretEnv: ['SECRET_VALUE']
availableSecrets:
secretManager:
- versionName: "projects/$PROJECT_ID/secrets/TEST-SECRET/versions/1"
env: 'SECRET_VALUE'
But then it throws the error:
ERROR: (gcloud.builds.submit) INVALID_ARGUMENT: failed to get secret name from secret version "projects/$PROJECT_ID/secrets/TEST-SECRET/versions/1"
I have been able to add a TRIGGER level env var (SECRET_MANAGER_PROJECT_ID), and that works fine. The only issue that as that is a trigger env, it is not available on rebuild, which breaks a lot of things.
Does anyone know how to get the PROJECT_ID of a Secret Manager from within CloudBuild without using a Trigger Param?
For now, it's not possible to set dynamic value in the secret field. I already provided this feedback directly to the Google Cloud PM, it has been take into account, but I don't have more info to share, especially for the availability.
EDIT 1
(January 22). Thanks to Seza443 comment, I tested again and now it works with automatically populated variable (PROJECT_ID and PROJECT_NUMBER), but also with customer defined substitution variables!
It appears that Cloud Build now allows for the use of substitution variables within the availableSecrets field of a build configuration.
From Google Cloud's documentation on using secrets:
After all the build steps, add an availableSecrets field to specify the secret version and environment variables to use for your secret. You can include substitution variables in the value of the secretVersion field. You can specify more than one secret in a build.
I was able to use the $PROJECT_ID variable in my own build configuration like so:
...
availableSecrets:
secretManager:
- versionName: projects/$PROJECT_ID/secrets/api-key/versions/latest
env: API_KEY
Granted, there appears to be (at least at present) some discrepancy between the documentation quoted above and the recommended configuration file schema. In the documentation they refer to secretVersion, but that appears to have changed to versionName. In either case, it seems to work properly.
Use the $PROJECT_NUMBER instead.
https://cloud.google.com/build/docs/configuring-builds/substitute-variable-values#using_default_substitutions
I've 2 multibranch jenkins pipeline jobs connecting same github project. I'm triggering first MB job ex. dev. branch, now on success of this job I want to trigger second MB job with same branch name. (I've separate Jenkinsfile fot both MB job).
I've tried below options but didn't work:
(1)
build job: 'jobName', parameters: [[$class: 'StringParameterValue', name: 'BRANCH_NAME', value: env.BRANCH_NAME]]
This gives me error "Waiting for non-job items is not supported"
(2)
build job: 'jobName/${branch_name}'
This gives me jobname/${branch_name} does not exist means variable here is not resolving, I've created this variable in environment directive
Simple jobName/dev if I'd give hardcode in Jenkinsfile then it's working, but I'd need it with automatically selecting branch name.
please advise, thanks.
When you want to use variables inside your string you need to use double quotes, example:
$"BRANCH_NAME"
From the groovy docs:
"Any Groovy expression can be interpolated in all string literals,
apart from single and triple-single-quoted strings. "
More here:
https://docs.groovy-lang.org/latest/html/documentation/#all-strings
An alternative could also be to use concatenation:
build job:'job/'+somebranchvariable, parameters:[string(name: 'BUILD_NAME', value: build_name)]
I have tried this format, it is working.
build(job: "JOBNAME" + "/" + "${BRANCH_NAME}".replaceAll('/', '%2F'))
Forward slash (/) is converted to %2f in the job name:
It’s a common practice to use a forward slash (/) when creating feature branches within developer teams, so whenever a new Pipeline gets created automatically with the creation of a new branch, / will get converted into %2f.
Whenever you are referencing the branch name in the job or calling any job as a downstream, you will have to make sure of this conversion. Look at the below example where we are trying to call a downstream project.
I've found another solution for this issue:
In the jenkinsfile triggering pipeline job with gitParameter:
build job: 'jobName', parameters: [gitParameter(name: 'GITBRANCH', value: env.BRANCH_NAME)]
Installed "git parameter" plugin, and configure in job which is being triggered in jenkinsfile
parameters {
gitParameter branchFilter: 'origin/(.*)', defaultValue: 'master', name: 'GITBRANCH', type: 'PT_BRANCH'
}
And calling this parameter while checkout code
stage("Checkout Branch wise"){
steps{
git branch: "${params.GITBRANCH}", url: 'https://xxx/xxx/xxxx.git'
}
}
which solved the problem.
I am getting
task config 'get-snapshot-jar/infra/hw.yml
not found error. I have written a very simple pipeline .yml, this yml will connect to artifactory resource and run another yml which is defined in task section.
my pipeline.yml looks like:
resources:
- name: get-snapshot-jar
type: docker-image
source: <artifactory source>
repository: <artifactory repo>
username: {{artifactory-username}}
password: {{artifactory-password}}
jobs:
- name: create-artifact
plan:
- get: get-snapshot-jar
trigger: true
- task: copy-artifact-from-artifact-repo
file: get-snapshot-jar/infra/hw.yml
Artifactiory is working fine after that I am getting an error
enter image description here
copy-artifact-from-artifact-repo
task config 'get-snapshot-jar/infra/hw.yml' not found
You need to specify an input for your copy-artifact-from-artifact-repo task which passes the get-snapshot-jar resource to the tasks docker container. Take a look at this post where someone runs into a similar problem Trigger events in Concourse .
Also your file variable looks weird. Your are referencing a docker-image resource which, according to the official concourse-resource github repo, has no yml files inside.
Generally i would keep my task definitions as close as possible to the pipeline code. If you have to reach out to different repos you might loose the overview if your pipeline keeps growing.
cheers,
I have a fairly complex Gradle build process, one step of which requires communication with a service, for which I need a special token file. I can generate this token file just fine manually, but I'd like to add it to my build so that, if the token file is not present, the user is prompted for their credentials.
I have referenced the following blog post about this: https://www.timroes.de/2014/01/19/using-password-prompts-with-gradle-build-files/, but it is missing a small detail for me. I do not want to unconditionally prompt the user for their credentials; I only want to prompt them if the token file needs to be created.
Let's say I have the following task:
task createToken(type: Exec) {
dependsOn userCredentialPrompt
outputs.file 'creds.token'
environment 'USERNAME', project.username
environment 'PASSWORD', project.password
executable './create_token_file'
}
task somethingRequiringToken {
dependsOn: createToken
inputs.file 'creds.token'
// Other task-specific configuration here.
}
And the userCredentialPrompt task is one that prompts the user for their credentials and assigns the project.username and project.password to the correct values (using logic like the linked blog post above).
The problem here is that both the username and password are null. I'm pretty sure I understand the reason why. They're set at configuration time, but the prompt for user credentials takes place at execution time. That said, I don't know the right way to get Gradle to behave so that the following happens:
When somethingRequiringToken is executed, if the creds.token file is missing, it runs createToken.
When somethingRequiringToken is executed, if the creds.token file is present, the createToken task is not executed.
When createToken is executed, the user is prompted for their credentials.
When the user credentials are acquired, they are passed (reasonably securely) to an external executable that generates the creds.token file.