if i having this kustomize.yaml file :
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
patches:
- patch: |-
- op: replace
path: /spec/rules/0/host
value: the.new.domain.com
target:
kind: Ingress
name: the_name_of_ingress
and i want to replace this value:the.new.domain.com with a new domain name using kustomize command with github actions like this : kustomize edit set
any idea how to make it ? even if u have another idea can letting me implement it inside the github actions its ok
tnx anyway.
You can make use of some yaml processor like yq for this.
Example:
yq -i '.patches[0].patch = "- op: replace
path: /spec/rules/0/host
value: chetantalwar.com"' tes.yaml
I used this using CLI and it updated the file, and similarly you can put it in Github Action as well like give below.
- name: Set foobar to cool
uses: mikefarah/yq#master
with:
cmd: yq -i '.patches[0].patch = "Your Value"' 'kustomize.yml'
Links:
YQ
YQ Github Action
There is one more option which you can try is, templating your kustomize.yaml and in Github Action you can update the respective value using sed.
I had same usecase as #stack-acc and heavily inspired by response from #Chetan, found this
kustomization.yaml
patches:
- patch: |-
- op: replace
path: "/metadata/name"
value: proc-cls-s2e2-tcp
yq command to replace just the value
yq -i '.patches.[0].patch |= sub("value: .*?$", "value: publ-cls-s2e2-udp")' kustomization.yaml
Don't have to repeat whole op: replace section in the script and just replace the value.
Related
Why is every section in this file indented by 2 except the step
image: atlassian/default-image:2
pipelines:
default:
- step:
deployment: production
script:
- git submodule update --recursive --init
- zip -r $FILENAME . -x bitbucket-pipelines.yml *.git*
- pipe: atlassian/bitbucket-upload-file:0.1.6
variables:
BITBUCKET_USERNAME: $BITBUCKET_USERNAME
BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD
FILENAME: $FILENAME
And gives an error if changed to two?
https://bitbucket-pipelines.prod.public.atl-paas.net/validator
This topic seems to say because YAML does not consider - to be the first character? But after pipe is the same sequence with only two?
https://community.atlassian.com/t5/Bitbucket-questions/Is-it-intentional-that-bitbucket-pipelinese-yml-indentation/qaq-p/582084
You will usually see two things in YAML
Sequences — in other programming languages, this is commonly referred as arrays, lists, ...:
- apple
- banana
- pear
Mappings — in other programming languages, this is commonly referred as objects, hashes, dictionaries, associative arrays, ...:
question: how do I foobar?
body: Can you help?
tag: yaml
So if you want to store a mapping of mapping you would do:
pier1:
boat: yes
in_activity: yes
comments: needs some painting in 2023
## The mapping above this comment refers to the pier1,
## as it is indented one level below the key `pier1`
pier2:
boat: no
in_activity: no
comments: currently inactive, needs urgent repair
## The mapping above this comment refers to the pier2,
## as it is indented one level below the key `pier2`
While, if you store a list in a mapping, you could go without the indent, indeed:
fruits:
- apple
- banana
- pear
Is strictly equal to
fruits:
- apple
- banana
- pear
But, if you are trying to indent the first step of your pipeline only, like so:
pipelines:
default:
- step:
deployment: production
You end up with an valid YAML, but, a YAML that does not have the same meaning anymore.
When your original yaml means: I do have a default pipeline with a list of actions, the first action being a step that consists of a deployment to production.
The resulting incorrectly indented YAML means: I do have a default pipeline with a list of actions, the first action having two properties, the first property being an empty step, and the second property is that it is a deployment to production.
So, here, really, the deplyement key that was a property of the step mapping became a property of the first element of the list default!
To indent all this as you would like, you will have to go:
pipelines:
default:
- step:
deployment: production
## ^-- This property is now further indented too
So, you end up with the YAML:
image: atlassian/default-image:2
pipelines:
default:
- step:
deployment: production
script:
- git submodule update --recursive --init
- zip -r $FILENAME . -x bitbucket-pipelines.yml *.git*
- pipe: atlassian/bitbucket-upload-file:0.1.6
variables:
BITBUCKET_USERNAME: $BITBUCKET_USERNAME
BITBUCKET_APP_PASSWORD: $BITBUCKET_APP_PASSWORD
FILENAME: $FILENAME
I'm trying to download multiple artifacts into different servers(like web, db) using environments. Currently i have added the task DownloadPipelineArtifact#2 in a file and using template to add that task in azure-pipelines.yml. As i'm having multiple artifacts, im trying to use for loop where i'm getting issues.
#azure-pipelines.yml
- template: artifacts-download.yml
parameters:
pipeline:
- pipeline1
- pipeline2
- pipeline3
path:
- path1
- path2
- path3
I need to write loop in yaml so that it should download the pipeline1 artifacts to path1 and so on. Can someone please help??
Object-type parameters are your friend. They are incredibly powerful. As qBasicBoy answered, you'll want to make sure that you group the multiple properties together. If you're finding that you have a high number of properties per object, though, you can do a multi-line equivalent.
The following is an equivalent parameter structure to what qBasicBoy posted:
parameters:
- name: pipelines
type: object
default:
- Name: pipeline1
Path: path1
- Name: pipeline2
Path: path2
- Name: pipeline3
Path: path3
An example where you can stack many properties to a single object is as follows:
parameters:
- name: big_honkin_object
type: object
default:
config:
- appA: this
appB: is
appC: a
appD: really
appE: long
appF: set
appG: of
appH: properties
- appA: and
appB: here
appC: we
appD: go
appE: again
appF: making
appG: more
appH: properties
settings:
startuptype: service
recovery: no
You can, in essence, create an entire dumping ground for everything that you want to do by sticking it in one single object structure and properly segmenting everything. Sure, you could have had "startuptype" and "recovery" as separate string parameters with defaults of "service" and "no" respectively, but this way, we can pass a single large parameter from a high level pipeline to a called template, rather than passing a huge list of parameters AND defining said parameters in the template yaml scripts (remember, that's necessary!).
If you then want to access JUST a single setting, you can do something along the lines of:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "Apps start as a "${{ parameters.settings.startuptype }}
Write-Host "Do the applications recover? "${{ parameters.settings.recovery }}
This will give you the following output:
Apps start as a service
Do the applications recover? no
YAML and Azure Pipelines are incredibly powerful tools. I can't recommend enough going through the entire contents of learn.microsoft.com on the subject. You'll spend a couple hours there, but you'll come out the other end with an incredibly knowledge of how these pipelines can be tailored to do everything you could ever NOT want to do yourself!
Notable links that helped me a TON (only learned this a couple months ago):
How to work with the YAML language in Pipelines
https://learn.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema
How to compose expressions (also contains useful functions like convertToJSON for your object parameters!)
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/expressions?view=azure-devops
How to create variables (separate from parameters, still useful)
https://learn.microsoft.com/en-us/azure/devops/pipelines/build/variables?view=azure-devops&tabs=yaml
SLEEPER ALERT!!! Templates are HUGELY helpful!!!
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops
You could use an object with multiple properties
parameters:
- name: pipelines
type: object
default:
- { Name: pipeline1, Path: path1 }
- { Name: pipeline2, Path: path2 }
- { Name: pipeline3, Path: path3 }
steps:
- ${{each pipeline in parameters.pipelines}}:
# use pipeline.Name or pipeline.Path
Goal:
In GitHub Actions, to define my commit message dynamically from shell:
- name: Commit changes
uses: EndBug/add-and-commit#v7
with:
message: "added on $(date -I)"
However, it seems that I have to define a environment variable then use it. I'm following How do I set an env var with a bash expression in GitHub Actions? and other help files like this, but still cannot tell how to make use of such environment variable that I've define previously. This is what I tried but failed:
- name: Checkout repo
uses: actions/checkout#v2
- run: |
touch sample.js
echo "today=$(date -I)" >> $GITHUB_ENV
- name: Commit changes
uses: EndBug/add-and-commit#v7
with:
message: "added on ${today}"
How to make it works?
If you want to reference an environment variable set using the $GITHUB_ENV environment file in the arguments to another task, you'll need to use workflow syntax to access the appropriate key of the top level env key, like this:
- name: Commit changes
uses: EndBug/add-and-commit#v7
with:
message: "added on ${{env.today}}"
You can access it as a standard environment from inside of a running task, for example:
- name: Show an environment variable
run: |
echo "today is $today"
In that example, the expression $today is expanded by the shell,
which looks up the environment variable named today. You could also
write:
- name: Show an environment variable
run: |
echo "today is ${{env.today}}"
In this case, the expansion would be performed by github's workflow
engine before the run commands execute, so the shell would see a
literal command that looks like echo "today is 2021-07-14".
You can accomplish something similar using output parameters, like this:
- name: "Set an output parameter"
id: set_today
run: |
echo "::set-output name=today::$(date -I)"
- name: Commit changes
uses: EndBug/add-and-commit#v7
with:
message: "added on ${{steps.set_today.outputs.today}}"
Using output parameters is a little more granular (because they are
qualified by the step id), and they won't show up in the environment
of processes started by your tasks.
I'm getting the following error in a yaml file
Error : end of the stream or a document separator is expected at line 2, column 11:
apiVersion: v1
^
Line : undefined undefined
This is the entire contents of the file
<<EOT
apiVersion: v1
clusters:
- cluster:
server: https://3F46DDD9022BD149B88DA7ED4AFB2B30.gr7.eu-west-1.eks.amazonaws.com
certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjR> name: kubernetes
contexts:
- context:
cluster: kubernetes
user: aws
name: aws
current-context: aws
kind: Config
preferences: {}
users:
- name: aws
user:
exec:
apiVersion: client.authentication.k8s.io/v1alpha1
command: aws-iam-authenticator
args:
- "token"
- "-i"
- "terraform-eks-demo"
EOT
I've been googling this for hours but I'm getting nowhere tried . I've tried every YAML linter I can find, can anybody point me in the right direction? I am new to this, if that wasn't obvious
You need to remove <<EOT and EOT. You probably copied them from some code that injected YAML into a command line utility via a heredoc but they should not part of the YAML content.
What happens is that when the YAML processor sees <<EOT (which parses as part of a YAML scalar because < is not a special character) and then a new line. A multiline scalar at root level must be the single root node of a document. When the second line is read, the YAML processor sees a : which is not allowed here. At this point, both lines have been read as multiline scalar, and a multiline scalar cannot be used as mapping key. Therefore, the processor complains that after the single root node of the YAML document, the document must end, but instead you try to start a mapping.
I think you should keep a space before 'v1' at line2.
From here... https://cloud.google.com/cloud-build/docs/configuring-builds/substitute-variable-values
I am trying to use substitutions inside my cloudbuild.yaml file but almost all my substitutions depend on the project ID of the project I am deploying to.
I have my yaml file like this...
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['functions', 'deploy', 'functionName', ...other args... , '--service-account', '${_SERVICE_ACCOUNT}', '--source', '${_SOURCE_REPO}']
substitutions:
_SOURCE_REPO: 'https://source.developers.google.com/projects/$PROJECT_ID/repos/my-repo-id/moveable-aliases/master/paths/functions/src'
_SERVICE_ACCOUNT: 'blah#${PROJECT_ID}.iam.gserviceaccount.com'
And whatever I try with the substitutions (I have tried both formats $_FOO and ${_FOO}) and what I end up with is blah#${PROJECT_ID}.iam.gserviceaccount.com with the ${PROJECT_ID} text still in there rather than it having the actual project ID.
If I move the text into the step and just use it instead of the substitution then it works. But ideally I'd like to use this method as these values will be used a lot so would like to save repetition.
EDIT
Well, I've tried a few different options including some of the suggestions that #ralemos mentioned but nothing seems to be working.
If I use this format...
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['functions', 'deploy', 'functionName', ...other args... , '--service-account', '${SERVICE_ACCOUNT}', '--source', '${SOURCE_REPO}']
options:
env:
SOURCE_REPO: 'https://source.developers.google.com/projects/${PROJECT_ID}/repos/my-repo-id/moveable-aliases/master/paths/functions/src'
SERVICE_ACCOUNT: 'blah#${PROJECT_ID}.iam.gserviceaccount.com
It complains that the env lines are invalid.
If I use this format...
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['functions', 'deploy', 'functionName', ...other args... , '--service-account', '${SERVICE_ACCOUNT}', '--source', '${SOURCE_REPO}']
options:
env:
'SOURCE_REPO=https://source.developers.google.com/projects/${PROJECT_ID}/repos/my-repo-id/moveable-aliases/master/paths/functions/src'
'SERVICE_ACCOUNT=blah#${PROJECT_ID}.iam.gserviceaccount.com
It complains that SERVICE_ACCOUNT and SOURCE_REPO are not valid built-in substitutions.
If I use $$SOURCE_REPO as the syntax it just replaces it with $SOURCE_REPO rather than using the substitution.
It seems for now that what I'm trying just isn't possible.
You could use env variables instead of substitutions, as you can see on the answer of this community post.
The result would be something like this:
steps:
- name: 'gcr.io/cloud-builders/gcloud'
args: ['functions', 'deploy', 'functionName', ...other args... , '--service-account', '${SERVICE_ACCOUNT}', '--source', '${SOURCE_REPO}']
options:
env:
SOURCE_REPO: 'https://source.developers.google.com/projects/${PROJECT_ID}/repos/my-repo-id/moveable-aliases/master/paths/functions/src'
SERVICE_ACCOUNT: 'blah#${PROJECT_ID}.iam.gserviceaccount.com