I'm triggering a build on a web hook from a pull-request, and then it reads in the payload of the request. The payload has things like PR name, author, commit hash, etc.
I'd like to inject some of these into the tag attribute but their format doesn't comply:
invalid build: invalid build tag "Ari Example": must match format "^[\\w][\\w.-]{0,127}$"
Is there any way I could modify the tag dynamically, in the example above for a users name?
Ari Example -> ari-example
My cloudbuild.yaml looks like so:
substitutions:
# These a dynamically populated by the PR and substitutions
# below is just for illustrative purposes
_PR_AUTHOR_ID: '827364872-23472634-2352'
_PR_AUTHOR: 'Ari Example'
_PR_TITLE: 'fix: update some file'
steps:
- id: 'Pull Request Payload'
name: 'gcr.io/cloud-builders/git'
entrypoint: '/bin/bash'
args:
- '-c'
- |
echo Author: $_PR_AUTHOR (id: $_PR_AUTHOR_ID)
echo Title: $_PR_TITLE
tags: ['$_PR_AUTHOR']
Pure bash. Below you replace any spaces with dashes and convert the string to lowercase:
_PR_AUTHOR='Ari Example'
echo ${_PR_AUTHOR// /-} | awk '{print tolower($0)}'
Output: ari-example
If you want to convert special characters and add more validation you might need to use regex.
Related
The yaml file is like this:
settings:
clusterName: dev
dbServer: test.db.com
dcsHostName: dev
overrideDevRepo: "true"
globalSuspend: "true"
sharedConfig: "true"
I want to pass globalSuspend as a toggle variable, update true to false or false to true, but as helm need the quote, so quote are needed, also yaml is format sensitive, the leading spaces must be kept as same. Need a way to do it saltstack simple cmd.run or state playbook. Or at least please help on how to do in bash. But I find my command works in bash does not work in saltstack as : colon is a trouble in salt. thanks.
"toggle the suspend value":
cmd.run:
- name: sed -i '/globalSuspend/!b;c \ \ globalSuspend: "{{suspend}}"' settings.yaml
- runas: vagrant
The above does not work because of the colon and also does not keep the leading spaces.
Colon is a special character in YAML. Use a literal block scalar to treat everything as content:
"toggle the suspend value":
cmd.run:
- name: |-
sed -i '/globalSuspend/!b;c \ \ globalSuspend: "{{suspend}}"' settings.yaml
- runas: vagrant
| starts a literal block scalar. Each following line that is more indented is part of the scalar. The - means that the final line break is not part of the scalar (wich it would otherwise be).
The simplest way to do this is with a file.serialize state.
toggle the suspend value:
file.serialize:
- name: settings.yaml
- dataset:
settings:
globalSuspend: "{{ suspend }}"
- serializer: yaml
- merge_if_exists: true
More specific states are always preferred to a generic cmd.run, as they are state-aware and more efficient to run.
I have the following YAML object:
# profile fields
firstName: string
lastName: string
phoneNum: string # optional
active: boolean
notes: string # optional
role: string # admin | market | client
by: User_Id
Is it allowed to add comments beside the properties' names such as:
notes: string # optional
role: string # admin | market | client
I couldn't find a resource that mentions that, and I have no way to test this
Yes, it's allowed by the YAML spec, latest version 1.2.2: https://yaml.org/spec/1.2.2/.
The same examples that you posted appear on the spec's section on 2.1 Collections:
Example 2.2 Mapping Scalars to Scalars (player statistics)
hr: 65 # Home runs
avg: 0.278 # Batting average
rbi: 147 # Runs Batted In
The full comment specs has its own section, 6.6 Comments. The main rules are that it starts with a # and "must be separated from other tokens by white space characters". It can also span multiple lines.
firstName: john # comment line 1
# comment line 2
lastName: doe
As for knowing if it would work and
I have no way to test this
You can use any of the popular YAML linters (based on your comment, see What is "Linting"?). It can either be a command-line tool (like yq), a website (like YAMLlint), an integrated tool on where you would be using your YAML file (like Gitlab's CI Lint Tool), an extension on your IDE/editor, or some language-specific library (like ruamel.yaml for Python as demonstrated by this answer on a related question).
On YAMLlint, it shows your YAML as valid, but it strips out the comments because it "give you a nice clean UTF-8 version of it." (also, as the YAML spec says, "Comments are a presentation detail" only and "must not be used to convey content information". Processors should just ignore them). The image does show the payload to the site included the comments, and the site responded with 200/OK:
I personally use the yq command line utility.
$ cat test.yaml
# profile fields
firstName: string
lastName: string
phoneNum: string # optional
active: boolean
notes: string # optional
role: string # admin | market | client
by: User_Id
$ yq '.role' test.yaml
string
$ cat invalid.yaml
key1: [value2]
key2: key3: key4: value3
$ yq '.key2' invalid.yaml
Error: bad file 'invalid.yaml': yaml: line 2: mapping values are not allowed in this context
This is very funny but very frustrating problem. I am using an ENV variable, which specifies date. I provide an ISO 8601 compliant version and in application, I retrieve it and parse. When I specify it in GH action workflow, it is get parsed as a date (rather than a string) and formatted. Therefore, my application parsing fails.
Example:
.github/workflows/rust.yaml
env:
MY_DATE: '2020-10-07T12:00:00+01:00'
run: echo $MY_DATE
Result (GH action UI):
env:
TMOU_GAME_END: 10/07/2020 11:00:00
10/07/2020 11:00:00
It is specific to GitHub action and their yaml parsing, it works OK on Heroku, on various local setups, etc.
Things I tried and they don't work:
using no quotes, single quotes ('), double quotes (")
setting another ENV var, LC_TIME to en_DK.UTF-8
using !!str shorthand (see https://yaml.org/spec/1.2/spec.html, section Example 2.23. Various Explicit Tags); this one fails either with The workflow is not valid. .github/workflows/rust.yml: Unexpected tag 'tag:yaml.org,2002:str' or with The workflow is not valid. .github/workflows/rust.yml: The scalar style 'DoubleQuoted | SingleQuoted' on line 29 and column 24 is not valid with the tag 'tag:yaml.org,2002:str'
Is there any help? Any secret parameter I can turn on? Any escaping sequence? I just wanna GH Actions yaml parser to treat the value as a string.
Surprisingly, it seems GitHub Actions workflow YAML parser does not fully implement the standard and using explicit typing (like !!str) does not work. You can, however, workaround it by setting the environment variable to the desired value not in the YAML file itself but dynamically during workflow execution using a dedicated workflow command:
steps:
- name: Dynamically set MY_DATE environment variable
run: echo "MY_DATE=2020-10-07T12:00:00+01:00" >> $GITHUB_ENV
- name: Test MY_DATE variable
run: echo ${{ env.MY_DATE }}
This should do the trick.
Following this tutorial to connect a local docker registry to the KIND cluster there's the below chunk of code in the bash script. I want to use my config file, but I don't know how the below chunk fits in (there's a lot of dashes and pipes in the syntax).
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:${reg_port}"]
endpoint = ["http://${reg_name}:${reg_port}"]
EOF
My config file:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
extraPortMappings:
- containerPort: 8080
hostPort: 80
protocol: TCP
- role: worker
- role: worker
- role: worker
- role: worker
In the shell fragment you show, everything between the first and last lines, including the dashes and pipes, is a valid YAML file; the only processing the shell does is replacing ${reg_name} and ${reg_port} with the values of the corresponding environment variables.
If you want to merge this with your existing kind config file, you should be able to just combine the top-level keys:
apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
nodes:
- role: control-plane
et: cetera
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:5000"]
endpoint = ["http://kind-registry:5000"]
If you had other containerdConfigPatches, the sequence of items starting with - on each line is a YAML list (like you have in nodes:) and you could add this patch to the end of the list. (This is a little unlikely since this option isn't documented in the kind Configuration documentation.)
Here documents in YAML are problematic anyway. Try using e.g. printf instead, maybe?
printf '%s\n' \
'kind: Cluster' \
'apiVersion: kind.x-k8s.io/v1alpha4' \
'containerdConfigPatches:' \
'- |-' \
' [plugins."io.containerd.grpc.v1.cri".registry.mirrors."localhost:${reg_port}"]' \
' endpoint = ["http://${reg_name}:${reg_port}"]' |
kind create cluster --config=-
Lucky your string doesn't contain any single quotes, so we can safely use those for wrapping. Also lucky your data doesn't contain any shell variable expansions or command substitutions, so we can use single (verbatim) quotes.
For the record, if you need to embed a literal single quote,
'you can'"'"'t get there from here'
produces the literal quoted string
you can't get there from here
(look closely; that's a single-quoted string, adjacent to a double-quoted literal single quote "'", adjacent to another single-quoted string) and if you need to expand variables or command substitutions, you will need to switch to double quotes around those strings. Example:
printf '%s\n' \
'literal $dollar sign in single quotes, the shell won'"'"'t touch it' \
"inside double quotes, $HOME expands to your home directory" \
'you can combine the two, like '"$(echo '"this"')"', too!'
I'm using the CI Lint tester to try and figure out how to store an expected JSON result, which I later compare to a curl response. Neither of these work:
Attempt 1
---
image: ruby:2.1
script:
- EXPECT_SERVER_OUTPUT='{"message": "Hello World"}'
Fails with:
did not find expected key while parsing a block mapping at line 4 column 5
Attempt 2
---
image: ruby:2.1
script:
- EXPECT_SERVER_OUTPUT="{\"message\": \"Hello World\"}"
Fails with:
jobs:script config should be a hash
I've tried using various combinations of echo as well, without a working solution.
You could use literal block scalar1 style notation and put the variable definition and subsequent script lines on separate lines2 without worrying about quoting:
myjob:
script:
- |
EXPECT_SERVER_OUTPUT='{"message": "Hello World"}'
or you can escape the nested double quotes:
myjob:
script:
- "EXPECT_SERVER_OUTPUT='{\"message\": \"Hello World\"}'"
but you may also want to just use variables like:
myjob:
variables:
EXPECT_SERVER_OUTPUT: '{"message": "Hello World"}'
script:
- dothething.sh
Note: variables are by default expanded inside variable definitions so take care with any $ characters inside the variable value (they must be written as $$ to be literal). This feature can also be turned off.
1See this answer for an explanation of this and related notation
2See this section of the GitLab docs for more info on multi-line commands
I made it work like this:
script: |
"EXPECT_SERVER_OUTPUT='{\"message\": \"Hello World\"}'"
echo $EXPECT_SERVER_OUTPUT