I have metrics like route/api_1_test/POST/time/200.avg where api_1_test is the route, POST is the method, time is the metric_name, 200 is the status_code and avg is the metric_type. I have arrays for route, method, status_code and metric_type. I would like to create a config file with all possible combinations of arrays with the following text
- name: finagle_route_<metric_name>
path: $.route/<route>/<method>/<metric_name>/<status_code>.<metric_type>
labels:
route: <route>
method: <method>
status: <status_code>
type: <metric_type>
How do I write a for loop for this?
#!/bin/bash
set -f
for route in $(<routes); do
for method in $(<methods); do
for name in $(<metric_names); do
for code in $(<http_status_codes); do
for type in $(<types); do
echo -e "- name: fingale_route_$name\n path: \$.route/$route/$method/$name/$code.$type\n labels:\n status:$code\n type:$type\n method:$method\n route:$route\n"
done
done
done
done
done
Related
I'm currently writing a bash script and struggling with something that looked fairly simple at first.
I'm trying to create a function that calls a kubectl (Kubernetes) command. The command is expecting the path to a file as an argument although I'd like to pass the content itself (multiline YAML text). It works in the shell but can't make it work in my function. I've tried many things and the latest looks like that (it's just a subset of the the YAML content):
#!/bin/bash
AGENT_NAME="default"
deploy_agent_statefulset() {
kubectl apply -n default -f - $(cat <<- END
kind: ConfigMap
metadata:
name: $AGENT_NAME
apiVersion: v1
data:
agent.yaml: |
metrics:
wal_directory: /var/lib/agent/wal
END
)
}
deploy_agent_statefulset
The initial command that works in the shell is the following.
cat <<'EOF' | NAMESPACE=default /bin/sh -c 'kubectl apply -n $NAMESPACE -f -'
kind: ConfigMap
...
I'm sure I m doing a lot of things wrong - keen to get some help
Thank you.
name: grafana-agent
In your function, you didn't contruct stdin properly :
#!/bin/bash
AGENT_NAME="default"
deploy_agent_statefulset() {
kubectl apply -n default -f - <<END
kind: ConfigMap
metadata:
name: $AGENT_NAME
apiVersion: v1
data:
agent.yaml: |
metrics:
wal_directory: /var/lib/agent/wal
END
}
deploy_agent_statefulset
this one should work:
#!/bin/bash
AGENT_NAME="default"
deploy_agent_statefulset() {
cat << EOF | kubectl apply -n default -f -
apiVersion: v1
kind: ConfigMap
metadata:
name: $AGENT_NAME
data:
agent.yaml: |
metrics:
wal_directory: /var/lib/agent/wal
EOF
}
deploy_agent_statefulset
To point out what is wrong in your yaml which are all indentations,
you don't need to add the indentations in the beginning
name goes under metadata, so it needs to be intended.
agent.yaml is the key, for the data in the ConfigMap, so it needs to be intended as well
i'm new to benthos, hope following configuration to work, i looked at the benthos doc and tried to google, but didn't find an answer, any answer is greatly appreciated
actually, the sign will be the calculated value, but now I'm stuck on the first step, i can't get the sign value to be successfully assigned to the header
input:
processors:
- bloblang: |
meta sign = "1233312312312312"
meta device_id = "31231312312"
http_client:
url: >-
https://test/${!meta("device_id")}
verb: GET
headers:
sign: ${!meta("sign")}
after #Mihai Todor helped, now i have new question.
this config below can work.(first)
input:
http_client:
url: >-
https://test/api
verb: GET
headers:
sign: "signcode"
but this one returned invalid signature error
input:
mapping: root = {}
generate:
count: 1
pipeline:
processors:
- http:
url: >-
https://test/api
verb: GET
headers:
sign: "signcode"
output:
stdout: {}
update(more detail screenshot)
first
second
Finally i got it work with the #Mihai helped.
the reason why i got 'signure is invaild' is because a space character in paramter headers->stringToSign, for reason i need add paramter 'stringTosign' and the value need to be empty, but somehow i copy an invisible space character in it, this will make benthos to add a Content-Length: 1 to the request header(i don't know why), this Content-Length: 1 cause my request always failed with error 'signure invaild'.
after i deleted the space character, all is ok.
Input processors operate on the messages returned by the input, so you can't set metadata that way. Also, metadata is associated with in-flight messages and doesn't persist across messages (use an in-memory cache if you need that).
One workaround is to combine a generate input with an http processor like so:
input:
generate:
mapping: root = ""
interval: 0s
count: 1
processors:
- mapping: |
meta sign = "1233312312312312"
meta device_id = "31231312312"
pipeline:
processors:
- http:
url: >-
https://test/${!meta("device_id")}
verb: GET
headers:
sign: ${!meta("sign")}
output:
stdout: {}
Note that the mapping processor (the replacement for the soon-to-be deprecated bloblang one) can also reside under pipeline.processors, and, if you just need to set those metadata fields, you can also do it inside the mapping field of the generate input (root = {} is implicit).
Update: Following the comments, I ran two configs and used nc to print the full HTTP request each of them make:
generate input with http processor:
input:
generate:
mapping: root = ""
interval: 0s
count: 1
processors:
- mapping: |
meta sign = "1233312312312312"
meta device_id = "31231312312"
pipeline:
processors:
- http:
url: >-
http://localhost:6666/${!meta("device_id")}
verb: GET
headers:
sign: ${!meta("sign")}
output:
stdout: {}
HTTP request dump:
> nc -l localhost 6666
GET /31231312312 HTTP/1.1
Host: localhost:6666
User-Agent: Go-http-client/1.1
Sign: 1233312312312312
Accept-Encoding: gzip
http_client input:
input:
http_client:
url: >-
http://localhost:6666/31231312312
verb: GET
headers:
sign: 1233312312312312
output:
stdout: {}
HTTP request dump:
> nc -l localhost 6666
GET /31231312312 HTTP/1.1
Host: localhost:6666
User-Agent: Go-http-client/1.1
Sign: 1233312312312312
Accept-Encoding: gzip
I used Benthos v4.5.1 on OSX and, for both configs, the request looks identical.
My best guess is that you're seeing a transitive issue on your end (some rate limiting perhaps).
Is it possible, or is there a way, to use a yaml anchor reference in a BASH if clause.
If so, how?
This is what I'm attempting so far.
create-cluster:
needs:
- terra-bootstrap
script:
- export TF_VAR_state_bucket_prefix="${TF_VAR_vsad}/${TF_VAR_cluster_name}"
- pushd terra-cluster
- *init_with_gcs_state
- |
if [[ "${CLUSTER_EXISTS}" == 'false' ]]; then
terraform apply -auto-approve
*does_cluster_exist
fi
- popd
stage: create-cluster
tags:
- gke
No, the YAML spec does not allow you to do this. YAML anchors cannot be used within a string (or any other scalar).
A better approach for GitLab CI YAML would be to define your reusable bash steps as functions then utilize those within your job script logic as-needed.
For example, you can define a function does_cluster_exist and reference it using anchors/reference/etc.
.helper_functions:
script: |
function cluster_exists() {
cluster="$1"
# ... complete the function logic
}
function another_function() {
return 0
}
another_job:
before_script:
# ensure functions are defined
- !reference [.helper_functions, script]
script:
# ...
# use the functions in an `if` or wherever...
- |
if cluster_exists "$cluster"; then
another_function
else
echo "fatal, cluster does not exist" > /dev/stderr
exit 1
fi
I'm having some issues with a Jenkins Job Builder YAML file which contains an attribute (message-content) with "{}" characters in it:
- job-template:
id: senderjob
name: '{job-prefix}{id}'
command: 'echo "executed with $PARAMETER"'
type: freestyle
properties:
- ownership:
enabled: true
owner: user1
- build-discarder:
num-to-keep: 100
parameters:
- string:
name: PARAMETER
default: 'param'
description: 'default parameter for message.'
# template settings
builders:
- shell: '{command}'
publishers:
- ci-publisher:
override-topic: VirtualTopic.abcd.message
message-type: 'Custom'
message-properties: |
release-name=$PARAMETER
PARAMETER=$PARAMETER
message-content: '{"release-name" : "1.0"}'
The error reported is:
jenkins_jobs.errors.JenkinsJobsException: release-name parameter missing to format {release-name : 1.0}
So it looks like it's trying to expand "release-name" with a parameter.
So I have added it as parameter:
- string:
name: release-name
default: '1.0'
description: 'default parameter for release.'
It still reports the same error. Should I include the parameter somewhere else or escape it ? I couldn't find any YAML escape for "{}" characters though.
Any idea?
You should add the following to the configuration file
[job_builder]
allow_empty_variables = True
Link: https://jenkins-job-builder.readthedocs.io/en/latest/definition.html#job-template-1
In a bash script in a Heat template, is it possible to use a parameter value from that template?
Yes, according to the Heat Orchestration Template specification, you can accomplish this with the str_replace function. They give an example that uses str_replace, together with get_param, to use a parameter value DBRootPassword in a bash script:
parameters:
DBRootPassword:
type: string
label: Database Password
description: Root password for MySQL
hidden: true
resources:
my_instance:
type: OS::Nova::Server
properties:
# general properties ...
user_data:
str_replace:
template: |
#!/bin/bash
echo "Hello world"
echo "Setting MySQL root password"
mysqladmin -u root password $db_rootpassword
# do more things ...
params:
$db_rootpassword: { get_param: DBRootPassword }
Each key in params is replaced in template with its value. Since $db_rootpassword's value is set to the result of get_param, that means the parameter is passed into the bash script wherever $db_rootpassword is used.