I have this yaml file:
id:
providerName: admin
apiName: LagerAPI
version: 1.0.0
uuid: 1297c3e5-24aa-4806-be05-0cc3f2decaff
type: HTTP
context: /wso/1.0.0
contextTemplate: /wso/{version}
tags: []
documents: []
lastUpdated: Apr 14, 2020 4:33:37 PM
and wanna change it the value wso in lines context: and contextTemplate: to test, that this will be the result:
id:
providerName: admin
apiName: LagerAPI
version: 1.0.0
uuid: 1297c3e5-24aa-4806-be05-0cc3f2decaff
type: HTTP
context: /test/1.0.0
contextTemplate: /test/{version}
tags: []
documents: []
lastUpdated: Apr 14, 2020 4:33:37 PM
How do I do this with a bash command?
Thank you :)
I found a solution:
sed -i 's/wso/test/' file.yaml
Just in case there are more wso in that file this should replace what was intended.
sed '/^context/s/wso/test/;/^contextTemplate/s/wso/test/' file.yml
Related
I want to build a pipeline function that replaces a value in a yaml file. For that I want to make both the
pattern and the replacement value variable. I have seen the env-variables-operators article in the yq docs, however I cannot find the relevant section.
I have a yaml file with the following content:
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "1.0.0"
I now want to build a pipeline function that will replace the value of the value key in the yaml.
I can do so with:
$ yq '.spec.source.helm.parameters[0].value = "2.0.0"' myyaml.yml
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "2.0.0"
Now I want to make this command customizable.
What works:
$ VALUE=3.0.0
$ replacement=$VALUE yq '.spec.source.helm.parameters[0].value = env(replacement)' myyaml.yml
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "3.0.0"
What doesn't work
$ VALUE=3.0.0
$ PATTERN=.spec.source.helm.parameters[0].value
$ replacement=$VALUE pattern=$PATTERN yq 'env(pattern) = env(replacement)'
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "1.0.0"
I have also tried to use strenv and wrapping the replacement pattern in quotes, but it is not working.
Can anyone help me with the correct syntax?
You can import data with env but not code. You could inject it (note the changes in the quoting), but this is bad practice as it makes your script very vulnerable:
VALUE='3.0.0'
PATTERN='.spec.source.helm.parameters[0].value'
replacement="$VALUE" yq "${PATTERN} = env(replacement)" myyaml.yml
---
spec:
source:
helm:
parameters:
- name: "image.tag"
value: "3.0.0"
Better practice would be to import the path in a form that is interpretable by yq, e.g. as an array and using setpath:
VALUE='3.0.0'
PATTERN='["spec","source","helm","parameters",0,"value"]'
replacement="$VALUE" pattern="$PATTERN" yq 'setpath(env(pattern); env(replacement))' myyaml.yml
When I start a linux server with Cloud-init, I have a few scripts in /etc/cloud/cloud.cfg.d/ and they run in reverse alphabetical order
# ll /etc/cloud/cloud.cfg.d/
total 28
-rw-r--r-- 1 root root 173 Dec 10 12:38 00-cloudinit-lifecycle-hook.cfg
-rw-r--r-- 1 root root 2120 Jun 1 2021 05_logging.cfg
-rw-r--r-- 1 root root 590 Oct 26 17:55 10_aws_yumvars.cfg
-rw-r--r-- 1 root root 29 Dec 1 18:22 20_amazonlinux_repo_https.cfg
-rw-r--r-- 1 root root 586 Dec 10 12:38 50-cloudinit-tomcat.cfg
-rw-r--r-- 1 root root 585 Dec 10 12:40 60-cloudinit-newrelic.cfg
The last to execute is 00-cloudinit-lifecycle-hook.cfg, in which I complete the lifecycle for the Auto Scaling Group with a CONTINUE. The ASG fails if it doesn't receive this signal after a given time out.
The issue is that even if there's an error in 50-cloudinit-tomcat.cfg, it still runs 00-cloudinit-lifecycle-hook.cfg instead of stopping
How can I ensure cloud-init stops and never reaches the last script? I would like the ASG to never receive the CONTINUE signal if there's any error.
Here are the files:
EC2 instance user-data:
#cloud-config
bootcmd:
- [cloud-init-per, once, "app-volume", mkfs, -t, "ext4", "/dev/nvme1n1"]
mounts:
- ["/dev/nvme1n1", "/app-volume", "ext4", "defaults,nofail", "0", "0"]
merge_how:
- name: list
settings: [append]
- name: dict
settings: [no_replace, recurse_list]
50-cloudinit-tomcat.cfg
#cloud-config
merge_how:
- name: list
settings: [append]
- name: dict
settings: [no_replace, recurse_list]
runcmd:
- "#!/bin/bash -e"
- set +x
- echo ' '
- echo '# ===================================='
- echo '# Tomcat Cloud Init '
- echo '# /etc/cloud/cloud.cfg.d/'
- echo '# ===================================='
- echo ' '
- echo '#===================================='
- echo '# Run Ansible'
- echo '#===================================='
- echo ' '
- set -x
- ansible-playbook /opt/init-config/tomcat/tomcat-config.yaml
when I run ansible-playbook /opt/init-config/tomcat/tomcat-config.yaml directly in the instance I get an error, and I know it returns 2
ansible-playbook /opt/init-config/tomcat/tomcat-config.yaml #shows errors
echo $? # shows "2"
00-cloudinit-lifecycle-hook.cfg
#cloud-config
merge_how:
- name: list
settings: [append]
- name: dict
settings: [no_replace, recurse_list]
runcmd:
- "/opt/lifecycles/lifecycle-hook-continue.sh"
An alternative I can think of, is to send a ABANDON signal instead of CONTINUE as soon as there's en error in one of the cloud-init config. But I can't find in the documentation on to define if there's an error
I'm having some issues with a Jenkins Job Builder YAML file which contains an attribute (message-content) with "{}" characters in it:
- job-template:
id: senderjob
name: '{job-prefix}{id}'
command: 'echo "executed with $PARAMETER"'
type: freestyle
properties:
- ownership:
enabled: true
owner: user1
- build-discarder:
num-to-keep: 100
parameters:
- string:
name: PARAMETER
default: 'param'
description: 'default parameter for message.'
# template settings
builders:
- shell: '{command}'
publishers:
- ci-publisher:
override-topic: VirtualTopic.abcd.message
message-type: 'Custom'
message-properties: |
release-name=$PARAMETER
PARAMETER=$PARAMETER
message-content: '{"release-name" : "1.0"}'
The error reported is:
jenkins_jobs.errors.JenkinsJobsException: release-name parameter missing to format {release-name : 1.0}
So it looks like it's trying to expand "release-name" with a parameter.
So I have added it as parameter:
- string:
name: release-name
default: '1.0'
description: 'default parameter for release.'
It still reports the same error. Should I include the parameter somewhere else or escape it ? I couldn't find any YAML escape for "{}" characters though.
Any idea?
You should add the following to the configuration file
[job_builder]
allow_empty_variables = True
Link: https://jenkins-job-builder.readthedocs.io/en/latest/definition.html#job-template-1
I use Stat command to to find the last modification date of a file and result is:
File: ‘first.php’
Size: 145 Blocks: 8 IO Block: 4096 regular file
Device: 801h/2049d Inode: 1771067 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1000/ fn) Gid: ( 1000/ fn)
Access: 2017-02-28 12:44:32.853785914 +0330
Modify: 2017-02-27 23:22:11.280625946 +0330
Change: 2017-02-27 23:22:11.344625946 +0330
Birth: -
What command should I use to show just first.php 2017-02-27 23:22:11.280625946 +0330 in output?
Try reading the manual page:
man stat
And then use the format option:
stat -c '%n %y'
I'm trying to push logs to elasticSearch using fileBeat ( No Logstash )
I want to send following log in single message but it get broken into multiple messages, every line becomes separate message
20161014 17:49:09.169 [ERROR] [Thread-2974] some.java.class.:70 - some.java.Exception: write failed. History: [requestHost=123-some.org.com, time=Fri Oct 14 17:49:05 GMT-07:00 2016, exception=java.net.SocketTimeoutException]
[requestHost=123-some.org.com, time=Fri Oct 14 17:49:07 GMT-07:00 2016, exception=java.net.SocketTimeoutException]
[requestHost=123-some.org.com, time=Fri Oct 14 17:49:09 GMT-07:00 2016, exception=java.net.SocketTimeoutException]
Tried 3 times
at java.lang.Thread.run(Thread.java:745)
20161014 17:49:09.169 [ERROR] [Thread-3022]
I want to merge all lines between 2 dates (1st and last line)
Here is my filebeat.yml snippet
paths:
- /test.log
multiline.pattern: '^\[0-9]{8}'
multiline.negate: true
multiline.match: after
I need to know correct regex
I'm trying to solve this without using logstash
Using the following Filebeat configuration with the provided log sample produces two events where each message begins with the date.
I ran ./filebeat -c filebeat.yml -e -v -d "*" with the config below to test. I also tested the pattern on the Go playground.
filebeat.yml:
filebeat:
prospectors:
- paths: ["input.txt"]
multiline:
pattern: '^[0-9]{8}'
negate: true
match: after
output:
console:
pretty: false
Output:
{
"#timestamp": "2016-10-17T14:13:31.292Z",
"beat": {
"hostname": "host.example.com",
"name": "host.example.com",
},
"input_type": "log",
"message": "20161014 17:49:09.169 [ERROR] [Thread-2974] some.java.class.:70 - some.java.Exception: write failed. History: [requestHost=123-some.org.com, time=Fri Oct 14 17:49:05 GMT-07:00 2016, exception=java.net.SocketTimeoutException]\n[requestHost=123-some.org.com, time=Fri Oct 14 17:49:07 GMT-07:00 2016, exception=java.net.SocketTimeoutException]\n[requestHost=123-some.org.com, time=Fri Oct 14 17:49:09 GMT-07:00 2016, exception=java.net.SocketTimeoutException]\n Tried 3 times\n at java.lang.Thread.run(Thread.java:745)",
"offset": 519,
"source": "input.txt",
"type": "log"
}
{
"#timestamp": "2016-10-17T14:17:21.686Z",
"beat": {
"hostname": "host.example.com",
"name": "host.example.com",
},
"input_type": "log",
"message": "20161014 17:49:09.169 [ERROR] [Thread-3022]",
"offset": 563,
"source": "input.txt",
"type": "log"
}