I have the following yaml file called file.yaml
# This is a file store
- file-store:
version: 2
enabled: no
- file-store2:
version: 4
enabled: yes
Is there a way to enable the file-store option for the first group?. I tried to use this one but it doesn't work:
sed -i '/^ *file-store:/,/^ *[^:]*:/s/enabled: no/enabled: yes/' file.yaml
Expected output:
# This is a file store
- file-store:
version: 2
enabled: yes
- file-store2:
version: 4
enabled: yes
In addition to the recommendations given by #HatLess you can try to use this pure sed solution. I know, it's too wordy but it's more reliable even though not 100%.
# Capture the block between "- file-store:" and "- file store.*:"
/^[ ]*- file-store:/,/^[ ]*- file-store[^:]*:/ {
# Look for te string having "enabled:"
# and replace the last non-whitespace sequence with "yes"
/enabled:/ s/[^ ][^ ]*$/yes/
}
and the short inline version of the script:
sed '/^[ ]*- file-store:/,/^[ ]*- file-store[^:]*:/ { /enabled:/ s/[^ ][^ ]*$/yes/ }'
You can use yq
$ cat file
file1:
version: 2
enabled: no
file2:
version: 4
enabled: yes
To change the value of enabled, use:
$ yq -e '.file1.enabled = "yes"' file
{
"file1": {
"version": 2,
"enabled": "yes"
},
"file2": {
"version": 4,
"enabled": true
}
}
To save in place, add -yi flag
As your question includes sed, here is a fragile solution
$ sed '/file1/ {n;N;s/no/yes/}' file
file1:
version: 2
enabled: yes
file2:
version: 4
enabled: yes
Related
I have a big yaml file containing multiple declaration blocks, related to different services.
The structure is similar to the following (but repeated for multiple applications):
- name: commerce-api
type: helm
version: 0.0.5
I would like to find the block of code that is containing commerce-api and replace the version property value with something else.
The thing is, I wrote this script:
bumpConfig() {
LINE=$(awk "/- name: $1$/{print NR + $2}" "$CONFIG_YML")
sed -i "" -E "${LINE}s/version: $3.*$/\version: $4/" "$CONFIG_YML"
}
bumpConfig "commerce-api" 2 "$OLD_APP_VERSION" "$NEW_APP_VERSION"
Which is kind of allowing me to do what I want, but the only problem is that, the property version is not always on the third line.
How can I make my script to look for the first occurrence of version given the service name to be commerce-api?
Is this even possible using awk?
Adding some variation to the input file:
$ cat config.yml
- name: commerce-api-skip
type: helm
version: 0.0.5
- name: commerce-api
type: helm
bogus line1: bogus value1
version: 0.0.5
bogus line2: bogus value2
- name: commerce-api-skip-too
type: helm
version: 0.0.5
One awk idea:
bumpConfig() {
awk -v name="$1" -v old="$2" -v new="$3" '
/- name: / { replace=0
if ($NF == name)
replace=1
}
replace && $1=="version:" { if ($NF == old)
$0=substr($0,1,index($0,old)-1) new
}
1
' "${CONFIG_YML}"
}
Taking for a test drive:
CONFIG_YML='config.yml'
name='commerce-api'
OLD_APP_VERSION='0.0.5'
NEW_APP_VERSION='0.0.7'
bumpConfig "${name}" "${OLD_APP_VERSION}" "${NEW_APP_VERSION}"
This generates:
- name: commerce-api-skip
type: helm
version: 0.0.5
- name: commerce-api
type: helm
bogus line1: bogus value1
version: 0.0.7
bogus line2: bogus value2
- name: commerce-api-skip-too
type: helm
version: 0.0.5
Once OP is satisfied with the result:
if running GNU awk the file can be updated 'in place' via: awk -i inplace -v name="$1" ...
otherwise the output can be saved to a temp file and then copy the temp file over the original: awk -v name="$1" ... > tmpfile; mv tmpfile "${CONFIG_YML}"
Entirely in sed
sed -i '' "s/^version: $3/version: $4/' "$CONFIG_YML"
/^- name: $1\$/,/^- name:/ restricts the s command to just the lines between the requested name and the next - name: line.
#!/bin/bash
OLD_APP_VERSION=0.0.5
NEW_APP_VERSION=0.0.7
CONFIG_YML=config.yml
bumpConfig() {
gawk -i inplace -v name="$1" -v old="$2" -v new="$3" '
1
/^- name: / && $3 == name {
while (getline > 0) {
if (/^ version: / && $2 == old)
$0 = " version: " new
print
if (!NF || /^-/ || /^ version: /)
break
}
}
' "${CONFIG_YML}"
}
bumpConfig commerce-api "${OLD_APP_VERSION}" "${NEW_APP_VERSION}"
im trying to edit big textfiles with sed in one command
what i try to do is to execute sed and replace a certain string with the same string and added newlines+strings, so i can add text.
example command:
sed -i "s/openssl_verify_mode: 'none'/openssl_verify_mode: 'none'\n& domain: 'test.net'\n& user_name: 'admin'\n& password: 'mypassword'/g" /var/www/test-pro/config/configuration.yml
i use \n& to get the newlines. what am i doing wrong/what do i have to write?
sed has an append command that you can you use to add text after a match:
sed -e "/openssl_verify_mode: 'none'/a\\
openssl_verify_mode: domain: 'test.net'\\
openssl_verify_mode: user_name: 'admin'\\
openssl_verify_mode: password: 'mypassword'\\
" input-file
But it would probably be much cleaner to add the new text to a file and use the read command:
$ cat > new-text << EOF
openssl_verify_mode: domain: 'test.net'
openssl_verify_mode: user_name: 'admin'
openssl_verify_mode: password: 'mypassword'
EOF
$ sed -e "/openssl_verify_mode: 'none'/rnew-text" input
If you want to parameterize the string "openssl_verify_mode" (ie, if you want to use a different string), do it at the shell level. eg:
$ s="openssl_verify_mode"
$ sed -e "/${s}: 'none'/a\\
$s: domain: 'test.net'\\
$s: user_name: 'admin'\\
$s: password: 'mypassword'\\
" input
I strongly recommend against -i, and have omitted it from the answer. Feel free to abuse it if you like.
I am finding a bash command for a conditional replacement with offset. The existing posts that I've found are conditional replacement without offset or with a fixed offset.
Task: If uid contains 8964, then insert the line FORBIDDEN before DOB.
Each TXT file below represents one user, and it contains (in the following order)
some property(ies)
unique uid
some quality(ies)
unique DOB
a random lorem ipsum
I hope I can transform the following files
# file1.txt (uid doens't match 8964)
admin: false
uid: 123456
happy
movie
DOB: 6543-02-10
lorem ipsum
seo varis lireccuni paccem noba sako
# file2.txt (uid matches 8964)
citizen: true
hasSEAcct: true
uid: 289641
joyful hearty
final debug Juno XYus
magazine
DOB: 1234-05-06
saadi torem lopez dupont
into
# file1.txt (uid doens't match 8964)
admin: false
uid: 123456
happy
movie
DOB: 6543-02-10
lorem ipsum
seo varis lireccuni paccem noba sako
# file2.txt (uid matches 8964)
citizen: true
hasSEAcct: true
uid: 289641
joyful hearty
final debug Juno XYus
magazine
FORBIDDEN
DOB: 1234-05-06
saadi torem lopez dupont
My try:
If uid contains 8964, then do a 2nd match with DOB, and insert FORBIDDEN above DOB.
sed '/^uid: [0-9]*8964[0-9]*$/{n;/^DOB: .*$/{iFORBIDDEN}}' file*.txt
This gives me an unmatched { error.
sed: -e expression #1, char 0: unmatched `{'
I know that sed '/PAT/{n;p}' will execute {n;p} if PAT is matched, but it seems impossible to put /PAT2/{iTEXT} inside /PAT/{ }.
How can I perform such FORBIDDEN insertion?
$ awk '
/^uid/ && /8964/ {f=1} #1
/^DOB/ && f {print "FORBIDDEN"; f=0} #2
1 #3
' file
If a line starting with "uid" matches "8964", set flag
If a line starts with "DOB" and flag is set, print string and unset flag
print every line
$ awk -v RS='' '/uid: [0-9]*8964/{sub(/DOB/, "FORBIDDEN\nDOB")} 1' file
Alternatively, treat every block separated by a blank line as a single record, then sub in "FORBIDDEN\nDOB" if there's a match. I think the first one's better practice. As a very general rule, once you start thinking in terms of fields/records, it's time for awk/perl.
In my opinion, this is a good use-case for sed.
Here is a GNU sed solution with some explanation:
# script.sed
/^uid:.*8964/,/DOB/ { # Search only inside this range, if it exists.
/DOB/i FORBIDDEN # Insert FORBIDDEN before the line matching /DOB/.
}
Testing:
▶ gsed -f script.sed FILE2
citizen: true
hasSEAcct: true
uid: 289641
joyful hearty
final debug Juno XYus
magazine
FORBIDDEN
DOB: 1234-05-06
saadi torem lopez dupont
▶ gsed -f script.sed FILE1
admin: false
uid: 123456
happy
movie
DOB: 6543-02-10
lorem ipsum
seo varis lireccuni paccem noba sako
Or on one line:
▶ gsed -e '/^uid:.*8964/,/DOB/{/DOB/i FORBIDDEN' -e '}' FILE*
tried on gnu sed
sed -Ee '/^uid:\s*\w*8964\w*$/{n;/^DOB:/iFORBIDDEN' -e '}' file*.txt
Suppose I have two similar files:
a.yaml
data:
- name: a1
args: ["cmd", "something"]
config:
- name: some
val: thing
- name: a2
args: ["cmd2", "else"]
[...other array values...]
tags: ["something-in-a"]
values: ["else-in-a"]
substitutions:
key1: a-value
key2: a-value
key3: a-value
b.yaml
data:
- name: b1
args: ["cmd", "something"]
config:
- name: some
val: thing
- name: b2
args: ["cmd2", "else"]
[...other array values...]
tags: ["something-in-b"]
values: ["else-in-b"]
substitutions:
key1: b-value
key2: b-value
key3: b-value
My goal is to combine parts of a and b file such that I have a new file which consists of file content before substitutions: from b.yaml and content including and after substitutions: from a.yaml
So in this case, my desired output would be like this:
c.yaml
data:
- name: b1
args: ["cmd", "something"]
config:
- name: some
val: thing
- name: b2
args: ["cmd2", "else"]
[...other array values...]
tags: ["something-in-b"]
values: ["else-in-b"]
substitutions:
key1: a-value
key2: a-value
key3: a-value
The parts before and after substitutions: in both file contents might have different lengths.
Currently, my method is like this:
head -q -n `awk '/substitution/ {print FNR-1}' b.yaml` b.yaml >! c.yaml ; \
tail -q -n `awk '/substitution/ {ROWNUM=FNR} END {print NR-ROWNUM+1}' a.yaml` a.yaml >> c.yaml; \
rm a.yaml b.yaml; mv c.yaml a.yaml; # optional newfile renaming to original
But I wonder if there's an alternative or better method for combining parts of different files based on a common text string in bash?
Use awk, you just need to flag the flow based on the string:
awk '$1 == "substitutions:"{skip = FNR==NR ? 1:0}!skip' b.yaml a.yaml
Explaination:
FNR==NR: if true, process lines in the first file b.yaml, otherwise the 2nd file a.yaml
!skip: if TRUE, print the line, otherwise skip the line.
{
head -B9999 'substitutions:' a.yaml | head -n -1
head -A9999 'substitutions:' b.yaml
} > c.yaml
A oneliner:
{ head -B9999 'substitutions:' a.yaml | head -n -1; head -A9999 'substitutions:' b.yaml; } > c.yaml
The -A9999 and -B9999 are a bit dirty, here's a solution with sed's:
{
sed '/substitutions:/,$d' a.yaml
echo substitutions:
sed '1,/substitutions:/d' b.yaml
} > c.yaml
I have file like test.yaml file, the text content in the file like below.
servers:
- uri: "http://demo.nginx1.com/status"
name: "NGinX Monitor1"
- uri: "http://demo.nginx2.com/status"
name: "NGinX Monitor2"
I want to remove - uri line and immediate next line (start with name:) where host name = demo.nginx1.com.
I want out put like below.
servers:
- uri: "http://demo.nginx2.com/status"
name: "NGinX Monitor2"
I tied like below..
cat test.yaml | grep -v demo.nginx1.com | grep -v Monitor1 >> test_back.yaml
mv test_back.yaml test.yaml
I am getting expected out put. But it's re creating the file and I don't want to re create the file
Please help me with suitable command that i can use..
Just a simple logic using GNU sed
sed '/demo.nginx1.com/,+1 d' test.yaml
servers:
- uri: "http://demo.nginx2.com/status"
name: "NGinX Monitor2"
For in-place replacement, add a -i flag as -i.bak
sed -i.bak '/demo.nginx1.com/,+1 d' test.yaml
To see the in-place replacement:-
cat test.yaml
servers:
- uri: "http://demo.nginx2.com/status"
name: "NGinX Monitor2"
As I dislike using regular expressions to hack something you can parse - here's how I'd tackle it, using perl and the YAML module:
#!/usr/bin/env perl
use strict;
use warnings;
use YAML;
use Data::Dumper;
#load yaml by reading files specified as args to stdin,
#or piped in. (Just like how you'd use 'sed')
my $config = Load ( do { local $/ ; <>} );
#output data structure for debug
print Dumper $config;
#use grep to filter the uri you don't want.
#{$config -> {servers}} = grep { not $_ -> {uri} =~ m/demo.nginx2/ } #{$config -> {servers}};
#resultant data structure
print Dumper $config;
#output YAML to STDOUT
print Dump $config;