Consider the following hello.yaml:
foos:
- foo: foo1
bar: hello
- foo: foo2
bar: world
If I want to update the bar value where foo = "foo1", I can invoke the following command:
yq '( .foos[] | select(.foo == "foo1") | .bar) |= "goodbye cruel"' hello.yaml
And that correctly outputs:
foos:
- foo: foo1
bar: goodbye cruel
- foo: foo2
bar: world
However, if I do not know that I have an item that matches, I would like to insert the appropriate entries e.g. something like yq '( .foos[] | select(.foo == "foo3") | .bar) ... would output
foos:
- foo: foo1
bar: hello
- foo: foo2
bar: world
- foo: foo3
bar: goodbye cruel
Is there a way in yq to "upsert" the array, or do I have to evaluate if the key exists upfront and perform one of two commands to insert or update?
Many thanks
Like Inian said; there is no upsert operation (at the moment). This is how I would do it - not sure if there is a better way?
yq '
with(.foos ;
select( all_c(.foo != "foo3")) | . += {"foo": "foo3"}
) |
(.foos[] | select(.foo == "foo3") | .bar) = "cool"
' hello.yaml
Explanation:
In the with block, match arrays that don't have foo: foo3, and add it.
Next, find all the elements with foo: foo3 and update them.
Disclaimer: I wrote yq
Related
I have a simple yaml file called foo.yaml
foo:
- a
- c
bar:
- foo: bar
foo2: bar2
I'm trying to add a new value (b) to foo, in alphabetical order. I can add the value with +=, but it doesn't get alphabatized
$ yq '.foo += "b"' foo.yaml
foo:
- a
- c
- b
bar:
- foo: bar
foo2: bar2
If I use + I can use sort, but I only get the raw values. e.g.:
$ yq '.foo + "b" | sort()' foo.yaml
- a
- b
- c
I tried to set this into a bash variable and then use it with =, but it appears as a multi-line text
$ variable=$(yq '.foo + "b" | sort()' foo.yaml)
$ yq ".foo = \"$variable\"" foo.yaml
foo: |-
- a
- b
- c
bar:
- foo: bar
foo2: bar2
Is there an easier way to insert a new value into foo alphabetically, while keeping the rest of the yaml in tact?
The reason you are getting the raw values is that you've told yq to traverse into 'foo'. Instead try:
yq '.foo = (.foo + "b" | sort)' file.yaml
yields:
foo:
- a
- b
- c
bar:
- foo: bar
foo2: bar2
Explanation:
you need to update the entry in 'foo'
then, in brackets, set the new value. Normally you can use +=, but because you want to sort I've used '='
Disclaimer: I wrote yq
I have a file with a yaml data as below:
cat x.yml
foo:
- bar: 1
- zoo: 2
I am able to insert the text but this is messing the indentation(see 2nd line):
sed -r '/^[ ]+- bar:/a- hoo: 3' x.yml
foo:
- bar: 1
- hoo: 3
- zoo: 2
Then, I tried to backreference the leading spaces but seems like it is not working with /a flag.
sed -r '/^([ ]+)- bar:/a\1- hoo: 3' x.yml
foo:
- bar: 1
1- hoo: 3
- zoo: 2
Any help to get the following using a one-liner ?
foo:
- bar: 1
- hoo: 3
- zoo: 2
I suggest to switch to GNU sed's s command:
sed -E 's/( *)- bar:.*/&\n\1- hoo: 3/' file
Output:
foo:
- bar: 1
- hoo: 3
- zoo: 2
See: man sed and The Stack Overflow Regular Expressions FAQ
Best option is probably to use a parser. If you know exactly where the values should be, you can just pop them in there. Otherwise you'd have to loop and look for the "bar" key. This is using the YAML module.
use strict;
use warnings;
use YAML;
my $yaml = do { local $/; <> };
my $href = Load($yaml);
for (0 .. $#{ $href->{foo} }) {
if (grep { $_ eq "bar" } keys %{ $href->{foo}[$_] }) {
splice #{ $href->{foo} }, $_+1, 0, { hoo => 1 };
}
}
print Dump $href;
It outputs:
foo:
- bar: 1
- hoo: 1
- zoo: 2
Otherwise you can use Perl like so:
$ perl -pe's/^( *- *)bar.*\K/$1hoo: 1\n/s' x.yml
foo:
- bar: 1
- hoo: 1
- zoo: 2
Capture from beginning of line ^ a dash surrounded by dashes. Expect "bar", then absorb everything after it into the regex match, including the newline at the end (hence the /s modifier). Keep (\K) everything that was matched, and after it, add on the captured dash-string, plus your new content and a newline. Done.
First off, I agree with Inian saying a YML parser would be more appropriate here.
Nevertheless, you could use the s command and capture groups instead like
$ sed -r 's/^([ ]+)- bar:(.+)$/\1- bar:\2\n\1- hoo: 3/' x.yml
which gives
foo:
- bar: 1
- hoo: 3
- zoo: 2
Assume a text file that contains lines starting with foo and bar, respectively. Assume further that I would like to print only every fourth line of the ones starting with bar; the lines starting with foo should always be printed.
foo bar qux
# Deliberate empty line
bar baz 1
bar baz 2
bar baz 3
bar baz 4
bar baz 5
bar baz 6
bar baz 7
bar baz 8
# A miscellaneous code comment
The following code prints every fourth line irrespective of the first word and is thus not what I am looking for.
awk '/^bar/ NR == 1 || NR % 4 == 0' infile
What would the correct code be (preferentially with awk)?
EDIT:
Thanks to fedorqui for his excellent suggestion. Considering the potential appearance of empty lines and comments in the input file, I am using the following code:
user$ awk '!/^bar/ || (/^bar/ && !(++c%4))' file
foo bar qux
# Deliberate empty line
bar baz 4
bar baz 8
# A miscellaneous code comment
Just use a counter:
awk '/^foo/ || (/^bar/ && !(++c%4))' file
This prints lines that accomplish either of these:
start with "foo"
start with "bar" and this happens for the 4th time, 8th... That is, every four times a line starts with "bar".
See it in action:
$ cat a
foo1
bar1
bar2
bar3
foo2
foo3
bar4
bar5
bar6
bar7
bar8
bar9
$ awk '/^foo/ || (/^bar/ && !(++c%4))' a
foo1
foo2
foo3
bar4
bar8
This should do the trick:
awk '/^bar/ && NR % 4 == 0 || /^foo/' infile
Given we have two formatted strings that are unrelated to each other.
#test.rb
string_1 = "Title\nfoo bar\nbaz\nfoo bar baz boo"
string_2 = "Unrelated Title\ndog cat farm\nspace moon"
How can I use ruby or call shell commands to have each of these string display as columns in terminal? The key is that the data of each string are not building a correlated row, ie this is not a table, rather 2 lists side by side.
Title Unrelated Title
foo bar dog cat farm
baz space moon
foo bar baz boo
You can try using paste and column command together. Note that this is a shell command so spaces between the assignment operator should be corrected.
$ string_1="Title\nfoo bar\nbaz\nfoo bar baz boo"
$ string_2="Unrelated Title\ndog cat farm\nspace moon"
$ paste -d '|' <(echo -e "$string_1") <(echo -e "$string_2") | column -s'|' -t
Title Unrelated Title
foo bar dog cat farm
baz space moon
foo bar baz boo
We paste the lines with | as delimiter and tell column command to use | as a reference to form columns.
In Ruby, you could do it this way:
#!/usr/bin/env ruby
string_1 = "Title\nfoo bar\nbaz\nfoo bar baz boo"
string_2 = "Unrelated Title\ndog cat farm\nspace moon"
a1 = string_1.split("\n")
a2 = string_2.split("\n")
a1.zip(a2).each { |pair| puts "%-20s%s" % [pair.first, pair.last] }
# or
# a1.zip(a2).each { |left, right| puts "%-20s%s" % [left, right] }
This produces:
Title Unrelated Title
foo bar dog cat farm
baz space moon
foo bar baz boo
Hi , If you Use temp files
string_1 = "Title\nfoo bar\nbaz\nfoo bar baz boo"
string_2 = "Unrelated Title\ndog cat farm\nspace moon"
echo -e $string_1 >a.txt
echo -e $string_2 >b.txt
paste a.txt b.txt
I hope it will help.
I'm wondering if it's possible to make autocompletion span an = sign. So, for example, I want to type foo BAR=[TAB][TAB], and have it fill in the possible values for BAR. I've tried the following: I have a file called 'bar', as follows:
#!/bin/bash
echo -e "BAR=100\nBAR=110\nBAR=200" | grep "^$2"
And then I do:
~> complete -C bar foo
If I type foo [TAB][TAB], it gives me some possible values for BAR. If, I type foo BAR=[TAB][TAB], it fails (it appends BAR=BAR= to the end of the command). (note, if I type bar 1 BAR=, it gives me a proper list of completions, so this is not an issue with the bar script).
This would be very useful for some scripts I have.
Create a function (in your .bashrc e.g.):
bar()
{
local POS=${COMP_WORDS[COMP_CWORD]}
if [ "${COMP_WORDS[1]}" = "BAR" ] && [ $COMP_CWORD -eq 3 ]; then
COMPREPLY=($(echo -e "100\n110\n200" | grep ^$POS ))
fi
}
and link function to command foo:
complete -F bar foo