I have this structure:
foo:
image: 123
bar:
image: 456
baz:
config: "my config"
and I'd like to print the root keys (i.e. foo, bar, baz) based on the existence of the child "image"
In yq version 3 I could do this:
$ yq read test.yaml --printMode p "*.image" | awk -F'.' '{print $1}'
foo
bar
But I can't find the equivalent in v4. The yq + jq solution would be:
$ yq -j e test.yaml | jq -r 'to_entries[] | select(.value | has("image")) | [.key][]'
foo
bar
Any idea how to do this with yq v4?
You can use the path operator to get the path of the matching object containing the tag image
yq e '.[] | select(has("image")) | path | .[]' yaml
Related
I have a yaml file called teams.yml that looks like this:
test:
bonjour: hello
loop:
bonjour: allo
I want to loop over an array and get itsvalues matching a key in the yaml. I have tried yq e 'keys | .[] | select(. == "test")' .github/teams.yml and it returns test whereas yq e 'keys | .[] | select(. == "abc")' .github/teams.yml returns nothing which is enough to get the information I am interested in.
The issue is that, when using the same logic in a for loop, yq returns nothing:
#!/bin/bash
yq e 'keys | .[] | select(. == "abc")' .github/teams.yml # Prints nothing
yq e 'keys | .[] | select(. == "test")' .github/teams.yml # Prints 'test'
ar=( "abc" "test" "xyz" )
for i in "${ar[#]}" # Prints nothing
do
yq e 'keys | .[] | select(. == "$i")' .github/teams.yml
done
What explains the lack of output in the for loop?
Call yq with the loop variable set in the environment, and use env within yq to read environment variables. See also the section Env Variable Operators in the manual.
ar=( "abc" "test" "xyz" )
for i in "${ar[#]}" # Prints nothing
do
i="$i" yq e 'keys | .[] | select(. == env(i))' .github/teams.yml
done
Another way could be to import the (preformatted) array from the environment and just make the array subtractions within yq (saving you the looping and calling yq multiple times):
ar='["abc","test","xyz"]' yq e 'env(ar) - (env(ar) - keys) | .[]' .github/teams.yml
Given the following yaml:
charts:
# repository with Helm charts for creation namespaces
path: ns
pathMonitoringPrometheus: prom
namespaces:
first:
description: "Description of first"
enabled: false
branch: master
bootstrapChart: bootstrap
syncAccessGroups: []
namespace:
role: k8s-role-of-first
istio: disabled
public: view
sources: []
second:
description: "Description of second"
enabled: false
branch: HEAD
bootstrapChart: bootstrap
namespace:
role: k8s-role-of-second
istio: 1-13-2
labels:
label: second
sources:
- http://url.of.second
How could we get a list of namespaces and their istio value if it is different to "disabled".
We are trying to use "yq" tool, but I guess any approach would be ok, although "yq" would be a preferred approach.
second, 1-13-2
Using kislyuk/yq you can base your filter on jq.
to_entries splits up the object into an array of key-value pairs
select selects those items matching your criteria
String interpolation in combination with the -r option puts together your desired output
yq -r '
.namespaces
| to_entries[]
| select(.value.namespace.istio != "disabled")
| "\(.key), \(.value.namespace.istio)"
'
second, 1-13-2
Using mikefarah/yq the filter is quite similar.
to_entries[] has to be split up to_entries | .[]
String interpolation is replaced using join and an array
yq '
.namespaces
| to_entries | .[]
| select(.value.namespace.istio != "disabled")
| [.key, .value.namespace.istio] | join(", ")
'
second, 1-13-2
this will do:
cat /path/tp/your.yaml |yq -r '.namespaces | to_entries[] | "\(.key) \(.value.namespace.istio)"'`
will result:
first disabled
second 1-13-2
I'm trying to determine the correct syntax for using yq to print all key/value pairs from a given yaml input using yq v4 - with the desired output having the full key "path". This was possible using v3 such as this:
$ cat << EOF | yq r -p pv - '**'
> a:
> b: foo
> c: bar
> EOF
a.b: foo
a.c: bar
but I'm having difficulty wrapping my head around the new syntax.
Any help is greatly appreciated.
$ cat << EOF | yq e '.. | select(. == "*") | {(path | join(".")): .} ' -
> a:
> b: foo
> c: bar
> EOF
a.b: foo
a.c: bar
What does this do? Let's go over it:
.. recursively select all values
select(. == "*") filter for scalar values (i.e. filter out the value of a)
(path | join(".")) gets the path as array and joins the elements with .
{…: .} create a mapping, having the joined paths as keys and their values as values
Edit: to get sequence indexes in square brackets ([0] etc), do
$ cat << EOF | yq e '.. | select(. == "*") | {(path | . as $x | (.[] | select((. | tag) == "!!int") |= (["[", ., "]"] | join(""))) | $x | join(".") | sub(".\[", "[")): .} ' -
This seems like there should be a simpler way to do it, but I don't know yq well enough to figure it out.
I am getting list of values as below using the curl command:
curl -s http://internal.registry.com/v2/_catalog | jq -r '.repositories[0:5] | to_entries | map( .value )[]'
Output:
centos
containersol/consul-server
containersol/mesos-agent
containersol/mesos-master
cybs/address-api
I want to make sure that output should not have the prefix cybs/ in it. for example, cybs/address-api should just be address-api
Just use sub:
curl ... | jq -r '.repositories[0:5][] | sub("^cybs/"; "")'
Also note that to_entries | map( .value ) is a NOP and should be removed.
Output:
centos
containersol/consul-server
containersol/mesos-agent
containersol/mesos-master
address-api
I have plenty of bash scripts with various files being piped into various scripts and it does my head in a bit.
I wondered if there was a way of visualising the pipeline in a bash script so I can easily see the flow.
You can add newlines after the pipe, and bash will continue to see it as a single pipeline:
foo | bar | baz | qux
can be written as
foo |
bar |
baz |
qux
Or, use line continuations, if the look appeals more:
foo \
| bar \
| baz \
| qux
Newlines are acceptable after |, && and ||
Try pv. It shows you how much data is being transferred in the pipe. Example usage (I got this image from the website):
pv prints its progress bars to stderr. In your case, you could use:
foo | pv -cN foo |
bar | pv -cN bar |
baz | pv -cN baz |
qux | pv -cN qux