Any example to set from and to path in $HOME/.dlv/config.yml ? I have tried setting:
case 1 ( with quotes in from and to)
substitute-path:
{"from":
"/private/var/tmp/_bazel/d3eb9a0ef06857aebc54b41ff425d2ee"
"to": "/Users/xxx/code/src"}
case 2: ( without quotes in from and to)
substitute-path:
{from:
"/private/var/tmp/_bazel/d3eb9a0ef06857aebc54b41ff425d2ee"
to: "/Users/xxx/code/src"}
case 2: ( with hyphan before '{from' )
substitute-path:
-{from:
"/private/var/tmp/_bazel/d3eb9a0ef06857aebc54b41ff425d2ee"
to: "/Users/xxx/code/src"}
case 2: ( with hyphan before '{from' )
substitute-path:
-{"from":
"/private/var/tmp/_bazel/d3eb9a0ef06857aebc54b41ff425d2ee"
"to": "/Users/xxx/code/src"}
all 4 cases fails with config. error.
any working sample to set the path?
I think you are missing a space. The quotes are relevant only if the values or keys have spaces in them, or some other non-printable, non-ascii characters.
The config here, without comments:
$ cat ~/.config/dlv/config.yml | sed '/^#/d; /^$/d'
aliases:
# command: ["alias1", "alias2"]
substitute-path:
- {from: /my/source/code/was/here, to: /but/now/its/here}
debug-info-directories: ["/usr/lib/debug/.build-id"]
Seems like valid yaml:
$ yq < ~/.config/dlv/config.yml
{
"aliases": null,
"substitute-path": [
{
"from": "/my/source/code/was/here",
"to": "/but/now/its/here"
}
],
"debug-info-directories": [
"/usr/lib/debug/.build-id"
]
}
The yq tool is a wrapper for jq.
$ yq --help | sed 8q
usage: yq [options] <jq filter> [input file...]
[jq_filter] [files [files ...]]
yq: Command-line YAML processor - jq wrapper for YAML documents
yq transcodes YAML documents to JSON and passes them to jq.
See https://github.com/kislyuk/yq for more information.
Related
I realize this is a simple question but I haven't been able to find the answer. Thank you to anyone who may be able to help me understand what I am doing wrong.
Goal: Search and replace a string in a specific key in a JSON file with a string in a Bash variable using jq.
For example, in the following JSON file:
"name": "Welcome - https://docs.mysite.com/",
would become
"name": "Welcome",
Input (file.json)
[
{
"url": "https://docs.mysite.com",
"name": "Welcome - https://docs.mysite.com/",
"Ocurrences": "679"
},
{
"url": "https://docs.mysite.com",
"name": "Welcome",
"Ocurrences": "382"
}
]
Failing script (using variable)
siteUrl="docs.mysite.com"
jq --arg siteUrl "$siteUrl" '.[].name|= (gsub(" - https://$siteUrl/"; ""))' file.json > file1.json`
Desired output (file1.json)
[
{
"url": "https://docs.mysite.com",
"name": "Welcome",
"Ocurrences": "679"
},
{
"url": "https://docs.mysite.com",
"name": "Welcome",
"Ocurrences": "382"
}
]
I've tried various iterations on removing quotes, changing between ' and ", and adding and removing backslashes.
Successful script (not using variable)
jq '.[].name|= (gsub(" - https://docs.mysite.com/"; ""))' file.json > file1.json
More specifically, if it matters, I am processing an export of a website's usage data from Azure App Insights. Unfortunately, the same page may be assigned different names. I sum the Ocurrences of the two objects with the newly identical url later. If it is better to fix this in App Insights I am grateful for that insight, although I know Bash better than Kusto queries. I am grateful for any help or direction.
Almost. Variables are not automatically expanded within a string. You must interpolate them explicitly with \(…):
jq --arg siteUrl 'docs.mysite.com' '.[].name |= (gsub(" - https://\($siteUrl)/"; ""))' file.json
Alternatively, detect a suffix match and extract the prefix by slicing:
jq --arg siteUrl 'docs.mysite.com' '" - https://\($siteUrl)/" as $suffix | (.[].name | select(endswith($suffix))) |= .[:$suffix|-length]' file.json
The below value of private_localaddress needs to passed to "range_start" in yaml file. how to do this ?
Here is the script:
#!/bin/bash
MAC=`curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/`
myarr=($MAC)
for val in ${myarr[#]}; do
interfaceindex=`curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/$val/device-number`
if [ $interfaceindex == 2 ];
then
private_localaddress=`curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/$val/local-ipv4s`
echo $private_localaddress
fi
done
Here is the YAML file :
---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: worker-private-eth2
spec:
config: '{
"cniVersion": "0.3.1",
"ipam": {
"type": "whereabouts",
"range": "10.30.11.0/25",
"range_start": $private_localaddress,
"range_end": "10.30.11.127",
"routes": [
{ "dst": "10.93.123.0/24", "gw": "10.30.11.1" }
],
Ffffff
Use a proper yaml parser. Here are some examples:
Using mikefarah/yq:
value="$private_localaddress" yq \
'.spec.config |= (fromjson | .ipam.range_start = strenv(value) | tojson)'
Using kislyuk/yq:
yq -y --arg value "$private_localaddress" \
'.spec.config |= (fromjson | .ipam.range_start = $value | tojson)'
Using itchyny/gojq:
gojq --arg value "$private_localaddress" --yaml-input --yaml-output \
'.spec.config |= (fromjson | .ipam.range_start = $value | tojson)'
Perhaps the simplest would be to make private_localaddress an environment variable and use envsubst to apply the changes to your YAML.
However to end up with correct YAML code, you need to ensure that you have a quoted string to the right of the colon. The best would be if you can already ensure that your YAML is written as
"range_start": "$private_localaddress",
If this is not feasible, you have to add the quotes when creating the variable, i.e.
export private_localaddress=\"$(curl -s http://169.254.169.254/latest/meta-data/network/interfaces/macs/$val/local-ipv4s)\"
I have the following JSON file:
{
"1":
{
"media_content":"test3.xspf"
},
"2":
{
"media_content":"test3.xspf"
}
}
In the terminal, using bash as shell, I can execute the following commands:
export schedules="1"
echo $(jq '.[env.schedules]["media_content"]' json_file.json)
Which results in outputing this:
test3.xspf
So it works as expected, but when I place that jq command in a script and run it, it just returns null.
I did echo the values of schedules to make sure the value is non-null inside the script, and it is ok:
echo $schedules
But I did not manage to find the reason, why this command works when run directly in shell and does not work when run in script.
I run the script in the following ways:
bash script.sh
./script.sh
PS: yes, I did offer execute permission: chmod +x script.sh
HINT: env.schedules represents the environment variable 'schedules', and I did make sure that it is assigned in the script before calling jq.
EDIT: I am posting now a whole script, specifying the files tree.
There is one directory containing:
script.sh
json_file.json
static.json
script.sh:
export zone=$(cat static.json | jq '.["1"]');
echo "json block: "$zone
export schedules="$(echo $zone | jq '.schedules')"
echo "environment variable: "$schedules
export media_content=$(jq '.[env.schedules]["media_content"]' json_file.json)
echo "What I want to get: \"test3.xspf\""
echo "What I get: "$media_content
json_file.json:
{
"1":
{
"media_content":"test3.xspf"
},
"2":
{
"media_content":"test3.xspf"
}
}
static.json:
{
"1":
{
"x": "0",
"y": "0",
"width": "960",
"height": "540",
"schedules":"1"
}
}
If I run the script, it displays:
json block: { "x": "0", "y": "0", "width": "960", "height": "540", "schedules": "1" }
environment variable: "1"
What I want to get: "test3.xspf"
What I get: null
If I hardcode the variable:
export schedules="1"
The problem no longer occurs
The problem is simple.
It's not jq's fault.
It the unproper way the schedule's value is piped to the next command.
You have to remove the "s that surround the variable's value, add the second command that uses sed to do that:
export schedules="$(echo $zone | jq '.schedules')"
schedules=$( echo $schedules | sed s/\"//g )
Long answer
Let's see:
here schedules is a string and echo shows its value as being 1:
export schedules="1" ; echo $schedules
here even though double quotes are not mentioned:
export schedules=1 ; echo $schedules
But the result from this also generates additional "s:
export schedules=$(echo $zone | jq '.schedules')
If you print it now you will see additional "s:
echo $schedules # "1"
So just remove the "s from the value:
schedules=$( echo $schedules | sed s/\"//g )
I am having below code in JSON file.
{
"comment": {
"vm-updates": [],
"site-ops-updates": [
{
"comment": {
"message": "You can start maintenance on this resource"
},
"hw-name": "Machine has got missing disks. "
}
]
},
"object_name": "4QXH862",
"has_problems": "yes",
"tags": ""
}
I want to separate "hw-name" from this JSON file using jq. I've tried below combinations, but nothing worked.
cat jsonfile | jq -r '.comment[].hw-name'
cat json_file.json | jq -r '.comment[].site-ops-updates[].hw-name'
Appreciated help from StackOverflow!!!
It should be:
▶ cat jsonfile | jq -r '.comment."site-ops-updates"[]."hw-name"'
Machine has got missing disks.
Or better still:
▶ jq -r '.comment."site-ops-updates"[]."hw-name"' jsonfile
Machine has got missing disks.
From the docs:
If the key contains special characters, you need to surround it with double quotes like this: ."foo$", or else .["foo$"].
I'm a bit new to batch scripting, so I apologize if this is glaringly obvious, but I couldn't find any similar information.
I'm trying to perform the following cURL call:
curl -H "Content-Type:application/json" -d '{"lt":"f","sort":"n","max":"1500","offset":"1500"}' [API_KEY]#api.nal.usda.gov/ndb/list
When I run that line in a command line (or Cygwin) it does exactly what I want it to.
However, when I try to call it from a bat file, it seems my parameters are getting messed up somehow.
FOR /L %%A IN (0, 1500, 77500) DO (
curl -H "Content-Type:application/json" -d '{"lt":"f","sort":"n","max":"1500","offset":"%%A"}' [API_KEY]#api.nal.usda.gov/ndb/list > %%A.txt
)
I'm getting output into the correct .txt file, but it doesn't seem like the %%A in offset is getting replaced. I'm getting a "bad parameter" exception from the API. From the output on the command line, it looks accurate.
I'm open to using bash scripting instead if it would make more sense, but I was having the same issue using bash.
(Note: I replaced my API key with a placeholder in the example.. that's not the problem)
In bash at least, the problem is that variable expansion does not occur inside single quotes; you need to use double quotes and escape the nested double quotes:
for a in 0 1500 77500; do
curl -H "Content-Type:application/json" -d "{\"lt\":\"f\",\"sort\":\"n\",\"max\":\"1500\",\"offset\":\"$a\"}" [API_KEY]#api.nal.usda.gov/ndb/list > "$a".txt
)
I suspect you need to do the equivalent in a batch file.
You can concatenate adjacent single- and double-quoted strings to minimized the number of escaped quotes:
... -d '{"lt": "f", "sort": "n", "max": "1500", "offset": "'"$a"'"}' ...
but you may want to consider one of two other options. First, read the data from a here document instead of using a hard-coded string:
curl -H "..." -d#- [API_KEY]#api.nal.usda.gov/ndb/list > "$a".txt <<EOF
{"lt": "f", "sort": "n", "max": "1500", "offset": "$a"}
EOF
or use something like jq to generate the JSON for your:
curl -H "..." \
-d "$(jq --arg a "$a" '{lt: "f", sort: "n", max: "1500", offset: $a}') \
[API_KEY]#api.nal.usda.gov/ndb/list > "$a".txt
The jq solution would be preferable in general, since you don't have to worry about pre-escaping any variable values.