In my shell, I have a JSON response like you can see below. When I am printing, it prints "" with JSON, but I want to remove them.
{
"Grade": "tenth"
}
I am using
curl -s "<<API>>"| awk '{print $2;}'
Use jq JSON parser instead of awk:
curl -s "<<API>>" | jq -r '.Grade'
-r is the raw mode. It outputs the string without quote.
Related
I'm trying to get jq to parse a JSON structure like:
{
"a" : 1,
"b" : 2,
"c" : "{\"id\":\"9ee ...\",\"parent\":\"abc...\"}\n"
}
That is, an element in the JSON is a string with escaped json.
So, I have something along the lines of
$ jq [.c] myFile.json | jq [.id]
But that crashes with jq: error: Cannot index string with string
This is because the output of .c is a string, not more JSON.
How do I get jq to parse this string?
My initial solution is to use sed to replace all the escape chars (\":\", \",\" and \") but that's messy, I assume there's a way built into jq to do this?
Thanks!
edit:
Also, the jq version available here is:
$ jq --version
jq version 1.3
I guess I could update it if required.
jq has the fromjson builtin for this:
jq '.c | fromjson | .id' myFile.json
fromjson was added in version 1.4.
You can use the raw output (-r) that will unescape characters:
jq -r .c myfile.json | jq .id
ADDENDUM: This has the advantage that it works in jq 1.3 and up; indeed, it should work in every version of jq that has the -r option.
Motivation: you want to parse JSON string - you want to escape a JSON object that's wrapped with quotes and represented as a String buffer, and convert it to a valid JSON object. For example:
some JSON unescaped string :
"{\"name\":\"John Doe\",\"position\":\"developer\"}"
the expected result ( a JSON object ):
{"name":"John Doe","position":"developer"}
Solution: In order to escape a JSON string and convert it into a valid JSON object use the sed tool in command line and use regex expressions to remove/replace specific characters:
cat current_json.txt | sed -e 's/\\\"/\"/g' -e 's/^.//g' -e 's/.$//g'
s/\\\"/\"/g replacing all backslashes and quotes ( \" ) into quotes only (")
s/^.//g replacing the first character in the stream to none character
s/.$//g replacing the last character in the stream to none character
I'm trying to create 2 variables via bash $lat, $long base on the result of my curl response.
curl ipinfo.io/33.62.137.111 | grep "loc" | awk '{print $2}'
I got.
"42.6334,-71.3162",
I'm trying to get
$lat=42.6334
$long=-71.3162
Can someone give me a little push ?
IFS=, read -r lat long < <(
curl -s ipinfo.io/33.62.137.111 |
jq -r '.loc'
)
printf 'Latitude is: %s\nLongitude is: %s\n' "$lat" "$long"
The ipinfo.io API is returning JSON data, so let parse it with jq:
Here is the JSON as returned by the query from your sample:
{
"ip": "33.62.137.111",
"city": "Columbus",
"region": "Ohio",
"country": "US",
"loc": "39.9690,-83.0114",
"postal": "43218",
"timezone": "America/New_York",
"readme": "https://ipinfo.io/missingauth"
}
We are going to JSON query the loc entry from the main root object ..
curl -s ipinfo.io/33.62.137.111: download the JSON data -s silently without progress.
jq -r '.loc': Process JSON data, query the loc entry of the main object and -r output raw string.
IFS=, read -r lat long < <(: Sets the Internal Field Separator to , and read both lat and long variables from the following command group output stream.
Although the answer from #LeaGris is quite interesting, if you don't want to use an external library or something, you can try this:
Playground: https://repl.it/repls/ThoughtfulImpressiveComputer
coordinates=($(curl ipinfo.io/33.62.137.111 | sed 's/ //g' | grep -P '(?<=\"loc\":").*?(?=\")' -o | tr ',' ' '))
echo "${coordinates[#]}"
echo ${coordinates[0]}
echo ${coordinates[1]}
Example output:
39.9690 -83.0114 # echo "${coordinates[#]}"
39.9690 # ${coordinates[0]}
-83.0114 # ${coordinates[1]}
Explanation:
curl ... get the JSON data
sed 's/ //g' remove all spaces
grep -P ... -o
-P interpret the given pattern as a perl regexp
(?<=\"loc\":").*?(?=\")
(?<=\"loc\":") regex lookbehind
.*? capture the longitude and latitude part with non-greedy search
(?=\") regex lookahead
-o get only the matching part which'ld be e.g. 39.9690,-83.0114
tr ',' ' ' replace , with space
Finally we got something like this: 39.9690 -83.0114
Putting it in parentheses lets us create an array with two values in it (cf. ${coordinates[...]}).
Hello I am using curl to get some info which I need to clean up.
This is from curl command:
{"ip":"000.000.000.000","country":"Italy","city":"Milan","longitude":9.1889,"latitude":45.4707, etc..
I would need to get "Ita" as output, that is the first three letter of the country.
After reading sed JSON regular expression i tried to adapt resulting in
sed -e 's/^.*"country":"[a-zA-Z]{3}".*$/\1/
but this won't work.
Can you please help?
Using jq, you can do:
curl .... | jq -r '.country[0:3]'
If you need to set the country to the first 3 chars,
jq '.country = .country[0:3]'
some fairly advanced bash:
{
read country
read city
} < <(
curl ... |
jq -r '.country[0:3], .city[0:3]'
)
Then:
$ echo "$country $city"
Ita Mil
I'm trying to send an emoji/emoticon to my Telegram bot using a bash script. This bash script calls the Telegram API as follows:
curl -s -X POST 'https://api.telegram.org/'$API'/sendMessage' -F chat_id=$chat -F text=$text
Since the bash script isn't unicode, I cannot simply copy/paste the emojis from the web. Therefore I tried using the UTF-8 emoji variants, but the backslash character keeps getting escaped.
The expected json output should be as follows: "text":"\ud83d\udd14"
Instead, this is what I get:
Input: $text = \xF0\x9F\x98\x81
JSON Output = "text":"\\xF0\\x9F\\x98\\x81\\"
Input: $text = u'\U0001F604'
JSON Output = "text": "u'\\U0001F604'\"
Input: $text = \U0001F514
JSON Output = "text":"\\U0001F514"
Input: $text = "(1f600)"
JSON Output = "text":"\"(1f600)\""
Input: $text = \ud83d\ude08
JSON Output = "text":"\\ud83d\\ude08"
Input: $text = \\\ud83d\\\udd14
JSON Output = "text":"\\\\\\ud83d\\\\\\udd14"
What is the correct syntax to send an emoji using a bash script and curl to my Telegram bot?
Thank you very much!
Generating JSON For The Telegram API
If your question is about JSON encoding, let jq figure it out for you:
s='🔔' ## or s=$'\360\237\224\224'
json=$(jq -anc --arg id "$chat" --arg s "$s" '{"chat_id": $id, "text": $s}')
curl -X POST -H "Content-Type: application/json" -d "$json" \
"https://api.telegram.org/$API/sendMessage"
From JSON To String Literal
In bash 4.0 or newer, the shell itself can be asked to give you an ASCII-printable literal string which will correspond to a multi-byte character.
LC_ALL=C printf "s=%q\n" "$(jq -r . <<<'"\ud83d\udd14"')"
...will output:
s=$'\360\237\224\224'
From String Literal To JSON
To go in the other direction:
s=$'\360\237\224\224'
jq -anM --arg s "$s" '$s'
...emits as output:
"\ud83d\udd14"
You can use echo -e '\U0001F514' for get emoji
curl -F "text=`echo -e '\U0001F514'` - it's a bell" "https://api.telegram.org/$API/sendMessage?chat_id=$chat"
or $'\U0001F514'
curl -F "text="$'\U0001F514'" - it's a bell" "https://api.telegram.org/$API/sendMessage?chat_id=$chat"
I have problem with assigning curl as variable and assign curl's output to variable:
#get results url, format json
URL=$(curl https://api.apifier.com/xy)
#jq is a cli json interpreter
#resultUrl contains the final URL which we want download
OK= "$URL" | jq '.resultsUrl'
#api probably is running
sleep 5
curl "$OK"
Maybe it is trivial, but I don't know where is the problem.
My guess is:
jq '.resultsUrl'
outputs the field resultsUrl with quotes, so curl does not process it correctly. Furthermore, $URL | ... does not work, you would have to use echo or curl directly.
Try
OK=$(curl -s https://api.apifier.com/v1/xHbBnrZ9rxF4CdKjo/crawlers/Example_Alcatraz_Cruises/execute?token=nJ9ohCHZPaJRFEb7nFqtzm76u | jq -r '.resultsUrl')
curl -s "$OK"
which results for me in
[{ "id": 2, "url": "https://www.alcatrazcruises.com/SearchEventDaySpan.aspx?date=02-25-2016&selected=", "loadedUrl": "https://www.alcatrazcruises.com/SearchEventDaySpan.aspx?date=02-25-2016&selected=", "requestedAt": "2016-02-25T23:24:52.611Z", "loadingStartedAt": "2016-02-25T23:24:54.663Z", "loadingFinishedAt": "2016-02-25T23:24:55.642Z", "loadErrorCode": null, "pageFunctionStartedAt": "2016-02-25T23:24:55.839Z", "pageFunctionFinishedAt": "2016-02-25T23:24:55.841Z", "uniqueKey": "https://www.alcatrazcruises.com/SearchEventDaySpan.aspx?date=02-25-2016&selected=", "type": "UserEnqueued", ...
This should be what you expect.
However, sometimes the first API call yields an error:
{
"type": "ALREADY_RUNNING",
"message": "The act is already running and concurrent execution is not allowed"
}
so resultsURL will be null, you will have to handle this error case.
Your line
OK= "$URL" | jq '.resultsURL'
sets the environment variable OK to an empty string, then tries to execute "$URL" as a command and pipe its output to jq. If you want to setOK to the result of a command, you have to use $OK=(...), just like you did when setting URL. The correct syntax is:
OK=$(echo "$URL" | jq '.resultsURL')
And to remove the quotes from the output of .jq, you can do:
OK=$(echo "$URL" | jq '.resultsURL' | tr -d '"')