Hello I am using curl to get some info which I need to clean up.
This is from curl command:
{"ip":"000.000.000.000","country":"Italy","city":"Milan","longitude":9.1889,"latitude":45.4707, etc..
I would need to get "Ita" as output, that is the first three letter of the country.
After reading sed JSON regular expression i tried to adapt resulting in
sed -e 's/^.*"country":"[a-zA-Z]{3}".*$/\1/
but this won't work.
Can you please help?
Using jq, you can do:
curl .... | jq -r '.country[0:3]'
If you need to set the country to the first 3 chars,
jq '.country = .country[0:3]'
some fairly advanced bash:
{
read country
read city
} < <(
curl ... |
jq -r '.country[0:3], .city[0:3]'
)
Then:
$ echo "$country $city"
Ita Mil
Related
I am trying to extract the JIRA Ticket number from a string.
The Jira ticket might be mentioned any where in the line like:
Merge pull request #1387 from Config-change/REL-12345
REL-12345: Enable XAPI at config level
I just want REL-12345 as the output.
grep -Eow 'REL-[0-9]+'
+ is one or more, to specifiy N numbers (eg 5):
grep -Eow 'REL-[0-9]{5}
Ranges: {3,6} is 3 to 6, {5,} is 5 or more, etc.
On GNU/Linux: man grep -> /Repetition for more details.
-o prints only matching strings
-w matches full words only, ie. to avoid matching WREL-12345 (for example)
grep -Eow 'REL-[[:alnum:]]+' for both letters and numbers (after REL-).
If this is the standard.....
Input: Merge pull request #1387 from Config-change/REL-12345
echo "Merge pull request #1387 from Config-change/REL-12345" | cut -d/ -f2
Input: REL-12345: Enable XAPI at config level
echo "REL-12345: Enable XAPI at config level" | cut -d: -f1
You can pass a String to sed and use substitution with REGEX, like this:
myString="This is REL-12345 a test string "
sed -n 's/.*\(\REL-5*[0-9]*\).*/\1/p' <<< $myString
this should return: REL-12345
Sample data:
$ cat jira.dat
Merge pull request #1387 from Config-change/REL-12345
REL-12346: Enable XAPI at config level
One idea using bash regex matching and the resulting BASH_REMATCH[]:
regex='(REL-[[:digit:]]+)'
while read -r line
do
printf "\n########## ${line}\n"
[[ "${line}" =~ ${regex} ]] && echo "${BASH_REMATCH[1]}"
done < jira.dat
This generates:
REL-12345
REL-12346
Sample data:
$ cat jira.dat
Merge pull request #1387 from Config-change/REL-12345
REL-12346: Enable XAPI at config level
One idea using grep:
$ grep -Eo 'REL-[[:digit:]]+' jira.dat
REL-12345
REL-12346
i have a query with this Json and grep :-
[
{
"name":"Jon",
"id":123
},
{
"name":"Ray",
"id":1234
},
{
"name":"Abraham",
"id":12345
}
]
How can one extract name from this json where id matches say 1234 , can be random , using grep or sed?
I would suggest to use jq but if you want to use grep try
grep -B1 'id.*1234' < input_file | grep name
from man page
-B num, --before-context=num
Print num lines of leading context before each match. See also the -A and -C options.
please suggest the jq command
I take the liberty to fulfill the request.
jq -r '.[]|select(.id==1234).name' file
.[] - iterates the array elements
select(.id==1234) - filters element with desired id
.name - extracts name
The option -r causes the name to be written unquoted.
I'm trying to create 2 variables via bash $lat, $long base on the result of my curl response.
curl ipinfo.io/33.62.137.111 | grep "loc" | awk '{print $2}'
I got.
"42.6334,-71.3162",
I'm trying to get
$lat=42.6334
$long=-71.3162
Can someone give me a little push ?
IFS=, read -r lat long < <(
curl -s ipinfo.io/33.62.137.111 |
jq -r '.loc'
)
printf 'Latitude is: %s\nLongitude is: %s\n' "$lat" "$long"
The ipinfo.io API is returning JSON data, so let parse it with jq:
Here is the JSON as returned by the query from your sample:
{
"ip": "33.62.137.111",
"city": "Columbus",
"region": "Ohio",
"country": "US",
"loc": "39.9690,-83.0114",
"postal": "43218",
"timezone": "America/New_York",
"readme": "https://ipinfo.io/missingauth"
}
We are going to JSON query the loc entry from the main root object ..
curl -s ipinfo.io/33.62.137.111: download the JSON data -s silently without progress.
jq -r '.loc': Process JSON data, query the loc entry of the main object and -r output raw string.
IFS=, read -r lat long < <(: Sets the Internal Field Separator to , and read both lat and long variables from the following command group output stream.
Although the answer from #LeaGris is quite interesting, if you don't want to use an external library or something, you can try this:
Playground: https://repl.it/repls/ThoughtfulImpressiveComputer
coordinates=($(curl ipinfo.io/33.62.137.111 | sed 's/ //g' | grep -P '(?<=\"loc\":").*?(?=\")' -o | tr ',' ' '))
echo "${coordinates[#]}"
echo ${coordinates[0]}
echo ${coordinates[1]}
Example output:
39.9690 -83.0114 # echo "${coordinates[#]}"
39.9690 # ${coordinates[0]}
-83.0114 # ${coordinates[1]}
Explanation:
curl ... get the JSON data
sed 's/ //g' remove all spaces
grep -P ... -o
-P interpret the given pattern as a perl regexp
(?<=\"loc\":").*?(?=\")
(?<=\"loc\":") regex lookbehind
.*? capture the longitude and latitude part with non-greedy search
(?=\") regex lookahead
-o get only the matching part which'ld be e.g. 39.9690,-83.0114
tr ',' ' ' replace , with space
Finally we got something like this: 39.9690 -83.0114
Putting it in parentheses lets us create an array with two values in it (cf. ${coordinates[...]}).
Looking for a one liner with csvkit.
From a plain json object
{
"whatever": 2342,
"otherwise": 119,
"and": 1,
"so": 2,
"on": 3
}
Want this csv
whatever,2342
otherwise,119
and,1
so,2
on,3
I basically want this command to work, but it doesn't.
echo $the_json | in2csv -f json
> When converting a JSON document with a top-level dictionary element, a key must be specified.
Seems like something csvkit can do, and I just haven't found the right options.
short answer
variant A: in2csv (csvkit) + csvtool
wrap your json in brackets
use in2csv's -I option to avoid unexpected behavior
use a command to transpose the two-row CSV, e.g. csvtool
echo "[$the_json]" | in2csv -I -f json | csvtool transpose -
variant B: use jq instead
This is a solution using only jq: (https://stedolan.github.io/jq/)
echo "$the_json" | jq -r 'to_entries[] | [.key, .value] | #csv'
taken from How to map an object to arrays so it can be converted to csv?
long answer (csvkit + csvtool)
the input
in2csv -f json expects a list of JSON objects, so you need to wrap the single object ({...}) into square brackets ([{...}]).
On POSIX compatible shells, write
echo "[$the_json]"
which will print
[{
"whatever": 2342,
"otherwise": 119,
"and": 1,
"so": 2,
"on": 3
}]
the csvkit command
You may pipe the above data directly into in2csv. However, you might run into issues with the ”type inference“ (CSV data interpretation) feature of csvkit:
$ echo "[$the_json]" | in2csv -f json
whatever,otherwise,and,so,on
2342,119,True,2,3
1 has become True. For details, see the Tips and Troubleshooting part of the docs. It's suggested to turn off type inference using the -I option:
$ echo "[$the_json]" | in2csv -I -f json
whatever,otherwise,and,so,on
2342,119,1,2,3
Now the result is as expected
transpose the data
Still, you need to transpose the data. The csvkit docs say:
To transpose CSVs, consider csvtool.
(csvtool is available on github, opam, debian and probably other distribution channels.)
Using csvkit + csvtool, your final command looks like this:
echo "[$the_json]" | in2csv -I -f json | csvtool transpose -
with the hyphen (-) meaning to take the data from stdin. This is the result:
whatever,2342
otherwise,119
and,1
so,2
on,3
that's it.
I think there is no one-liner solution with csvtool only, you'll need in2csv. You may, however, use jq instead, see the short answer.
FTR, I'm using csvkit version 1.0.3.
Tested the first posted answer works! But it is a bit confusing because "[$the_json]" means the raw content of the json. So an example of command could be this:
echo '[{"a":"b","c":"d"}]' | in2csv -I -f json | csvtool transpose -
and if you want to do it with a file name instead, for instance myfile.json one can add the brackets with a sed command and pipe it to in2csv:
sed -e '1s/^/[/' -e 's/$/,/' -e '$s/,$/]/' myfile.json | in2csv -I -f json > myfile.csv
Example with the full transposition command:
sed -e '1s/^/[/' -e 's/$/,/' -e '$s/,$/]/' myfile.json | in2csv -I -f json | csvtool transpose - > myfile.csv
source: How to add bracket at beginning and ending in text on UNIX
I want to parsing JSON data using jq(as described here ) and delete any newlines character from the resulting string.
I've already tried to use tr but this approach remove also all the white spaces between parsed values.
My code:
IP=$(curl -s https://ipinfo.io/ip) # Get ip address
curl -s https://ipinfo.io/${IP}/geo | jq -r '.ip, .city, .country' | tr -d '\n' # parse only few values from the JSON data and remove new lines.
What i get with the code above is the following string:
XXX.XXX.XXX.XXXCity_NameCountry_Name but i want something like this:
XXX.XXX.XXX.XXX City_Name Country_Name
You could craft a single string from the three pieces of data so that it appears on a single line (single result by input) :
IP=$(curl -s https://ipinfo.io/ip)
curl -s https://ipinfo.io/${IP}/geo | jq -r '.ip + " " + .city + " " + .country'
> myIp myCity myCountryCode
Another option for a different but similar output format would be to use the #csv output format, where you would want to output an array of cells for each input :
IP=$(curl -s https://ipinfo.io/ip)
curl -s https://ipinfo.io/${IP}/geo | jq -r '[.ip, .city, .country] | #csv'
> "myIp","myCity","myCountryCode"
This result could be easily worked on from any spreadsheet software.