This question already has an answer here:
How to convert a JSON object stream into an array with jq
(1 answer)
Closed 6 years ago.
Say I have the input:
{
"name": "John",
"email": "john#company.com"
}
{
"name": "Brad",
"email": "brad#company.com"
}
How do I get the output:
[
{
"name": "John",
"email": "john#company.com"
},
{
"name": "Brad",
"email": "brad#company.com"
}
]
I tried both:
jq '[. | {name, email}]'
and
jq '. | [{name, email}]'
which both gave me the output
[
{
"name": "John",
"email": "john#company.com"
}
]
[
{
"name": "Brad",
"email": "brad#company.com"
}
]
I also saw no options for an array output in the documentations, any help appreciated
Use slurp mode:
o --slurp/-s:
Instead of running the filter for each JSON object
in the input, read the entire input stream into a large
array and run the filter just once.
$ jq -s '.' < tmp.json
[
{
"name": "John",
"email": "john#company.com"
},
{
"name": "Brad",
"email": "brad#company.com"
}
]
Related
I'm trying to use JSONata to convert arrays of "key/value" objects into properties of the parent object. My input looks like this:
[
{
"city": "Ottawa",
"properties": [
{
"name": "population",
"value": 37
},
{
"name": "postalCode",
"value": 10001
},
{
"name": "founded",
"value": 1826
}
]
},
{
"city": "Toronto",
"properties": [
{
"name": "population",
"value": 54
},
{
"name": "postalCode",
"value": 10002
}
]
}
]
I'm struggling to generate the output I need, I've seen examples that reference explicit elements, like in this answer, but I need the properties to be converted "dynamically" since I don't know them in advance. I think I need something like this, but I'm missing some particular function:
$[].{
"city": city,
properties.name: properties.value
}
This is the output I need to generate:
[
{
"city": "Ottawa",
"population": 37,
"postalCode": 10001,
"founded": 1826
},
{
"city": "Toronto",
"population": 54,
"postalCode": 10002
}
]
The properties arrays don't always contain the same keys, but the city attributes are always present.
You can use the reduce operator, as described in the Grouping docs here:
$[].(
$city := city;
properties{ "city": $city, name: value }
)
You can play with it live: https://stedi.link/uUANwtE
Please try this expression.
$[].{
"city": $.city,
$.properties[0].name: $.properties[0].value,
$.properties[1].name: $.properties[1].value,
$.properties[2].name: $.properties[2].value,
$.properties[3].name: $.properties[3].value
}
https://try.jsonata.org/s1Ea4kUvo
I have a JSON file that I want to convert into a CSV file using the jq in a shell script. I want to create a single row from this entire JSON file. I have to extract value from values. The row output should be something like
null,642,642,412,0,null,null
Here is my JSON file
{
"data": [
{
"name": "exits",
"period": "lifetime",
"values": [
{
"value": {}
}
],
"title": "Exits",
"description": "Number of times someone exited the carousel"
},
{
"name": "impressions",
"period": "lifetime",
"values": [
{
"value": 642
}
],
"title": "Impressions",
"description": "Total number of times the media object has been seen"
},
{
"name": "reach",
"period": "lifetime",
"values": [
{
"value": 412
}
],
"title": "Reach",
"description": "Total number of unique accounts that have seen the media object"
},
{
"name": "replies",
"period": "lifetime",
"values": [
{
"value": 0
}
],
"title": "Replies",
"description": "Total number of replies to the carousel"
},
{
"name": "taps_forward",
"period": "lifetime",
"values": [
{
"value": {}
}
],
"title": "Taps Forward",
"description": "Total number of taps to see this story's next photo or video"
},
{
"name": "taps_back",
"period": "lifetime",
"values": [
{
"value": {}
}
],
"title": "Taps Back",
"description": "Total number of taps to see this story's previous photo or video"
}
]
}
Hi tried using this jq command :
.data | map(.values[].value) | #csv
This is giving the following output:
jq: error (at :70): object ({}) is not valid in a csv row
exit status 5
So when I am getting this empty JSON object it is reflecting an error.
Please Help!!
The row output should be something like
null,642,642,412,0,null,null
Using length==0 here is dubious at best. To check for {} one could write:
jq '.data | map(.values[].value | if . == {} then "null" else . end) | #csv'
Similarly for [].
If you run the command without the #csv part you will see that the output is:
[
{},
642,
412,
0,
{},
{}
]
By replacing the empty objects with "null": (length == 0)
jq '.data | map(.values[].value) | map(if (type == "object" and length == 0 ) then "null" else . end) | #csv'
Output:
"\"null\",642,412,0,\"null\",\"null\""
Per suggestion from #aaron (see comment). The following can produce the requested output without extra post-processing. Disclaimer: this is not working with my jq 1.5, but working on jqplay with jq 1.6.
jq --raw-output '.data | map(.values[].value) | map(if (type == "object" and length == 0 ) then "null" else . end) | join(",")'
Output:
null,642,412,0,null,null
I have to parse a JSON which includes an array of of class/name:
{
"_class": "model.ListView",
"jobs": [
{
"_class": "hudson.matrix.MatrixProject",
"name": "tests-different-node-full"
},
{
"_class": "hudson.matrix.MatrixProject",
"name": "tests-jms-activemq-full"
},
{
"_class": "hudson.matrix.MatrixProject",
"name": "tests-txpropag-jpa-full"
}
]
}
I need to retrieve the list of "name".
Looking at the examples I've found of jq, I have tried with:
cat jobs.json | jq '.[].name'
It fails with:
jq: error (at <stdin>:0): Cannot index string with string "name"
How should I reference the name element of the array?
Thanks
found it:
cat jobs.json | jq '.jobs[].name'
I have been working with Openrefine for the last days trying to figure out how to export a Google Data sheet into a JSON file.
I have the following data that I want to export to a JSON file.
id first name last name friends first name friends last name family first name family last name
1 James Brown Judy Garland Mary Brown
John Neverland Marlene Brown
Paul Garland Judy Brown
2 John Buller Amy Garland Francis Buller
Peter Flake John Buller
Jules Peter Judy Buller
The JSON that I'm expecting is:
{
"results": [
{
"id": 1,
"firstName": "James",
"lastName": "Brown",
"has": {
"friends": [
{
"firstName": "Judy",
"lastName": "Garland"
},
{
"firstName": "John",
"lastName": "Neverland"
},
{
"firstName": "Paul",
"lastName": "Garland"
}
],
"family": [
{
"firstName": "Mary",
"lastName": "Brown"
},
{
"firstName": "Marlene",
"lastName": "Brown"
},
{
"firstName": "Judy",
"lastName": "Brown"
}
]
}
},
{
"id": 2,
"firstName": "John",
"lastName": "Buller",
"has": {
"friends": [
{
"firstName": "Amy",
"lastName": "Garland"
},
{
"firstName": "Peter",
"lastName": "Flake"
},
{
"firstName": "Jules",
"lastName": "Peter"
}
],
"family": [
{
"firstName": "Francis",
"lastName": "Buller"
},
{
"firstName": "John",
"lastName": "Buller"
},
{
"firstName": "Judy",
"lastName": "Buller"
}
]
}
}
]
}
So far I have tried several approaches:
1) using excel-to-json but it's limited to single nesting and it has some limitations as to column names
2) using Openrefine and the Templating tool but I have encountered several issues:
- Although they are detected as records in openrefine, you export rows and not records so it will export 6 rows to JSON, 4 of them containing empty data
- If i try filling down columns it will aso export 6 rows to JSON, 4 of them with duplicates thus loosing the relations between the person and his family members and friends
Any help would be much appreciated as I'm trying to export about 150,000 records of this type that have to be in this JSON format.
OpenRefine only support one level of nesting. You might need to go with a programming language or an ETL solution to have nested element.
I am using ruby 2.0.0, At present i have one demo.json file with following hash values:
{
"users":
{
"#jon" :
{
"name": "pradeep",
"Email": "pradeep#yahoo.com",
"area": "#jon",
"location": "#newyork"
},
"#smith" :
{
"name": "Smith",
"Email": "Joe#yahoo.com",
"area": "#smith",
"location": "#lverginia"
}
}
}
now i am taking json values in object using following codes:
require 'json'
json = File.read('demo.json')
obj = JSON.parse(json)
here #jon and #smith is usernames, Now i wants to take usernames via keyboard inputs and than all the other values inside of #jon with same keyboard.
Suppose i have one new user #david and his other values are like:
"name": "pradeep",
"Email": "pradeep#yahoo.com",
"area": "#jon",
"location": "#newyork"
i wants to add this in above demo.json file without remove other values, Any idea how can i do?
I tried to do this in this way:
obj["users"]
But as i got username via input so i cant hard code username after user key on "obj" object, Hope this make sense..
Read this JSON also .
I'd do as below :
require 'json'
json = JSON.parse(File.read("test.json"))
new_information_arry = ["users", "name", "Email", "area", "location"].map do |elem|
puts "please give the value of #{elem}"
[elem,gets.chomp]
end
new_information_hash = Hash[new_information_arry[1..-1]]
json['users'][new_information_arry.first.last] = new_information_hash
File.write("outputfile.json",JSON.pretty_generate(json))
I put the below content to my file 'test.json' :
{
"users":
{
"#jon" :
{
"name": "pradeep",
"Email": "pradeep#yahoo.com",
"area": "#jon",
"location": "#newyork"
},
"#smith" :
{
"name": "Smith",
"Email": "Joe#yahoo.com",
"area": "#smith",
"location": "#lverginia"
}
}
}
Then I ran the above code as below :
please give the value of users
#david
please give the value of name
pradeep
please give the value of Email
pradeep#yahoo.com
please give the value of area
#jon
please give the value of location
#newyor
And now my output file outputfile.json contains :
{
"users": {
"#jon": {
"name": "pradeep",
"Email": "pradeep#yahoo.com",
"area": "#jon",
"location": "#newyork"
},
"#smith": {
"name": "Smith",
"Email": "Joe#yahoo.com",
"area": "#smith",
"location": "#lverginia"
},
"#david": {
"name": "pradeep",
"Email": "pradeep#yahoo.com",
"area": "#jon",
"location": "#newyor"
}
}
}