How to use jsonata to return multiple objects, distributing the first key/value, which will always be dateTime, with the remaining keys varying - jsonata

How can jsonata be used to transform:
{
"data": [
{
"dateTime": "2019-10-19T12:53:54.043Z",
"Reactor3.Level": 1.51860072870498,
"Reactor3.Temp": 27.1360543141452
},
{
"dateTime": "2019-10-19T12:55:54.043Z",
"Reactor3.Press": 88.9,
"Reactor3.Temp": 24.1418981047159
}
]
}
Into a series of objects containing only two keys of {"dateTime":"2019-10-19T12:53:54.043Z", key[1]:value[1]},{"dateTime":"2019-10-19T12:53:54.043Z",key[2]:value[2]}
Such as the following:
{
"data": [
{
"dateTime": "2019-10-19T12:53:54.043Z",
"Reactor3.Level": 1.51860072870498
},
{
"dateTime": "2019-10-19T12:53:54.043Z",
"Reactor3.Temp": 27.1360543141452
},
{
"dateTime": "2019-10-19T12:55:54.043Z",
"Reactor3.Press": 88.9
},
{
"dateTime": "2019-10-19T12:55:54.043Z",
"Reactor3.Temp": 24.1418981047159
}
]
}
Where the first key will always be dateTime, the other keys will vary, and I'd like to break out all other keys/values by dateTime?
Here Reactor3.Level, Reactor3.Temp, Reactor3.Press are just examples that will change.
EDIT: In the following I am adding a more generic version of my problem.
I am essentially looking for a JSONata query to transform this input:
{
"data": [
{
"TS": "TS1",
"V1": 1,
"V2": 2
},
{
"TS": "TS2",
"V2": 9,
"V3": 8,
"V4": 7
}
]
}
where key "TS" is the first key in every set,
and the other keys will vary.
And transform it into this output:
{
"data": [
{
"TS": "TS1",
"V1": 1
},
{
"TS": "TS1",
"V2": 2
},
{
"TS": "TS2",
"V2": 9
},
{
"TS": "TS2",
"V3": 8
},
{
"TS": "TS2",
"V4": 7
}
]
}
[SOLVED] SteveR solved my problem below, I just want to add a Node-RED "payload" friendly version of the solution here:
Link: http://try.jsonata.org/HJCjoHxoS
payload modified JSONata:
payload.$ ~> | $ | { "data": data.(
$obj := $;
$key := "dateTime";
$keys($obj)[$ != $key].{
$key : $lookup($obj, $key),
$: $lookup($obj, $)
}
) } |
Thanks again to #SteveR

Interesting challenge, Jeff -- but if object properties do not have a guaranteed order, it may not be possible to do reliably... is it always the "first" property that needs to be distributed over the rest of them, or can you say for sure that it will always be called TS (using your simplified example)?
That being said, I did find an expression that seems to work, albeit only within my limited testing:
data.(
$obj := $;
$keys($obj)[[1..100]].(
$key := $keys($obj)[0];
{
$key : $lookup($obj, $key),
$: $lookup($obj, $)
}
)
)
Essentially, for each of the incoming objects, get its keys -- for each key with index between 1 - 100 (I know, it's an artificial limit), create the new object { key[0]: val[0], key[i]: val[i] }
You can try to tweak it here if you would like => http://try.jsonata.org/ryh3hqM5H
If you can say for sure that the first property is called "TS", I would use that named property to simplify the expression, like this:
data.(
$obj := $;
$key := "TS";
$keys($obj)[$ != $key].{
$key : $lookup($obj, $key),
$: $lookup($obj, $)
}
)
I'm sure there are better ways to accomplish the same transformation, but I hope this helps...
Good luck!
EDIT: I see that I lost the outer object with its data property. You can just wrap it around the expression(s) I listed, or if there are other properties that need to be maintained, use the object transform syntax to only modify that one data property:
$$ ~> | $ | { "data": data.(
$obj := $;
$key := "dateTime";
$keys($obj)[$ != $key].{
$key : $lookup($obj, $key),
$: $lookup($obj, $)
}
) } |

Related

improve leftJoin nested mapObject when equivalence do and do not match

I am using the following dataweave function, and it does works.
%dw 2.0
import * from dw::core::Arrays
output application/json
var mysqlInvoices = [
{
"id": 1,
"owner": "Joseph"
},
{
"id": 2,
"owner": "Maria"
}
]
var sapInvoices = [
{
"number": 3,
"issuedBy": "XYZ"
},
{
"number": 4,
"issuedBy": "ABC"
}
]
---
leftJoin(mysqlInvoices, sapInvoices, (m) -> m.id, (s) -> s.number) map (item, index) ->
(item.l mapObject (sItem, sKey) ->
(if ((sKey) as String == "id") "identifier"
else if ((sKey) as String == "owner") "ownerName"
else (sKey)): sItem)
++
(if (item.r != null)
item.r mapObject (sItem, sKey) ->
(sKey): sItem
else
sapInvoices[0] mapObject
(sItem, sKey) -> (sKey): "")
However, I am thinking if I can improve this function at two points:
change the key conditions:
I dont think that is the best practice to check every key match an if condition to change it:
(if ((sKey) as String == "id") "identifier"
else if ((sKey) as String == "owner") "ownerName"
else (sKey)): sItem
Use the original object to map it as an empty string when the leftJoin do not match keys:
sapInvoices[0] mapObject (sItem, sKey) ->
(sKey): ""
I am uncomfortable with these two points, and I believe that there are ways to improve this code, I just dont know how.
If there is a very different way of doing the same task, I also appreciate that kind of suggestion.
Based on George's answer, you can remove pluck and match and directly combine left and right table. See below:
%dw 2.0
import * from dw::core::Arrays
output application/json
var mysqlInvoices = [
{
"id": 1,
"owner": "Joseph"
},
{
"id": 2,
"owner": "Maria"
}
]
var sapInvoices = [
{
"number": 3,
"issuedBy": "XYZ"
},
{
"number": 4,
"issuedBy": "ABC"
}
]
var fs2rn = {
id: "identifier",
owner: "ownerName"
}
var rightEmpty= {number:"",issuedBy:""}
---
leftJoin(
// Do the field renaming at the very begining
mysqlInvoices map ($ mapObject {(fs2rn[$$] default $$): $}),
sapInvoices,
(m) -> m.identifier,
(s) -> s.number
) map (item) -> item.l ++ (item.r default rightEmpty)
Give the following a try, if anything the code seems a bit simpler:
%dw 2.0
import * from dw::core::Arrays
output application/json
var mysqlInvoices = [
{
"id": 1,
"owner": "Joseph"
},
{
"id": 2,
"owner": "Maria"
}
]
var sapInvoices = [
{
"number": 3,
"issuedBy": "XYZ"
},
{
"number": 4,
"issuedBy": "ABC"
}
]
var fs2rn = {
id: "identifier",
owner: "ownerName"
}
var rightEmpty= {number:"",issuedBy:""}
---
leftJoin(
// Do the field renaming at the very begining
mysqlInvoices map ($ mapObject {(fs2rn[$$] default $$): $}),
sapInvoices,
(m) -> m.identifier,
(s) -> s.number
)
// Iterate over the results
// Get just the values, and colapse the objects into a single object
map (
{($ pluck $)}
)
// Iterate over the results and use pattern-matching to
//
map (
$ match {
// Check if you have an id but not a number fields
// In which case add the rightEmpty object
case o if (o.identifier? and not (o.number?)) -> o ++ rightEmpty
// Or give the object because you now have both an id and a number
else o -> o
}
)
The features and functions I used are:
Dynamic Elements, documentation
pluck, documentation
Pattern-matching using the match operator, documentation
If I was to give you an advice, it would be to better indent your code. Nonetheless, pretty good job!

How to flatten nested object to single depth object with JSONata?

I am new to JSONata and am having some trouble to create a flatten function.
I want to turn this input:
{
"user": {
"key_value_map": {
"CreatedDate": "123424",
"Department": {
"Name": "XYZ"
}
}
}
}
Into this:
{
"user.key_value_map.CreatedDate": "123424",
"user.key_value_map.Department.Name": "XYZ"
}
Can anyone help me? Searched here and on google and couldn't find something that would lead me in the right direction.
Thanks
(
$fn := function($o, $prefix) {
$each($o, function($v, $k) {(
$name := $join([$prefix,$k], '.');
$type($v) = 'object' ? $fn($v, $name): {
$name: $v
}
)}) ~> $merge()
};
$fn($)
)
https://try.jsonata.org/WCUxC-r4_

Is there a more efficient way to refactor the iteration of the hash on ruby?

I have a iteration here:
container = []
summary_data.each do |_index, data|
container << data
end
The structure of the summary_data is listed below:
summary_data = {
"1" => { orders: { fees: '25.00' } },
"3" => { orders: { fees: '30.00' } },
"6" => { orders: { fees: '45.00' } }
}
I want to remove the numeric key, e.g., "1", "3".
And I expect to get the following container:
[
{
"orders": {
"fees": "25.00"
}
},
{
"orders": {
"fees": "30.00"
}
},
{
"orders": {
"fees": "45.00"
}
}
]
Is there a more efficient way to refactor the code above?
Appreciate for any help.
You can use Hash#values method, like this:
container = summary_data.values
If the inner hashes all have the same structure, the only interesting information are the fees:
summary_data.values.map{|h| h[:orders][:fees] }
# => ["25.00", "30.00", "45.00"]
If you want to do some calculations with those fees, you could convert them to numbers:
summary_data.values.map{|h| h[:orders][:fees].to_f }
# => [25.0, 30.0, 45.0]
It might be even better to work with cents as integers to avoid any floating point error:
summary_data.values.map{|h| (h[:orders][:fees].to_f * 100).round }
=> [2500, 3000, 4500]
You need an array having values of provided hash. You can get by values method directly.
summary_data.values

loopback REST API filter by nested data

I would like to filter from REST API by nested data. For example this object:
[
{
"name": "Handmade Soft Fish",
"tags": "Rubber, Rubber, Salad",
"categories": [
{
"name": "women",
"id": 2,
"parent_id": 0,
"permalink": "/women"
},
{
"name": "kids",
"id": 3,
"parent_id": 0,
"permalink": "/kids"
}
]
},
{
"name": "Tasty Rubber Soap",
"tags": "Granite, Granite, Chair",
"categories": [
{
"name": "kids",
"id": 3,
"parent_id": 0,
"permalink": "/kids"
}
]
}
]
is comming by GET /api/products?filter[include]=categories
and i would like to get only products which has category name "women". How do this?
LoopBack does not support filters based on related models.
This is a limitation that we have never had bandwidth to solve, unfortunately :(
For more details, see the discussion and linked issues here:
Filter on level 2 properties: https://github.com/strongloop/loopback/issues/517
Filter by properties of related models (use SQL JOIN in queries): https://github.com/strongloop/loopback/issues/683
Maybe you want to get this data by the Category REST API. For example:
GET /api/categories?filter[include]=products&filter[where][name]=woman
The result will be a category object with all products related. To this, will be necessary declare this relation on the models.
Try like this.It has worked for me.
const filter = {
where: {
'categories.name': {
inq: ['women']**strong text**
}
}
};
Pass this filter to request as path parameters and the request would be like bellow
GET /api/categoriesfilter=%7B%22where%22:%7B%categories.name%22:%7B%22inq%22:%5B%women%22%5D%7D%7D%7D
Can you share how it looks like without filter[include]=categorie, please ?
[edit]
after a few questions in comment, I'd build a remote method : in common/models/myModel.js (inside the function) :
function getItems(filter, categorieIds = []) {
return new Promise((resolve, reject) => {
let newInclude;
if (filter.hasOwnProperty(include)){
if (Array.isArray(filter.include)) {
newInclude = [].concat(filter.include, "categories")
}else{
if (filter.include.length > 0) {
newInclude = [].concat(filter.include, "categories");
}else{
newInclude = "categories";
}
}
}else{
newInclude = "categories";
}
myModel.find(Object.assign({}, filter, {include: newInclude}))
.then(data => {
if (data.length <= 0) return resolve(data);
if (categoriesIds.length <= 0) return resolve(data);
// there goes your specific filter on categories
const tmp = data.filter(
item => item.categories.findIndex(
categorie => categorieIds.indexOf(categorie.id) > -1
) > -1
);
return resolve(tmp);
})
}
}
myModel.remoteMethod('getItems', {
accepts: [{
arg: "filter",
type: "object",
required: true
}, {
arg: "categorieIds",
type: "array",
required: true
}],
returns: {arg: 'getItems', type: 'array'}
});
I hope it answers your question...

Rethinkdb insert query results into a table

I'm trying to insert the results of a query from one table into another table. However, when I attempt to run the query I am receiving an error.
{
"deleted": 0 ,
"errors": 1 ,
"first_error": "Expected type OBJECT but found ARRAY." ,
"inserted": 0 ,
"replaced": 0 ,
"skipped": 0 ,
"unchanged": 0
}
Here is the the insert and query:
r.db('test').table('destination').insert(
r.db('test').table('source').map(function(doc) {
var result = doc('result');
return result('section_list').concatMap(function(section) {
return section('section_content').map(function(item) {
return {
"code": item("code"),
"name": item("name"),
"foo": result("foo"),
"bar": result("bar"),
"baz": section("baz"),
"average": item("average"),
"lowerBound": item("from"),
"upperBound": item("to")
};
});
});
});
);
Is there a special syntax for this, or do I have to retrieve the results and then run a separate insert?
The problem is that your inner query is returning a stream of arrays. You can't insert arrays into a table (only objects), so the query fails. If you change the outermost map into a concatMap it should work.
The problem here was that the result was a sequence of an array of objects. i.e
[ [ { a:1, b:2 }, { a:1, b:2 } ], [ { a:2, b:3 } ] ]
Therefore, I had to change the outer map call to a concatMap call. The query then becomes:
r.db('test').table('destination').insert(
r.db('test').table('source').concatMap(function(doc) {
var result = doc('result');
return result('section_list').concatMap(function(section) {
return section('section_content').map(function(item) {
return {
"code": item("code"),
"name": item("name"),
"foo": result("foo"),
"bar": result("bar"),
"baz": section("baz"),
"average": item("average"),
"lowerBound": item("from"),
"upperBound": item("to")
};
)});
});
});
}
Thanks goes to #AtnNn on the #rethinkdb freenode for pointing me in the right direction.

Resources