I just start with JSONata and I am really struggling to get my head around that syntax, can I ask you for help in understanding this example.
I have a list which consist of 2 other lists.
[
[
{
"part": "1300350",
"quantity": 334.93272
}
],
[
{
"part": "3305355",
"quantity": 20643
},
{
"part": "3305355",
"quantity": 1389
},
{
"part": "3305355",
"quantity": 11894
},
{
"part": "3305355",
"quantity": 12570
},
{
"part": "3305355",
"quantity": 16904
},
{
"part": "3305355",
"quantity": 4136
}
]
]
What I would like to achieve is to get the output as a sum for given part number, so:
[
{
"part": "1300350",
"total_qty": 334.93272
},
{
"part": "3305355",
"total_qty": 67536
}
]
Also do you know if there are some good training platforms for JSONata with some exercises and examples so one can get comfortable to use this language for querying data?
Related
Due to a (somehow) special requirement to store data alongside a FHIR resource, I am looking for the cleanest way to extend it to maintain such extra data, which is basically dynamic key-value lists.
On the one hand I have something like the following, with 0..* sets of key-value, that I have tried to store nesting three extensions (adjusting the context by level) being the last a single limitation to the accepted types (string/integer), although it does not allow me to repeat ids, anyway I am getting around it using prefixes XD.
"n-lists-of-key-value": [
{
"list1-key1": "list1-value1",
"list1-key2": "list1-value1",
"list1-key3": "list1-value1"
},
{
"list2-key1": "list2-value1",
"list2-key2": "list2-value2"
}
]
becomes:
···,
"extension": [ {
"id": "n-lists-of-key-value",
"url": "http://x.org/fhir/Extension/NListOfKeyValueExtension",
"extension": [ {
"id": "list_1",
"url": "http://x.org/fhir/Extension/ListKeyValueExtension,
"extension": [ {
"id": "list1-key1",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list1-value1"
}, {
"id": "list1-key2",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list1-value1"
}, {
"id": "list1-key3",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list1-value1"
} ]
}, {
"id": "list_2",
"url": "http://x.org/fhir/Extension/ListKeyValueExtension,
"extension": [ {
"id": "list2-key1",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list2-value1"
}, {
"id": "list2-key2",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list2-value2"
} ]
} ]
} ],
But on the other hand I have this, which is what I find most difficult to model, because trying to reuse the extension definitions I run into (conceptual?) difficulties with slicing and the use of the discriminators, failing to get past the error "Element matches more than one slice".
"my-3-fixed-named-sets": {
"fixed-named-set1": {
"list1-key1": "list1-value1",
"list1-key2": "list1-value2",
"list1-key3": "list1-value3"
},
"fixed-named-set2": {
"list2-key1": "list2-value1",
"list2-key2": "list2-value2"
},
"fixed-named-set3": {
"list2-key1": "list2-value1",
"list2-key2": "list2-value2"
}
}
becomes:
···,
"extension": [ {
"id": "my-3-fixed-named-sets",
"url": "http://x.org/fhir/Extension/NListOfKeyValueExtension",
"extension": [ {
"id": "fixed-named-set1",
"url": "http://x.org/fhir/Extension/ListKeyValueExtension,
"extension": [ {
"id": "list1-key1",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list1-value1"
}, {
"id": "list1-key2",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list1-value2"
}, {
"id": "list1-key3",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list1-value3"
} ]
}, {
"id": "fixed-named-set2",
"url": "http://x.org/fhir/Extension/ListKeyValueExtension,
"extension": [ {
"id": "list2-key1",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list2-value1"
}, {
"id": "list2-key2",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list2-value2"
} ],
}, {
"id": "fixed-named-set3",
"url": "http://x.org/fhir/Extension/ListKeyValueExtension,
"extension": [ {
"id": "list2-key1",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list2-value1"
}, {
"id": "list2-key3",
"url": "http://x.org/fhir/Extension/KeyValueExtension,
"valueString": "list2-value3"
} ]
} ]
} ],
I am aware that by indiscriminately including new extensions with different names/urls, I can end up ingesting all the data, but I suspect that there is a much cleaner way and that I am failing in , probably, the way to include an extra condition into the discriminator (https://hl7.org/fhir/profiling.html#slicing), so that I could point to the proper slide? or de fact of extending extensions or extending value[x] slices?
"discriminator": [
{
"type": "value",
"path": "url"
},
{ "???": "??Extension.value.slideName??
···
Then my specific question would be... can I slice the value[x] of the extension of a specific profile, and which in turn make them extensions? this lasy is to get having n key value lists
Any comment is very welcome :)
I want to sort objects in ascending order but the sort doesn't work.
Here is a sort query below.
"sort":[
{
"category.position": {
"order":"asc",
"mode":"min",
"nested": {
"path": "category",
"filter": {
"term": {"category_category_id":42} }
}
}
}]
And here are the objects below.
"name": "Yeti",
"category": [
{
"category_id": 42,
"name": "Raamiga",
"position": 3
},
],
"name": "Venus",
"category": [
{
"category_id": 42,
"name": "Raamiga",
"position": 4
}
],
Please, help! Many thanks in advance!
Solved. There was a typo… Must be "category.category_id" indtead of "category_category_id".
I found an interesting article that used several data models on Vega-Lite. Tabular data were combined by key like in relational databases.
{
"$schema": "https://vega.github.io/schema/vega-lite/v2.json",
"title": "Test",
"datasets": {
"stores": [
{"cmdb_id1": 1, "group": "type1"},
{"cmdb_id1": 2, "group": "type1"},
{"cmdb_id1": 3, "group": "type2"}
],
"labelsTimelines": [
{"cmdb_id2": 1, "value": 83},
{"cmdb_id2": 2, "value": 53},
{"cmdb_id2": 3, "value": 23}
]
},
"data": {"name": "stores"},
"transform": [
{
"lookup": "cmdb_id1",
"from": {
"data": {"name": "labelsTimelines"},
"key": "cmdb_id2",
"fields": ["value"]
}
}
],
"mark": "bar",
"encoding": {
"y": {"aggregate": "sum", "field": "value", "type": "quantitative"},
"x": {"field": "group", "type": "ordinal"}
}
}
Vega Editor
The question arose as to whether it was possible to obtain the same result using the construction:
"data": {"url": "...."}
Changed the source for Elasticsearch query:
{
"$schema": "https://vega.github.io/schema/vega-lite/v3.json",
"datasets": {
"stores": [{
"url": {
"%context%": "true"
"index": "test_cmdb"
"body": {
"size": 1000,
"_source": ["cmdb_id", "street","group"]
}
}
format: {property: "hits.hits"}
}]}
"data": {
"name": "stores"
},
"encoding": {
"x": {"field": "url.body.size", "type": "ordinal", "title": "X"},
"y": {"field": "url.body.size", "type": "ordinal", "title": "Y"}
},
"layer": [
{
"mark": "rect",
"encoding": {
"tooltip": [
{"field": "url"}]
}
}
]
}
I understand that there is a syntactical error, the data did not come from Elasticsearch.
Thanks in advance!
example.png
No, it is not currently possible to specify URL data within top-level "datasets". The relevant open feature request in Vega-Lite is here: https://github.com/vega/vega-lite/issues/4004.
Your much better off using Vega rather than Vega-lite for this. In Vega you can specify as many datasets as you like with a URL. For example...
...
data: [
{
name: dataset_1
url: {
...
}
}
{
name: dataset_2
url: {
...
}
}
]
...
This can actually get very interesting since it means you can combine data from multiple indices into one visualisation.
I know this is late, but figured this might help people who are looking around.
How do I search for a person with BOTH given names I provide?
I have the following 2 patients who are "close". Everything (in the Human Name area) is the same except one of the GivenNames are the same.
Note "Apple" vs "Banana".
{
"resourceType": "Bundle",
"id": "269caf66-0ccc-43e7-b9a5-f16f84db0149",
"meta": {
"lastUpdated": "2019-11-20T19:30:26.858917+00:00"
},
"type": "searchset",
"link": [
{
"relation": "self",
"url": "https://localhost:44348/Patient?given=Jingerheimer"
}
],
"entry": [
{
"fullUrl": "https://localhost:44348/Patient/504f6bd3-e9b4-4846-8948-97bf09c70722",
"resource": {
"resourceType": "Patient",
"id": "504f6bd3-e9b4-4846-8948-97bf09c70722",
"meta": {
"versionId": "1",
"lastUpdated": "2019-11-20T19:26:11.005+00:00"
},
"identifier": [
{
"system": "ssn",
"value": "111-11-1111"
},
{
"system": "uuid",
"value": "da55d068e0784b359fa97498a11543c5"
}
],
"name": [
{
"family": "Smith",
"given": [
"John",
"Apple",
"Jingerheimer"
]
}
]
},
"search": {
"mode": "match"
}
},
{
"fullUrl": "https://localhost:44348/Patient/10054ce9-6141-4eca-bc5b-0978f8c8afcb",
"resource": {
"resourceType": "Patient",
"id": "10054ce9-6141-4eca-bc5b-0978f8c8afcb",
"meta": {
"versionId": "1",
"lastUpdated": "2019-11-20T19:26:48.962+00:00"
},
"identifier": [
{
"system": "ssn",
"value": "222-22-2222"
},
{
"system": "uuid",
"value": "52d09f9436d44591816fd229dd139523"
}
],
"name": [
{
"family": "Smith",
"given": [
"John",
"Banana",
"Jingerheimer"
]
}
]
},
"search": {
"mode": "match"
}
}
]
}
One has GivenNames that include "Apple". The other includes GivenNames that include "Banana".
This search works fine:
https://localhost:44348/Patient/?given=Jingerheimer
What I have tried is:
https://localhost:44348/Patient/?given=Jingerheimer&given=Apple
but that gives me no results.
Note, omitting "given=Jingerheimer" is not an option....that filters a bunch of others.
I'm trying to get
"Has BOTH of the given names I provide"
Your syntax is correct, so I think the server does not handle the search correctly. Can you check the self link for your second search to see if it reflects the search you performed? Does the result Bundle have an OperationOutcome detailing something went wrong? If all that seems okay, you'll need to check your server's code.
I have documents in ElasticSearch with the following structure :
"_source": {
"last_updated": "2017-10-25T18:33:51.434706",
"country": "Italia",
"price": [
"€ 139",
"€ 125",
"€ 120",
"€ 108"
],
"max_occupancy": [
2,
2,
1,
1
],
"type": [
"Type 1",
"Type 1 - (Tag)",
"Type 2",
"Type 2 (Tag)",
],
"availability": [
10,
10,
10,
10
],
"size": [
"26 m²",
"35 m²",
"47 m²",
"31 m²"
]
}
}
Basically, the details records are split in 5 arrays, and fields of the same record have the same index position in the 5 arrays. As can be seen in the example data there are 5 array(price, max_occupancy, type, availability, size) that are containing values related to the same element. I want to extract the element that has max_occupancy field greater or equal than 2 (if there is no record with 2 grab a 3 if there is no 3 grab a four, ...), with the lower price, in this case the record and place the result into a new JSON object like the following :
{
"last_updated": "2017-10-25T18:33:51.434706",
"country": "Italia",
"price: ": "€ 125",
"max_occupancy": "2",
"type": "Type 1 - (Tag)",
"availability": 10,
"size": "35 m²"
}
Basically the result structure should show the extracted record(that in this case is the second index of all array), and add the general information to it(fields : "last_updated", "country").
Is it possible to extract such a result from elastic search? What kind of query do I need to perform?
Could someone suggest the best approach?
My best approach: go nested with Nested Datatype
Except for easier querying, it easier to read and understand the connections between those objects that are, currently, scattered in different arrays.
Yes, if you'll decide this approach you will have to edit your mapping and re-index your entire data.
How would the mapping is going to look like? something like this:
{
"mappings": {
"properties": {
"last_updated": {
"type": "date"
},
"country": {
"type": "string"
},
"records": {
"type": "nested",
"properties": {
"price": {
"type": "string"
},
"max_occupancy": {
"type": "long"
},
"type": {
"type": "string"
},
"availability": {
"type": "long"
},
"size": {
"type": "string"
}
}
}
}
}
}
EDIT: New document structure (containing nested documents) -
{
"last_updated": "2017-10-25T18:33:51.434706",
"country": "Italia",
"records": [
{
"price": "€ 139",
"max_occupancy": 2,
"type": "Type 1",
"availability": 10,
"size": "26 m²"
},
{
"price": "€ 125",
"max_occupancy": 2,
"type": "Type 1 - (Tag)",
"availability": 10,
"size": "35 m²"
},
{
"price": "€ 120",
"max_occupancy": 1,
"type": "Type 2",
"availability": 10,
"size": "47 m²"
},
{
"price": "€ 108",
"max_occupancy": 1,
"type": "Type 2 (Tag)",
"availability": 10,
"size": "31 m²"
}
]
}
Now, its more easy to query for any specific condition with Nested Query and Inner Hits. for example:
{
"_source": [
"last_updated",
"country"
],
"query": {
"bool": {
"must": [
{
"term": {
"country": "Italia"
}
},
{
"nested": {
"path": "records",
"query": {
"bool": {
"must": [
{
"range": {
"records.max_occupancy": {
"gte": 2
}
}
}
]
}
},
"inner_hits": {
"sort": {
"records.price": "asc"
},
"size": 1
}
}
}
]
}
}
}
Conditions are: Italia AND max_occupancy > 2.
Inner hits: sort by price ascending order and get the first result.
Hope you'll find it useful