I have my json as follows
{
"cluster": [
{
"id": "cluster1.1",
"color": "blue",
"segment": [
{
"id": "segment1.1",
"color": "green"
}
]
},
{
"id": "cluster1.2",
"color": [
"blue",
"red"
],
"segment": [
{
"id": "segment1.2",
"color": "Yellow"
}
]
},
{
"id": "cluster1.3",
"color": "Orange",
"segment": [
{
"id": "cluster1.3",
"color": "black"
},
{
"id": "cluster1.4",
"color": "Green"
},
{
"id": "cluster1.5",
"color": "red"
}
]
},
{
"id": "cluster1.4",
"color": [
"blue",
"red"
],
"segment": [
{
"id": "cluster1.4",
"color": "red"
},
{
"id": "cluster1.5",
"color": "blue"
},
{
"id": "cluster1.6",
"color": "Yellow"
}
]
}
]
}
I would like to loop this recursively through all nodes, I am getting the content as follows with the following code but I am not getting through all the nodes
$jsonData = (Get-Content -FilePath) -join "`n" | ConvertFrom-Json
for( $i=0; $i -lt $jsonData.cluster.Length; $i++)
{
$clusterInfo= $ReportingPackage.cluster[$i]
$clusterInfo.Color
}
I need to recursively find a way to loop through all segments and colors
Array.ElementProperty shorthand fetches the properties only for the immediate elements of the array.
Enumerate the sub-elements' properties manually:
ForEach ($cluster in $jsonData.cluster) {
$cluster.color
$cluster.segment.color
}
You may want to use a sanity check: if ($cluster.segment) { $cluster.segment.color }
To collect all colors in an array the simplest method is piping:
$allColors = $jsonData.cluster | ForEach {
$_.color
$_.segment.color
}
Related
I need to search in array of ElasticSearch. I've documents like
{
"product_name": "iPhone 9",
"features":[
{
"color": "black",
"memory": "128GB"
},
{
"color": "white",
"memory": "64GB"
}
],
},
{
"product_name": "iPhone 9",
"features":[
{
"color": "black",
"memory": "64GB"
},
{
"color": "white",
"memory": "64GB"
}
],
}
I want to search iphone 9 with color = black and memory = 64GB. I'm using following query
_search?q=product_name:"iPhone 9"+AND+features.color:"black"+AND+features.memory:"64GB"
Only the second record from the document should get listed, but this query is displaying both the records as it matches color with first array and memory with second array. How can I achieve the correct result?
Elasticsearch has no concept of inner objects. Therefore, it flattens object hierarchies into a simple list of field names and values.
Your document will be transformed internally and stored as
{
"product_name" : "iPhone 9",
"features.color" : [ "black", "white" ],
"features.memory" : [ "128GB", "64GB" ]
}
The associate between color and memory is lost.
If you need to maintain independence of each inner object of array , you need to use nested type
Nested type can be only queried using nested query.
PUT index-name
{
"mappings": {
"properties": {
"features": {
"type": "nested"
}
}
}
}
PUT index-name/_doc/1
{
"product_name": "iPhone 9",
"features":[
{
"color": "black",
"memory": "128GB"
},
{
"color": "white",
"memory": "64GB"
}
],
}
GET index-name/_search
{
"query": {
"nested": {
"path": "features",
"query": {
"bool": {
"must": [
{ "match": { "features.color": "black" }},
{ "match": { "features.memory": "64GB" }}
]
}
}
}
}
}
I am new to the mongoDB aggregation and I have this situation. I have this Json and I need to convert by "select" this object:
{
"type": "PF",
"code": 12345
"Name": Darth Vader,
"currency": "BRL",
"status": "SINGLE",
"adress": [
{
"localization": "DEATH STAR",
"createDate": 1627990848665
},
{
"localization": "TATOOINE",
"createDate": 1627990555665
},
]
}
this way:
{
"type": "PF",
"code": 12345
"Name": Darth Vader,
"currency": "BRL",
"status": "SINGLE",
"localization": "DEATH STAR",
"createDate": 1627990848665
},
{
"type": "PF",
"code": 12345
"Name": Darth Vader,
"currency": "BRL",
"status": "SINGLE",
"localization": "TATOOINE",
"createDate": 1627990555665
}
So, after my query is complete, I will have 02 objects instead of 01. How can I do this?
I would like to do this via select because after converting I will sort by createDate and limit the number of records to return to the API. I'm using Criteria em my project.
The way to do this is $unwind, this will make 1 copy of the document, for each member of the array.
Test code here
db.collection.aggregate([
{
"$unwind": {
"path": "$user.adress"
}
},
{
"$set": {
"user": {
"$mergeObjects": [
"$user",
"$user.adress"
]
}
}
},
{
"$unset": [
"user.adress"
]
},
{
"$sort": {
"createDate": 1
}
},
{
"$limit": 10
}
])
Edit1 (the above is if user is a field, if it was the name of the collection)
$$ROOT is a system variable that has as value all the document
Test code here
Query
db.collection.aggregate([
{
"$unwind": {
"path": "$adress"
}
},
{
"$replaceRoot": {
"newRoot": {
"$mergeObjects": [
"$$ROOT",
"$adress"
]
}
}
},
{
"$unset": [
"adress"
]
},
{
"$sort": {
"createDate": 1
}
},
{
"$limit": 10
}
])
Imagine that I have an index with the following three documents representing images and their colors.
[
{
"id": 1,
"intensity": {
"red": 0.6,
"green": 0.1,
"blue": 0.3
}
},
{
"id": 2,
"intensity": {
"red": 0.5,
"green": 0.6,
"blue": 0.0
}
},
{
"id": 3,
"intensity": {
"red": 0.98,
"green": 0.0,
"blue": 0.0
}
}
]
It the user wants a "red image" (selected in a dropdown or in a “tag cloud”), it is very convenient to do a range query over the floats (maybe intensity.red > 0.5). I can also use the score of that query to get the "red-est" image ranked highest.
However, if I would like to offer free-text search, it gets harder. My solution to that would to index the documents as the following (eg use the if color > 0.5 then append(colors, color_name) at index time):
[
{
"id": 1,
"colors": ["red"]
},
{
"id": 2,
"colors": ["green", "red"]
}
{
"id": 3,
"colors": ["red"]
}
]
I could now use a query_string or a match on the colors field and then search for "red", but all of a sudden I lost my ranking possibilities. ID 3 is far more red than ID 1 (0.98 vs 0.6) but the score will be similar?
My question is: Can I have the cake and eat it too?
One solution I see is to have one index that turns free text into "keywords" which I later use in the actual search.
POST image_tag_index/_search {query: "redish"} -> [ "red" ]
POST images/_search {query: {"red" > 0.5}} -> [ {id: 1}, {id: 3}]
But then I need to fire two searches for every search, but maybe that is the only option?
You can make use of nested data type along with function_score query to get the desired result.
You need to change the way you are storing image data. The mapping will be as below:
PUT test
{
"mappings": {
"_doc": {
"properties": {
"id": {
"type": "integer"
},
"image": {
"type": "nested",
"properties": {
"color": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"intensity": {
"type": "float"
}
}
}
}
}
}
}
Index the image data as below:
PUT test/_doc/1
{
"id": 1,
"image": [
{
"color": "red",
"intensity": 0.6
},
{
"color": "green",
"intensity": 0.1
},
{
"color": "blue",
"intensity": 0.3
}
]
}
The above corresponds to the first image data the you posted in the question. Similarly you can index other images data.
Now when a user search for red the query should be build as below:
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "image",
"query": {
"function_score": {
"query": {
"bool": {
"must": [
{
"match": {
"image.color": "red"
}
},
{
"range": {
"image.intensity": {
"gt": 0.5
}
}
}
]
}
},
"field_value_factor": {
"field": "image.intensity",
"modifier": "none",
"missing": 0
}
}
}
}
}
]
}
}
}
You can see in the above query that I have used the field value of image.intensity to calculate the score.
I am trying to perform conditional mapping of values from the following JSON.
My input,
{
"rating": [
{
"id": 1,
"locations": [
{
"num": 1
},
{
"num": 2
}
]
}
]
}
Expected output:
{
"rating": [
{
"id": 1,
"locations": [
{
"num": 1
}
],
"new_locations": [
{
"num": 2
}
]
}
]
}
My spec,
[
{
"operation": "shift",
"spec": {
"rating": {
"*": {
"locations": {
"*": {
"num": "#(3,id)"
}
}
}
}
}
}
]
If the num value matches with id,then it should stay in location array else should be moved to new_locations.
Can anyone please suggest me help.Thanks.
There isn't a way to do that kind of conditional matching logic with the "out of the box" Jolt transforms.
Shortly: when a field has multiple values, how can I get only those items where both my filter applies to the SAME value in a multiple-values field?
Details
I have stored in Elasticsearch some items which have a nested field with multiple values, e.g.
"hits": [
{
"name": "John",
"tickets": [
{
"color": "green",
"code": "001"
},
{
"color": "red",
"code": "002"
}
]
},
{
"name": "Frank",
"tickets": [
{
"color": "red",
"code": "001"
},
{
"color": "green",
"code": "002"
}
]
}
]
Now consider these filters:
...
filter: [
{ terms: { 'tickets.code': '001' } },
{ terms: { 'tickets.color': 'green' } },
]
...
Both items match, because each one of them has at least a ticket with code "001" and each one of them has ticket with color "green".
How do I write my filters so that only the first match, because it has a ticket which has code "001" AND color "green"?
Thank you in advance for any suggestion.
Your problem is caused by the fact that Elasticsearch flattens objects. So internally, your data is represented something like this:
{
"name": "John",
"tickets.color": ["green", "red"],
"tickets.code": ["001", "002"]
},
{
"name": "Frank",
"tickets.color": ["red", "green"],
"tickets.code": ["001", "002"]
}
It's impossible to know which color and code are on the same object. (The original source is also stored, in order to be returned when you make a request, but that's not the data that's queried when you search.)
There are two potential solutions here: denormalization, or nested data type. If you can at all get away with it, denormalization is the better choice here, because it's more efficient. If you denormalize your data, you might end up with a representation like this:
{
"name": "John",
"ticket": {
"color": "green",
"code": "001"
}
},
{
"name": "John",
"ticket": {
"color": "red",
"code": "002"
}
},
{
"name": "Frank",
"ticket": {
"color": "red",
"code": "001"
}
},
{
"name": , "Frank",
"ticket": {
"color": "green",
"code": "002"
}
}
If you use a nested data type, you'll have to use a mapping something like this:
{
"ticket": {
"type": "nested",
"properties": {
"color": {"type": "keyword"},
"code": {"type": "keyword"}
}
}
}