Filter on aggregated bucket keys? - elasticsearch

Given data model structure like this,
{
Id: 123,
"string_facet": [
{
"name": "make",
"value": "Audi"
},
{
"name": "carListType",
"value": "PERSON EU"
},
{
"name": "modelType",
"value": ""
},
{
"name": "engineBrand",
"value": "APT"
},
{
"name": "typeDescription",
"value": "8D2"
}
],
"number_facet": [
{
"name": "typeNumber",
"value": 4614
},
{
"name": "serialNumber",
"value": 2
},
{
"name": "engineSize",
"value": 18
},
{
"name": "horsePower",
"value": 125
},
{
"name": "kw",
"value": 92
},
{
"name": "engineVolume",
"value": 1781
},
{
"name": "listType",
"value": 0
}
],
"dateTime_facet": [
{
"name": "fromDate",
"value": "1999-04-01T00:00:00"
},
{
"name": "toDate",
"value": "2000-10-01T00:00:00"
}
]
}
I want to get aggregates facet names, and values per name. However, I'm only interested in facets that have specific names, such as: make and engineBrand. Note that facets are of type nested.
I have tried the following .NEST expression, but it still returns all of the facet names.
.Global("global", g => g
.Aggregations(ag => ag
.Filter("global_makes", f => f
.Filter(ff => ff
.Nested(n => n
.Path("string_facet")
.Filter(pf => pf.Term("string_facet.name", "make")))
)
.Aggregations(agg => agg
.Nested("nested_string_facet", nested => nested
.Path("string_facet")
.Aggregations(stringFacet => stringFacet
.Terms("name", nameAgg => nameAgg.Field("string_facet.name").Size(0)
.Aggregations(nameAggNext => nameAggNext
.Terms("value", valueAgg => valueAgg.Field("string_facet.value").Size(0))
)
)
)
)
)
)
)
)
);
I have a filter within global (to lose scope of a passed in query), and then filter only on string_facet.name which match "make", but results still include all other names as well. How do I filter out aggregation to include only buckets where name is "make"?

This helped. https://github.com/elastic/elasticsearch/issues/4449
Essentially had to move filter part deeper into aggregation.

Related

Transform array of values to array of key value pair

I have a json data which is in the form of key and all values in a array but I need to transform it into a array of key value pairs, here is the data
Source data
"2022-08-30T06:58:56.573730Z": [
{ "tag": "AC 3 Phase/7957", "value": 161.37313113545272 },
{ "tag": "AC 3 Phase/7956", "value": 285.46869739695853 }
]
}
Transformation looking for
[
{ "tag": "AC 3 Phase/7957",
"ts": 2022-08-30T06:58:56.573730Z,
"value": 161.37313113545272
},
{ "tag": "AC 3 Phase/7956",
"ts": 2022-08-30T06:58:56.573730Z,
"value": 285.46869739695853
}
]
I would do it like this:
$each($$, function($entries, $ts) {
$entries.{
"tag": tag,
"ts": $ts,
"value": value
}
}) ~> $reduce($append, [])
Feel free to play with this example on the playground: https://stedi.link/g6qJGcP

Merge / flatten sub aggs into main agg

Is there away in elasticsearch to get the results back in a sort of flattend form (multiple child/sub aggs?
For instance currently i am trying to get back all product types and their status (online / offline).
This is what i end up with:
aggs
[
{ key: SuperProduct, doc_count:3, subagg:[
{status:online, doc_count:1},
{status:offline, doc_count:2}
]
},
{ key: SuperProduct2, doc_count:10, subagg:[
{status:online, doc_count:7},
{status:offline, doc_count:3}
]
Charting libraries tend to like it flattened so i was wondering if elasticsearch could probide it in this sort of manner:
[
{ products_key: 'SuperProduct', status_key:'online', doc_count:1},
{ products_key: 'SuperProduct', status_key:'offline', doc_count:2},
{ products_key: 'SuperProduct2', status_key:'online', doc_count:7},
{ products_key: 'SuperProduct2', status_key:'offline', doc_count:3}
]
Thanks
It is possible with composite aggregation which you can use to link two terms aggregations:
// POST /i/_search
{
"size": 0,
"aggregations": {
"distribution": {
"composite": {
"sources": [
{"product": {"terms": {"field": "product.keyword"}}},
{"status": {"terms": {"field": "status.keyword"}}}
]
}
}
}
}
This results in following structure:
{
"aggregations": {
"distribution": {
"after_key": {
"product": "B",
"status": "online"
},
"buckets": [
{
"key": {
"product": "A",
"status": "offline"
},
"doc_count": 3
},
{
"key": {
"product": "A",
"status": "online"
},
"doc_count": 2
},
{
"key": {
"product": "B",
"status": "offline"
},
"doc_count": 1
},
{
"key": {
"product": "B",
"status": "online"
},
"doc_count": 4
}
]
}
}
}
If for any reason composite aggregation doesn't fulfill your needs, you can create (via copy_to or by concatenation) or simulate (via scripted fields) field that would uniquely identify bucket. In our project we went with concatenation (partially for the necessity to collapse on this field), e.g. {"bucket": "SuperProductA:online"}, which results in dirtier output (you'll have to decode that field back or use top hits to get original values) but still does the job.

How to query array of objects as part of term query

I am using elasticsearch 5.5.0.
Im my index i have data of type attraction part of the json in elastic looks like:
"directions": "Exit the M4 at Junction 1",
"phoneNumber": "03333212001",
"website": "https://www.londoneye.com/",
"postCode": "SE1 7PB",
"categories": [
{
"id": "ce4cf4d0-6ddd-49fd-a8fe-3cbf7be9b61d",
"name": "Theater"
},
{
"id": "5fa1a3ce-fd5f-450f-92b7-2be6e3d0df90",
"name": "Family"
},
{
"id": "ed492986-b8a7-43c3-be3d-b17c4055bfa0",
"name": "Outdoors"
}
],
"genres": [],
"featuredImage": "https://www.daysoutguide.co.uk/media/1234/london-eye.jpg",
"images": [],
"region": "London",
My next query looks like:
var query2 = Query<Attraction>.Bool(
bq => bq.Filter(
fq => fq.Terms(t => t.Field(f => f.Region).Terms(request.Region.ToLower())),
fq => fq.Terms(t => t.Field(f => f.Categories).Terms(request.Category.ToLower())))
The query generated looks like:
{
"query": {
"bool": {
"filter": [
{
"terms": {
"region": [
"london"
]
}
},
{
"terms": {
"categories": [
"family"
]
}
}
]
}
}
}
That returns no results. If i take out the categories bit i get results. So i am trying to do term filter on categories which is an array of objects. Looks like I am doing this query wrong. Anyone any hints on how to get this to work?
Regards
Ismail
You can still use strongly typed properties access by using:
t.Field(f => f.Categories.First().Name)
NEST's property inferrer will reader will read over .First() and yield categories.name.
t.Field(f => f.Categories[0].Name) works as well.

elasticsearch nested array of objects - complex query

Ok, here is my problem:
At backend I have java spring boot, with mongodb and elasticsearch.
I have documents structured like this in my elasticsearch:
{ "id": 1,
"prefixes": [
{
"prefName": "RS",
"value": "c"
},
{
"prefName": "99c",
"value": "c"
},
{
"prefName": "RT",
"value": "a"
},
{
"prefName": "001b",
"value": "a"
},
{
"prefName": "DT",
"value": "m"
},
{
"prefName": "001c",
"value": "m"
},
{
"prefName": "001d",
"value": "b"
},
{
"prefName": "RN",
"value": "16"
},
{
"prefName": "001e",
"value": "16"
},
{
"prefName": "BN",
"value": "2-228-32590-2"
},
{
"prefName": "010a",
"value": "2-228-32590-2"
},
{
"prefName": "SS",
"value": "d"
},
{
"prefName": "100b",
"value": "d"
},
{
"prefName": "PY",
"value": "1975"
},
{
"prefName": "100c",
"value": "1975"
},
{
"prefName": "LA",
"value": "fre"
},
{
"prefName": "101a",
"value": "fre"
},
{
"prefName": "CO",
"value": "fra"
},
{
"prefName": "102a",
"value": "fra"
},
{
"prefName": "KW",
"value": "Ekole et psychologie individuelle comparee"
}] }
From client app I'm passing object with this structure:
{
"pref1": "(prefixName - value)",
"pref2": "(prefixName - value)",
"pref3": "(prefixName - value)",
"pref4": "(prefixName - value)",
"pref5": "(prefixName - value)",
"text1": "(prefix value - value)",
"text2": "(prefix value - value)",
"text3": "(prefix value - value)",
"text4": "(prefix value - value)",
"text5": "(prefix value - value)",
"operator1": "(logical OR,AND,NOT)",
"operator2": "(logical OR,AND,NOT)",
"operator3": "(logical OR,AND,NOT)",
"operator4": "(logical OR,AND,NOT)"
}
So, I have 5 optional prefixes to chose( pref1, pref2... for example RS, RT..),
text fields are their values and should be matched by phrase prefix. The problem is that I have 4 logical operators that links those queries. Can someone direct me how to structure adequate elasticsearch query to fulfill these needs.
Search form img
Here is the search form and the result should be documet which:
Has prefix AU - Sheakspeare* AND prefix 041a - 99c OR prefix RN - 23 AND prefix LA NOT 23.

Group array items with eloquent

I have the following eloquent query:
$extras = EventExtra::select('id', 'category', 'name', 'price', 'description', 'company')->get();
It gets some data from me from my database. What i want is for the returned data to be grouped twice, first by the category and then second by the company so that in the end i have something like this returned to the client:
[
{
"name": "donation",
"collection": [
{
"name": "sampleCompany1",
"array": [
{
"name": "extra1",
"description": "",
"value": ""
},
{
"name": "extra4",
"description": "",
"value": ""
},
{
"name": "extra6",
"description": "",
"value": ""
}
]
}
]
},
{
"name": "donation",
"collection": [
{
"name": "sampleCompany2",
"array": [
{
"name": "extra2",
"description": "",
"value": ""
},
{
"name": "extra3",
"description": "",
"value": ""
}
]
}
]
}]
I just typed the above myself so it might not be valid object array but basically it shows what i want to accomplish here.
You can use Collection to build your custom object. Something like this:
$return_data = Collect();
To add items in the collection with a property, you can use the put function.
$inner_data->put('name',$extras->name);
You can also add a collection within a collection.
To just push an existing collection in the collection, use push function
$inner_data->push($some_collection)
EDIT: Since you want a working example, see this below:
Lets say you want to create the following using collection:
{
"name": "extra1"
"description": "",
"value": ""
}
You will do something like this:
$my_collection = Collect();
$my_collection->put('name','extra1');
$my_collection->put('description','');
$my_collection->put('value','');
Now you can add this collection to another collection where you don't need a key. So lets say now it looks like this:
[
{
"name": "extra1"
"description": "",
"value": ""
},
{
"name": "extra4"
"description": "",
"value": ""
}
]
You will now do:
$my_final_collection = Collect();
foreach($my_collections as $my_collection) {
$my_final_collection->push($my_collection); // and so on in a loop
}

Resources