Elasticsearch Query - how to? - elasticsearch

I have the data in the following format in Elastic Search (from sense)
POST slots/slot/1
{
locationid:"1",
roomid:"10",
starttime: "08:45"
}
POST slots/slot/2
{
locationid:"1",
roomid:"10",
starttime: "09:00"
}
POST slots/slot/3
{
locationid:"2",
roomid:"100",
starttime: "08:45"
}
POST slots/slot/4
{
locationid:"2",
roomid:"101",
starttime: "09:00"
}
POST slots/slot/5
{
locationid:"3",
roomid:"200",
starttime: "09:30"
}
In short , the data is in the following format.
A Location has multiple rooms and each room has multiple slots of 15 minutes. So slot 1 for Room10 starts at 8:45 and ends at 09:00, Slot 2 for same room starts at 09:00 and ends at 09:15
Locationid RoomId Starttime
--------------------------------------
1 10 08:45
1 10 09:00
2 100 08:45
2 101 09:00
3 200 09:30
Im trying to write a query/filter which will give me all locations where a room is available with two or three slots.
For e.g Find a location that has 08:45 slot and 09:00 slot (configurable)
Answer should be location 1 only
Should Not be location 2 as room 100 has 08:45 slot but not the 09:00 slot. Room 101 has 09:00 slot but doesnt have the 08:45 slot

I believe this is not the best approach , but my attempt for the answer
POST slots/slot/_search?pretty=true&search_type=count
{
"facets": {
"locationswithslots": {
"terms": {
"field": "locationid",
"script" : "term + \"_\" + _source.roomid",
"size": 10
},
"facet_filter":
{
"terms":
{
"starttime":
[
"08:45",
"09:00"
]
}
}
}
}
}
This gives the answer as below
{
"took": 12,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 5,
"max_score": 0,
"hits": []
},
"facets": {
"locationswithslots": {
"_type": "terms",
"missing": 0,
"total": 4,
"other": 0,
"terms": [
{
"term": "1_10",
"count": 2
},
{
"term": "2_101",
"count": 1
},
{
"term": "2_100",
"count": 1
}
]
}
}
}
Now I need to figure out a way to filter the facets that return count 2 as I passed in 2 slots in the filter.
Any other option possible?

Related

Can't get severity info via API

Java 11
SonarQube 8.9.2 LTS
For my java project the SonarQube show the next issues info:
Severity
Blocker 1.3k
Minor 1.1k
Critical 5.8k
Info 233
Major 1.3k
So I need to get this information via SonarQube WEB API.
I found only this api method:
GET http://some_url_sonar_qube/api/issues/search
And its return all issues on page = 1
And its return all issues on page = 1 with detail info
{
"total": 10049,
"p": 1,
"ps": 100,
"paging": {
"pageIndex": 1,
"pageSize": 100,
"total": 10049
},
"effortTotal": 50995,
"issues": [
{
"key": "dddd",
"rule": "css:S4670",
"severity": "CRITICAL",
...
This:
GET http://some_url_sonar_qube/api/issues/search?p=2
And its return all issues on page = 2
and so on.
Response example:
As you can see has 10049 issues. It's 100 pages.
But I need summary info. Smt like this in json format:
{
"Severity": {
"Blocker": 1300,
"Minor": 1100,
"Critical": 5800,
"Info": 233,
"Major": 1300
}
}
I'm not found api method for this
I found solution (thanks for #gawkface)
Use this method:
GET http://some_url_sonar_qube/api/issues/search?componentKeys=my_project_key&facets=severities
And here result (on section facets)
{
"total": 10049,
"p": 1,
"ps": 100,
"paging": {
"pageIndex": 1,
"pageSize": 100,
"total": 10049
},
"effortTotal": 50995,
"issues": [...],
"components": [...],
"facets": [
{
"property": "severities",
"values": [
{
"val": "CRITICAL",
"count": 5817
},
{
"val": "MAJOR",
"count": 1454
},
{
"val": "BLOCKER",
"count": 1286
},
{
"val": "MINOR",
"count": 1161
},
{
"val": "INFO",
"count": 331
}
]
}
]
}

How get a distinct list of document fields using NEST?

I have just started with Elasticsearch and am using the NEST API for my .Net application. I have an index and some records inserted. I am now trying to get a distinct list of document field values. I have this working in Postman. I do not know how to port the JSON aggregation body to a NEST call. Here is the call I am trying to port to the NEST C# API:
{
"size": 0,
"aggs": {
"hosts": {
"terms": {
"field": "host"
}
}
}
Here is the result which is my next question. How would I parse or assign a POCO to the result? I am only interested in the distinct list of the field value in this case 'host'. I really just want an enumerable of strings back. I do not care about the count at this point.
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": null,
"hits": []
},
"aggregations": {
"hosts": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "hoyt",
"doc_count": 3
}
]
}
}
}
I was able to get the results I am after with the following code:
var result = await client.SearchAsync<SyslogEntryIndex>(s => s.Size(0).Aggregations(a => a.Terms("hosts", t => t.Field(f => f.Host))));
List<string> hosts = new List<string>();
foreach (BucketAggregate v in result.Aggregations.Values)
{
foreach (KeyedBucket<object> item in v.Items)
{
hosts.Add((string)item.Key);
}
}
return hosts;

No results from search when passing more than one parameter in user metadata

I want to apply document level security in elastic, but once I provide more than one value in user metadata I get no matches.
I am creating a role and a user in elastic and passing values inside user metadata to the role on whose basis the search should happen. It works fine if I give one value.
For creating role:
PUT _xpack/security/role/my_policy
{
"indices": [{
"names": ["my_index"],
"privileges": ["read"],
"query": {
"template": {
"source": "{\"bool\": {\"filter\": [{\"terms_set\": {\"country_name\": {\"terms\": {{#toJson}}_user.metadata.country_name{{/toJson}},\"minimum_should_match_script\":{\"source\":\"params.num_terms\"}}}}]}}"
}
}
}]
}
And for user:
PUT _xpack/security/user/jack_black
{
"username": "jack_black",
"password":"testtest",
"roles": ["my_policy"],
"full_name": "Jack Black"
"email": "jb#tenaciousd.com",
"metadata": {
"country_name": ["india" , "japan"]
}
}
I expect the output to be results for india and japan only. If the user searches for anything else they should get no results.
However, I do not see any results at all:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}

Possible to get the size_in_bytes for records matching a specific query?

The documentation on the stats api indicates that we can do the following:
http://es.cluster.ip.addr:9200/indexname/_stats
Which resuls in an output like:
{
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"_all": {
"primaries": {
"docs": {
"count": 32930,
"deleted": 0
},
"store": {
"size_in_bytes": 3197332,
"throttle_time_in_millis": 0
},
// ... etc
}
}
}
My question is, is there a way to obtain the file size for a specific set of records, specific such as when we run a search query:
http://es.cluster.ip.addr:9200/indexname/type/_search?q=identifier:123
So essentially, the size_in_bytes for all records matching the identifier 123?

Compute difference between field and aggregated field

I have to run complex aggregation and one of its steps is computing sum of sold_qty field, and then I need to subtract this sum with non aggregated field all_qty. My data looks like:
{item_id: XXX, sold_qty: 1, all_qty: 20, price: 100 }
{item_id: XXX, sold_qty: 3, all_qty: 20, price: 100 }
{item_id: YYY, sold_qty: 1, all_qty: 20, price: 80 }
These are transactions from offer. The all_qty and price fields are redundant - express single values from other structure - offers and just duplicated in all transactions from single offer (identified by item_id).
In the terms of SQL what I need is:
SELECT (all_qty - sum(sold_qty)) * price GROUP BY item_id
What I've done is aggregation
'{
"query": {"term": {"seller": 9059247}},
"size": 0,
"aggs": {
"group_by_offer": {
"terms": { "field": "item_id", size: 0},
"aggs": { "sold_sum": {"sum": {"field": "sold_qty"}}}
}
}
}'
But I don't know what to do next to achieve my goal.
Since you are already storing redundant fields, if I were you, I would also store the result of all_price = all_qty * price and sold_price = sold_qty * price. It's is not mandatory but it will be faster at execution time than executing scripts to make the same computation.
{item_id: XXX, sold_qty: 1, sold_price: 20, all_qty: 20, price: 100, all_price: 2000 }
{item_id: XXX, sold_qty: 3, sold_price: 300, all_qty: 20, price: 100, all_price: 2000 }
{item_id: YYY, sold_qty: 1, sold_price: 80, all_qty: 20, price: 80, all_price: 1600 }
All you'd have to do next is to sum sold_price and average all_price and simply get the difference between both using a bucket_script pipeline aggregation:
{
"query": {
"term": {
"seller": 9059247
}
},
"size": 0,
"aggs": {
"group_by_offer": {
"terms": {
"field": "item_id",
"size": 0
},
"aggs": {
"sold_sum": {
"sum": {
"field": "sold_price"
}
},
"all_sum": {
"avg": {
"field": "all_price"
}
},
"diff": {
"bucket_script": {
"buckets_path": {
"sold": "sold_sum",
"all": "all_sum"
},
"script": "params.all - params.sold"
}
}
}
}
}
}

Resources