I'm trying to query an exact date in elasticsearch and when I submit, I received date that is not part of the query. I have this code
{
"query": {
"bool": {
"must": [
{
"match": {
"device.id": 1374
}
},
{
"match": {
"readingType.id": 1048
}
},
{
"match": {
"created": "2022-11-16"
}
}
]
}
},
"size": 1000,
"from": 0,
"sort": [
{
"created": {
"order": "desc"
}
}
]
}
so I try search date 2022-11-16 and I have this data
May I know why the elasticsearch add 2022-11-17 to the list? I already checked the documentation but I failed to see the issue of why I have this list. I all I want are the data for 2022-11-16
I have this setup for my created column.
{
"list": {
"aliases": {},
"mappings": {
"reading": {
"_meta": {
"model": "App\\Entity\\List"
},
"dynamic_date_formats": [],
"properties": {
"created": {
"type": "date"
},
"device": {
"properties": {
"id": {
"type": "long"
}
}
},
To query an exact date you need to do it like this:
{
"range": {
"created": {
"gte": "2022-11-16",
"lte": "2022-11-16"
}
}
}
Related
I am new on Elastic Search. I really need the result about calculating the difference of two set.
Here is the mapping of a index:
{
"mappings": {
"properties": {
"Date": { "type": "date", "format": "yyyyMMdd"},
"areaID": { "type": "keyword" },
"deviceID": { "type": "keyword" }
}
}
}
The date range is from October to November.
I want to get a response for counting November's all new distinct 'deviceID' which grouped by 'areaID'.
I have no idea about how to implement it in ES syntax. Any ES master could give me some hints?
THANKS SO MUCH!
You can using aggs of elasticseach to group by areaID.
This is example with kibana
GET your_index/_search
{
"size": 1000000,
"query": {
"range": {
"Date": {
"gte": "2020-10-01",
"lte": "2020-11-31
}
}
}
},
"aggs": {
"area_id": {
"terms": {
"field": "areaID.keyword"
},
"aggs": {
"Date": {
"date_range": {
"field": "Date",
"ranges": [
{
"from": "2020-11-01",
"to": "2020-11-31"
}
]
},
"aggs": {
"device_id": {
"terms": {
"field": "deviceID.keyword",
}
}
}
}
}
}
}
}
I have the below aggregations query.
{
"aggs": {
"selected_brand": {
"filter": {
"term": {
"brandId": "b1d28821-3730-4266-8f55-eb69596004fb"
}
}
},
"sponsorshipSets": {
"nested": {
"path": "sponsorshipSets"
},
"aggs": {
"sponsorships": {
"nested": {
"path": "sponsorshipSets.sponsorships"
},
"aggs": {
"count": {
"value_count": {
"field": "sponsorshipSets.sponsorships.id"
}
}
}
}
}
}
}
}
The response is the below.
{
"hits": {
"total": {
"value": 2980,
"relation": "eq"
}
},
"aggregations": {
"selected_brand": {
"doc_count": 314
},
"sponsorshipSets": {
"doc_count": 2635,
"sponsorships": {
"doc_count": 1076,
"count": {
"value": 1076
}
}
}
}
}
The response shows the count of sponsorship documents is 1076, now I want to retrieve the documents these documents and tried with the below query.
{
"query": {
"bool": {
"must": {
"nested": {
"path": "sponsorshipSets",
"query": {
"nested": {
"path": "sponsorshipSets.sponsorships",
"query": {
"bool": {
"must_not": [
{
"match": {
"sponsorshipSets.sponsorships": "null"
}
}
]
}
}
}
}
}
},
"filter": [
{
"term": {
"brandId": "b1d28821-3730-4266-8f55-eb69596004fb"
}
}
]
}
}
}
The interesting thing for the second query is the hits below is only 82.
"hits": {
"total": {
"value": 82,
"relation": "eq"
},
What I really want is to retrieve the count of all the sponsorshipSets.sponsorships documents that are not null or not exist. SponsorshipSets can be missing as well.
Find below the abbreviated template.
{
"index_patterns": "campaigns*",
"order": 4,
"version": 4,
"aliases": {
"campaigns": {
}
},
"settings": {
"number_of_shards": 5
},
"mappings": {
"dynamic": "false",
"properties": {
"brandId": {
"type": "keyword"
},
"sponsorshipSets": {
"type": "nested",
"properties": {
"id": {
"type": "keyword"
},
"sponsorships": {
"type": "nested",
"properties": {
"id": {
"type": "keyword"
}
}
}
}
}
}
You can use exists query to filter such documents. Below query should help.
Query Request:
POST <your_index_name>/_search
{
"query":{
"bool":{
"must":{
"nested":{
"path":"sponsorshipSets",
"query":{
"nested":{
"path":"sponsorshipSets.sponsorships",
"query":{
"bool":{
"must_not":[
{
"exists":{
"field":"sponsorshipSets.sponsorships"
}
}
]
}
}
}
}
}
},
"filter":[
{
"term":{
"brandId":"b1d28821-3730-4266-8f55-eb69596004fb"
}
}
]
}
}
}
This should return all three scenarios where your document JSON structure would be
sponsorshipSets.sponsorships: {} i.e. you have empty structure for sponsorships
sponsorshipSets.sponsorships: null i.e. the value is set as null
Or your document doesn't have sponsorships field in first place.
You don't need to use any aggregations for this as ES would return you the count of such documents in hits.total.value of the response.
Let me know if this helps!
Is it possible using the Elastic Search _count API and having the following abbreviated ES template to find the count of sponsorships for all the campaigns by brandId?
sponsorshipSets and sponsorships are optional so it can be null.
{
"index_patterns": "campaigns*",
"order": 4,
"version": 4,
"aliases": {
"campaigns": {
}
},
"settings": {
"number_of_shards": 5
},
"mappings": {
"dynamic": "false",
"properties": {
"brandId": {
"type": "keyword"
},
"sponsorshipSets": {
"type": "nested",
"properties": {
"id": {
"type": "keyword"
},
"sponsorships": {
"type": "nested",
"properties": {
"id": {
"type": "keyword"
}
}
}
}
}
}
filter aggregation can be used to fetch docs with certain brand Id. Two Nested aggregations to point to sponsorship and value_count aggregation to get the count.
Query
{
"aggs": {
"selected_brand": {
"filter": {
"term": {
"brandId": "1"
}
}
},
"sponsorshipSets": {
"nested": {
"path": "sponsorshipSets"
},
"aggs": {
"sponsorships": {
"nested": {
"path": "sponsorshipSets.sponsorships"
},
"aggs": {
"count": {
"value_count": {
"field": "sponsorshipSets.sponsorships.id"
}
}
}
}
}
}
}
}
I found a solution without using Aggregations, it seems more accurate from the above and I can use the _count API.
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "sponsorshipSets.sponsorships",
"query": {
"bool": {
"filter": {
"exists": {
"field": "sponsorshipSets.sponsorships"
}
}
}
}
}
},
{
"term": {
"brandId": "b1d28821-3730-4266-8f55-eb69596004fb"
}
}
]
}
}
}
I use ElasticSearch version 1.7.5 and I am trying to fetch all documents where missing some fields.
My mapping:
...
"participant": {
"properties": {
"id": {
"type": "string"
},
"firstName": {
"type": "string"
},
"lastName": {
"type": "string"
},
"name": {
"type": "string"
}
},
"coordinator": {
"properties": {
"id": {
"type": "string"
},
"firstName": {
"type": "string"
},
"lastName": {
"type": "string"
},
"name": {
"type": "string"
}
}
...
I want to query all documents that don't have assigned coordinator.id or participant.id yet.
My query looks like:
"query": {
"nested": {
"path": "coordinator, participant",
"query": {
"constant_score": {
"filter": {
"or": [
{
"missing": {
"field": "coordinator.id"
}
},
{
"missing": {
"field": "participant.id"
}
},
]
}
}
}
}
}
You do OR queries via the bool query:
https://www.elastic.co/guide/en/elasticsearch/reference/1.7/query-dsl-bool-filter.html
So this query would work:
{
"query": {
"bool": {
"should": [
{
"constant_score": {
"filter": {
"missing": {
"field": "participant.id"
}
}
}
},
{
"constant_score": {
"filter": {
"missing": {
"field": "coordinator.id"
}
}
}
}
]
}
}
}
I noticed that you were using a nested query though the mapping does not state that coordinator and participant are nested field types so that will not work:
https://www.elastic.co/guide/en/elasticsearch/reference/1.7/mapping-nested-type.html
Setting something as a nested type is only useful when you need to group search terms together so I don't think it is necessary for you.
I'm trying to learn Elasticsearch, and therefore I'm trying to find a good data structure, that will fit my needs for querying.
My data structure actually looks like this :
{
"questions": [
{
"id": "n1-pain-score",
"name": "Pain score",
"answer": {
"value": 3,
"label": "Small pain"
}
},
{
"id": "n2-temperature",
"name": "Temperature",
"answer": {
"value": 37.5,
"label": "37.5°C"
}
}
]
}
For a given patient, there are multiple questions and answers.
Is there a way to take advantage of Elasticsearch filtered queries and say :
I want to see the patients that have a pain score over 6 ?
Note that I'm only testing and I still can modify the data structure to get what I want :).
It is possible to do with a filtered query. A plain query will also do:
GET {index}/{type}/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"name": "Pain score"
}
},
{
"range": {
"answer.value": {
"gt":6
}
}
}
]
}
}
}
I suggest the following mapping and queries. Most important part - questions needs to be of type nested so that the id matching should of the same element of questions. Meaning the id with value n1-pain-score must belong to the same element that also contains answer.value: 3.
DELETE test
PUT /test
{
"mappings": {
"test": {
"properties": {
"questions": {
"type": "nested",
"properties": {
"answer": {
"properties": {
"label": {
"type": "string"
},
"value": {
"type": "long"
}
}
},
"id": {
"type": "string",
"fields": {
"notAnalyzed": {
"type": "string",
"index": "not_analyzed"
}
}
},
"name": {
"type": "string"
}
}
}
}
}
}
}
POST /test/test/1
{
"questions": [
{
"id": "n1-pain-score",
"name": "Pain score",
"answer": {
"value": 3,
"label": "Small pain"
}
},
{
"id": "n2-temperature",
"name": "Temperature",
"answer": {
"value": 37.5,
"label": "37.5°C"
}
}
]
}
GET /test/_search
{
"query": {
"nested": {
"path": "questions",
"query": {
"bool": {
"must": [
{
"term": {
"questions.id.notAnalyzed": {
"value": "n1-pain-score"
}
}
},
{
"range": {
"questions.answer.value": {
"gte": 3
}
}
}
]
}
}
}
}
}
GET /test/_search
{
"query": {
"nested": {
"path": "questions",
"query": {
"bool": {
"must": [
{
"term": {
"questions.id.notAnalyzed": {
"value": "n2-temperature"
}
}
},
{
"range": {
"questions.answer.value": {
"gte": 38
}
}
}
]
}
}
}
}
}