The following query is returning an empty result even though there are results matching the query.
GET abc*/_search
{
"query": {
"bool": {
"must": [
{"range": {
"timestamp": {
"gte": "2018-01-01T00:00:00.000",
"lte": "2018-01-02T12:00:00.465"
}
}}
]
}
}
}
When I replace must with must_not I am getting results.The following is my index data
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 6,
"successful": 6,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "abc",
"_type": "log",
"_id": "abc3423498239048290",
"_score": 1,
"_source": {
"0": {
"test": 289,
"testnested": {
"testnested1": 0,
"testnested2": 0.615,
},
"test1": 46.17,
"test2": 59.4,
"ts": "2018-01-01T00:08:20.396UTC"
},
"1":{
----------
----------
},
........
"10":{
------------------
},
"timestamp": "2018-01-01T00:08:20.396UTC",
}
-----------
}
Is there anything I am doing wrong?.timestamp field is of type "date_hour_minute_second_millis".I am using Elasticsearch version 5+. I have used this same query in another index and it is working there.
Related
I am trying to apply pagination to my range query to get full records between the given date range.
Below is the query i tried using
Query:
{
"query":
{
"range":
{
"sys_created_on":
{
"gte":"2022-01-01 01:00:00",
"lte":"2022-03-10 01:00:00"
}
}
},
"sort": [
{"sys_created_on": "asc","u_user_updated": "asc"}
]
}
payload:
{
"took": 62,
"timed_out": false,
"_shards": {
"total": 8,
"successful": 8,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": null,
"hits": [
{
"_type": "_doc",
"_id": "61c158191bbc89905ad62064604bcb39",
"_score": null,
"_source": {
"key1":"val1",
"key2":"val2",
--------
--------
"keyn":"valn",
}
}
}
In the above query "sys_created_on" and "u_user_updated" are the fields available in my payload response. As said in this documentation, [elastic_search_search_query], I am not getting the payload with "sort" as key. So that I can use the search_after
Try changing
"sort": [
{"sys_created_on": "asc","u_user_updated": "asc"}
]
for
"sort": [
{"sys_created_on": "asc"}, {"u_user_updated": "asc"}
]
I'm trying out a query in elastic search(version 6.0) where I have a base query and on top of that, I have filters applied to narrow down the search. It is as follows:
GET target_index/_search
{
"from": {start},
"size": {offset},
"_source": [
"id",
"name",
"email",
"company",
"created_at",
],
"query": {
"bool": {
"filter": {
"bool": {
"filter": [
{ "terms":{"name.raw": ["test","test2"] }},
{ "terms":{"email.raw": ["test#test.com","test2#test.com"] }}
]
}
},
"must": {
"query_string": {
"query": "test",
}
}
}
},
"highlight": {
"fields": {
"*":{
"type":"plain"
}
}
}
}
Current result -
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1.90374,
"hits": [
{
"_index": "index_name",
"_id": "my_id",
"_score": 1.90374,
"_source": {
"id": 2,
"name": "test",
"email": "test#test.com",
"company": "test company"
},
"highlight": {
"name.raw": [
"<em>test</em>"
],
"name": [
"<em>test</em>"
],
"company": [
"<em>test</em> company"
]
}
}
]
}
}
Desired result -
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1.90374,
"hits": [
{
"_index": "index_name",
"_id": "my_id",
"_score": 1.90374,
"_source": {
"id": 2,
"name": "test",
"email": "test#test.com",
"company": "test company"
},
"highlight": {
"company": [
"<em>test</em> company"
]
}
}
]
}
}
Here, in the highlights in the desired result, I don't want the data for "name" and "name.raw". This field should not be searched only for this particular query , so I cannot disable the field entirely from searching.
I have a lot of terms and cannot specify every term to include in the query. Is there a way to exclude only a few fields from query search?
related ES doc -
https://www.elastic.co/guide/en/elasticsearch/reference/6.0/index.html
Instead of excluding certain fields, you could include only those that you need:
{
"query": {
...
},
"highlight": {
"fields": {
"company":{ <---
"type":"plain"
}
}
}
}
When we call the elasticsearch, say as follows:
POST https:////_search with body:
{
"from": 0,
"size": 1,
"query": {
"bool": {
"must": [
{
"range": {
"createdAt": {
"gt": "2019-11-11T10:00:00"
}
}
}
]
}
},
"sort": [
{
"createdAt" : {
"order" : "desc"
}
}
]
}
I see that I get only 1 result as pagination is set to 1 but total inside hits in response shows 2. This is the response I get:
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": “<index-name>”,
"_type": "_doc",
"_id": "5113c843-dff3-499f-a12e-44c7ac103bcf_0",
"_score": null,
"_source": {
"oId": "5113c843-dff3-499f-a12e-44c7ac103bcf",
"oItemId": 0,
"createdAt": "2019-11-13T11:00:00"
},
"sort": [
1573642800000
]
}
]
}
}
Doesn’t total doesn’t capture the pagination part? And it only cares about the query report? It should show the total count of items matching the query irrespective of the pagination set, right?
Yes, You are right that total doesn't capture the pagination part and just cares about the query report ie. whatever the total no of the document matches for a given query.
To be precise, it is as explained in official ES docs .
total (Object) Metadata about the number of returned documents.
Returned parameters include:
value: Total number of returned documents. relation: Indicates whether
the number of documents returned. Returned values are:
eq: Accurate gte: Lower bound, including returned documents
It means its the total no of returned documents, but as pagination is set to 1 in your example, inner hits have just 1 document.You can cross-check this understanding easily by creating a sample example as below:
Create a sample index with just 1 text field:
URL:- http://localhost:9200/{your-index-name}/ --> PUT method
{
"mappings": {
"properties": {
"name": {
"type": "text"
}
}
},
"settings": {
"index": {
"number_of_shards": "1",
"number_of_replicas": "1"
}
}
}
Once the above index is created index below 4 documents:
URL:- http://localhost:9200/{your-index-name}/_doc/{1,2,like..} --> POST method
{
"name": "foo 1"
}
{
"name": "foo bar"
}
{
"name": "foo"
}
{
"name": "foo 2"
}
Now when you hit below search query without pagination:
{
"query": {
"bool": {
"must": [
{
"match": {
"name": "foo"
}
}
]
}
}
}
It gives below response:
{
"took": 9,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 4, --> Note 4 here
"relation": "eq"
},
"max_score": 0.12199639,
"hits": [
{
"_index": "59638303",
"_type": "_doc",
"_id": "1",
"_score": 0.12199639,
"_source": {
"name": "foo"
}
},
{
"_index": "59638303",
"_type": "_doc",
"_id": "3",
"_score": 0.12199639,
"_source": {
"name": "foo"
}
},
{
"_index": "59638303",
"_type": "_doc",
"_id": "2",
"_score": 0.09271725,
"_source": {
"name": "foo bar"
}
},
{
"_index": "59638303",
"_type": "_doc",
"_id": "4",
"_score": 0.09271725,
"_source": {
"name": "foo 1"
}
}
]
}
}
But when you hit a search query with pagination:
{
"from": 0,
"size": 1,--> note size 1
"query": {
"bool": {
"must": [
{
"match": {
"name": "foo"
}
}
]
}
}
}
it gives below response
{
"took": 23,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 4, --> this is still 4
"relation": "eq"
},
"max_score": 0.12199639,
"hits": [
{
"_index": "59638303",
"_type": "_doc",
"_id": "1",
"_score": 0.12199639,
"_source": {
"name": "foo"
}
}
]
}
}
Now in the above query, you can change the size and check only inner-hits array gets change but the outer hits object which contains total always remains same as 4, this confirms your understanding is correct.
I have posted data to elastic search i.e username , resume(type: blob) .
now i want to search a particular world from resume so for that i fire following query in elasticsearch tool :
{"query": {
"bool": {
"must": [
{
"match": {
"filecontent": "Documentation"
}
}
],
"must_not": [],
"should": []
}
},"from": 0,
"size": 10
}
but No results are there: my resume contains "Documentation" word.
so please suggest me the how to construct a query in filecontent.
Help?
here is my data which I posted in elastic search:
{
"took": 112,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.6931472,
"hits": [
{
"_index": "profiles",
"_type": "resume",
"_id": "2013",
"_score": 0.6931472,
"_source": {
"filetype": "doc",
"firstname": "akash",
"filecontent": "PK\u0003\u0004\u0014\u0000\u0006\u0000\b\u0000\u0000\u0000!\u00002oWf\u0001\u0000\u0000¥\u0005\u0000\u0000\u0013\u0000\b\u0002[Content_Types].xml ¢\u0004\u0002( \u0000\u0002\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\"
}
]
}
I have a query like the following
{
"script_fields": {
"my_script_field": {
"script": "..."
}
},
"query": {
"match": {
"my_script_field":"*"
}
}
}
and it returns empty.
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 10,
"successful": 10,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}
I can see that the script is actually working by simply retuning everything
like:
{
"fields": [
"_source"
],
"script_fields": {
"my_script_field": {
"script": "..."
}
},
"query": {
match_all: {}
}
}
Example results:
"hits": {
"total": 1008681,
"max_score": 1,
"hits": [
{
"_index": "logstash-2016.08.27",
"_type": "traffic",
"_id": "AVbLDW8qw2vffjMOfTxb",
"_score": 1,
"_source": {
"#version": "1",
"#timestamp": "2016-08-27T06:11:46.000Z",
.................
}
},
"fields": {
"my_scripted_field": [
"Asia"
]
}
but I cant seem to access it in the query context in any other way.
my question so is:
Where is the script output saved in memory? (_fields?)
How to access it via a query? (without copying the script over to a filter script)
Thanks!