Query on number field matching string fields - elasticsearch

Question

I have an ELK stack setup. When I am performing a query on number fields then it is also matching against string fields. For example, I am sending Load Balancer logs to ELK and if I perform backend_processing_time:>5 on that then it is matching against backend_processing_time with value 0.001 too.
On kibana interface, it is showing that the query is matching string in the request message. I am not able to understand how a query against a number field is matching against a string.
In the dev tools section on kibana i tried to run the same query
GET _search
{
"query": {
"range" : {
"backend_processing_time" : {
"gte" : 50000000000
}
}
}
}
Even with so much backend_processing_time i am getting results. I am not able to understand why this is happening.
I searched on other fields also which are of number type and found that all the queries done on number field are getting matched with string type fields.
I am providing a sample search result which i get for backend_processing_time:>500000000 query. It can be seen in this result that backend_processing_time field is so small but still getting a hit.
{
"_index": "logstash-2017.05.10",
"_type": "prod-quizelb-logs",
"_id": "AVvzYRgL49GPTZAKoDer",
"_score": null,
"_source": {
"backendport": 80,
"received_bytes": 0,
"request": "http://en.meaww.com:80/locales/en.json",
"backend_response": 200,
"verb": "GET",
"message": "2017-05-10T17:19:52.881044Z Prod-ELB 172.68.144.71:34803 10.1.91.253:80 0.000075 0.000606 0.000019 200 200 0 1881 \"GET http://en.meaww.com:80/locales/en.json HTTP/1.1\" \"Mozilla/5.0 (Linux; Android 6.0.1; SM-C900F Build/MMB29M; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/58.0.3029.83 Mobile Safari/537.36 [FB_IAB/FB4A;FBAV/122.0.0.17.71;]\" - -\n",
"type": "prod-quizelb-logs",
"clientport": 34803,
"request_processing_time": 0.000075,
"urihost": "en.meaww.com:80",
"response_processing_time": 0.000019,
"path": "/locales/en.json",
"#timestamp": "2017-05-10T17:21:18.280Z",
"port": "80",
"response": 200,
"bytes": 1881,
"clientip": "172.68.144.71",
"proto": "http",
"#version": "1",
"elb": "Prod-ELB",
"httpversion": "1.1",
"backendip": "10.1.91.253",
"backend_processing_time": 0.000606,
"timestamp": "2017-05-10T17:19:52.881044Z"
},
"fields": {
"#timestamp": [
1494436878280
],
"timestamp": [
1494436792881
]
},
"highlight": {
"backend_processing_time.keyword": [
"#kibana-highlighted-field#6.06E-4#/kibana-highlighted-field#"
],
"request": [
"#kibana-highlighted-field#http#/kibana-highlighted-field#://#kibana-highlighted-field#en.meaww.com#/kibana-highlighted-field#:#kibana-highlighted-field#80#/kibana-highlighted-field#/#kibana-highlighted-field#locales#/kibana-highlighted-field#/#kibana-highlighted-field#en.json#/kibana-highlighted-field#"
],
"elb.keyword": [
"#kibana-highlighted-field#Prod-ELB#/kibana-highlighted-field#"
],
"urihost.keyword": [
"#kibana-highlighted-field#en.meaww.com:80#/kibana-highlighted-field#"
],
"verb": [
"#kibana-highlighted-field#GET#/kibana-highlighted-field#"
],
"request.keyword": [
"#kibana-highlighted-field#http://en.meaww.com:80/locales/en.json#/kibana-highlighted-field#"
],
"type": [
"#kibana-highlighted-field#prod#/kibana-highlighted-field#-#kibana-highlighted-field#quizelb#/kibana-highlighted-field#-#kibana-highlighted-field#logs#/kibana-highlighted-field#"
],
"message": [
"2017-05-10T17:19:#kibana-highlighted-field#52.881044Z#/kibana-highlighted-field# #kibana-highlighted-field#Prod#/kibana-highlighted-field#-#kibana-highlighted-field#ELB#/kibana-highlighted-field# 172.68.144.71:34803 10.1.91.253:#kibana-highlighted-field#80#/kibana-highlighted-field# 0.000075 0.000606 0.000019 200 200 0 1881 \"#kibana-highlighted-field#GET#/kibana-highlighted-field# #kibana-highlighted-field#http#/kibana-highlighted-field#://#kibana-highlighted-field#en.meaww.com#/kibana-highlighted-field#:#kibana-highlighted-field#80#/kibana-highlighted-field#/#kibana-highlighted-field#locales#/kibana-highlighted-field#/#kibana-highlighted-field#en.json#/kibana-highlighted-field# #kibana-highlighted-field#HTTP#/kibana-highlighted-field#/1.1\" \"#kibana-highlighted-field#Mozilla#/kibana-highlighted-field#/5.0 (#kibana-highlighted-field#Linux#/kibana-highlighted-field#; #kibana-highlighted-field#Android#/kibana-highlighted-field# #kibana-highlighted-field#6.0.1#/kibana-highlighted-field#; #kibana-highlighted-field#SM#/kibana-highlighted-field#-#kibana-highlighted-field#C900F#/kibana-highlighted-field# #kibana-highlighted-field#Build#/kibana-highlighted-field#/#kibana-highlighted-field#MMB29M#/kibana-highlighted-field#; #kibana-highlighted-field#wv#/kibana-highlighted-field#) #kibana-highlighted-field#AppleWebKit#/kibana-highlighted-field#/#kibana-highlighted-field#537.36#/kibana-highlighted-field# (#kibana-highlighted-field#KHTML#/kibana-highlighted-field#, #kibana-highlighted-field#like#/kibana-highlighted-field# #kibana-highlighted-field#Gecko#/kibana-highlighted-field#) #kibana-highlighted-field#Version#/kibana-highlighted-field#/4.0 #kibana-highlighted-field#Chrome#/kibana-highlighted-field#/#kibana-highlighted-field#58.0.3029.83#/kibana-highlighted-field# #kibana-highlighted-field#Mobile#/kibana-highlighted-field# #kibana-highlighted-field#Safari#/kibana-highlighted-field#/#kibana-highlighted-field#537.36#/kibana-highlighted-field# [#kibana-highlighted-field#FB_IAB#/kibana-highlighted-field#/#kibana-highlighted-field#FB4A#/kibana-highlighted-field#;#kibana-highlighted-field#FBAV#/kibana-highlighted-field#/122.0.0.17.71;]\" - -\n"
],
"urihost": [
"#kibana-highlighted-field#en.meaww.com#/kibana-highlighted-field#:#kibana-highlighted-field#80#/kibana-highlighted-field#"
],
"path": [
"/#kibana-highlighted-field#locales#/kibana-highlighted-field#/#kibana-highlighted-field#en.json#/kibana-highlighted-field#"
],
"verb.keyword": [
"#kibana-highlighted-field#GET#/kibana-highlighted-field#"
],
"proto.keyword": [
"#kibana-highlighted-field#http#/kibana-highlighted-field#"
],
"port": [
"#kibana-highlighted-field#80#/kibana-highlighted-field#"
],
"type.keyword": [
"#kibana-highlighted-field#prod-quizelb-logs#/kibana-highlighted-field#"
],
"proto": [
"#kibana-highlighted-field#http#/kibana-highlighted-field#"
],
"elb": [
"#kibana-highlighted-field#Prod#/kibana-highlighted-field#-#kibana-highlighted-field#ELB#/kibana-highlighted-field#"
],
"backend_processing_time": [
"#kibana-highlighted-field#6.06E#/kibana-highlighted-field#-4"
],
"port.keyword": [
"#kibana-highlighted-field#80#/kibana-highlighted-field#"
]
},
"sort": [
1494436878280
]
}
EDIT
I got the mapping by running GET /logstash-2017.05.11/_mapping/prod-quizelb-logs query in kibana console.
The mapping which I am getting for backend_processing_time is showing this
"backend_processing_time": {
"type": "text",
"norms": false,
"fields": {
"keyword": {
"type": "keyword"
}
}
}
So it seems that this field is of text type thus causing this error to happen.
Now I have another confusion i.e. kibana is showing this as number but elasticsearch is showing this of type text. Also, this is getting mapped dynamically as i never created the mapping on my own. I think that they are getting created by logstash at the time grok filter is applied.

Answer 1

You need to take control of the mapping of those index(indices) so that your field will actually be a number. Otherwise, you will not be sure what kind of field type you'll have there. So, basically you need something like this, either in an index template, or a static mapping all the way:
"backend_processing_time": {
"type": "integer"
}

Answer 2

Remove space in your query_string. i.e Your query_string should look like this:
backend_processing_time:>0.5
Read more about query_string syntax here