I am trying to query Elasticsearch where source is as follows:
source => /home/user/logs/serviceA.inst-app3.test.log.INFO.20180204-084131.21231
I want to filter all messages where source is serviceA.*INFO*. However, Kibana returns empty list. For queries like serviceA.* or serviceA* queries, Kibana returns the result where serviceA.inst is highlighted. My suspicion is minus sign in the file name is causing the issue.
How to query Elasticsearch from Kibana with regex which has minus sign (-) in it?
Related
I am trying to search my logs within the search bar of Kibana UI for error codes that consist of:
a fixed 3 digit String
a minus
a 5 digit number
, e.g. TED-12345. The error codes can be located anywhere inside the message field
I tried the following Regex message: /.*TED-[0-9]{5}.*/ but it did not return the expected results. I probably mixed query and "search bar syntax". Can anybody suggest the correct regex?
Please make sure you have the Lucene syntax enabled for your queries, because Kibana Query Language does not support regular expressions.
From docs: https://www.elastic.co/guide/en/kibana/master/kuery-query.html
KQL has a different set of features than the Lucene query syntax. KQL is able to query nested fields and scripted fields. KQL does not support regular expressions or searching with fuzzy terms. To use the legacy Lucene syntax, click KQL next to the Search field, and then turn off KQL.
I am trying to create one query in the Kibana search bar to retrieve some specific documents.
The goal is to get the documents that either have the field "myDate" before 2019-10-08 or "myDate" does not exist.
I have documents that meet one or the other condition.
I started by creating this query :
myDate:<=2019-10-08 OR NOT _exists_:myDate
But no documents were returned.
Since it did not work, I tried some other ways i found online :
myDate:<=2019-10-08 OR NOT (_exists_:myDate)
myDate:<=2019-10-08 OR !(_exists_:myDate)
myDate:<=2019-10-08 OR NOT (myDate:*)
But still, no results.
When I use either "part" of the "OR" condition, it works perfectly : I get either the documents who have myDate<=2019-10-08 or the ones that do not have a "myDate" field filled.
But when I try with both conditions, I get no document.
I have to use only the search bar to find these documents, neither an elasticsearch rest query nor by using kibana filters.
Thank you for your help :)
Below query works. Use Inspect button in kibana to see what query is actually being fired and make sure you are using correct index pattern as well.
(myDate:<=2019-12-31) OR (NOT _exists_:myDate)
Take a look at Query DSL documentation for Boolean operators for more better understanding with different use cases
I have a doc in Elasticsearch with different fieldnames, eg: a,b,c,d...
I want to use templating in Grafana to query a term aggregation in such way that I get the values in a field. eg: i.
I'm trying to use this query:
{"find":"terms","field":"i","size":25}
but it does not return any values.
I know that there are some values as I query the same docs with Sense.
I have Grafana v 4.6.2 and Elasticsearch v 2.3.4
The field I wanted has a "-" in the string. ES sees it as a separator, this was the reason of the error.
Changing the field's mapping to "not analyzed" should help.
I have a analyzed field hostname and a not_analyzed field hostname.raw.
I'm trying to query a few hosts that have a dot in the field value, like
AP.MO.HALL-01
AP.MO.2FLOOR-01
When I try to query hostname:AP.MO.*, it also returns hosts with the value AP.MOOCA.HALL
When I try to query using the field that isn't analyzed, I get no results at all: hostname.raw:AP.MO.*
How can I make Kibana respect the dot before the wildcard?
I was able to get a similar issue resolved by using a RegEx query, and escaping the periods within the hostname.
Looking at your example, something similar to the following should work:
hostname: /.*AP\.MO\..*/
I am using ELK to create dashboards from my log files. I have a log file with entries that contain an id value and a "success"/"failure" value, displaying whether an operation with a given id succeeded or failed. Each operation/id can fail an unlimited number of times and succeed at most once. In my Kibana dashboard I want to display the count of log entries with a "failure" value for each operation id, but I want to filter out cases where a "success" log entry for the id exists. i.e. I am only interested in operations that never succeeded. Any hints for tricks that would achieve this?
This is easy in Kibana 5 search bar. Just add a filter
!(_exists_:"your_variable")
you can toggle the filter or write the inverse query as
_exists_:"your_variable"
In Kibana 4 and Kibana 3 you can use this query which is now deprecated
_missing_:"your_variable"
NOTE: In Elasticsearch 7.x, Kibana now has a pull down to select KQL or Lucene style queries in the search bar. Be mindful that syntax such as _exists_:FIELD is a Lucene syntax and you need to set the pulldown accordingly.
In newer ELK versions (I think after Elasticsearch 6) you should use field:* to check if the field exist and not field:* to check if it's missing.
elastic search reference:
https://www.elastic.co/guide/en/elasticsearch/reference/6.5/query-dsl-query-string-query.html#_wildcards
! (_exists_:NAME) is not working for me. I use suggestion from:
https://discuss.elastic.co/t/kibana-5-0-0--missing--is-not-working-anymore/64336
NOT _exists_:NAME
UPDATE The problem I faced is that ES syntax forbids spaces after negation operators. Use one of:
NOT _exists_:FIELD
!_exists_:FIELD
-_exists_:FIELD
Check tutorial: https://www.timroes.de/2016/05/29/elasticsearch-kibana-queries-in-depth-tutorial/
NOTE: In Elasticsearch 7.x, Kibana now has a pull down to select KQL or Lucene style queries in the search bar. Be mindful that syntax such as _exists_:FIELD is a Lucene syntax and you need to set the pulldown accordingly.
In newer versions of Kibana the default language is now KQL (Kibana Query Language) not Lucene anymore. So most answers here are outdated. The query if a field exists is the following:
your_variable:*
and to answer your question you can just negate that:
not your_variable:*
You can find more documation on here: https://www.elastic.co/guide/en/kibana/7.15/kuery-query.html
You can also toggle back to Lucene if you click on that button inside the search field but in my opinion the new language is way easier to use:
One option would be to create an own query for this criteria in Kibana. Then just have your panel that does the counting just to use this query.
value:failure
More information here:
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html#query-string-syntax