I would like to list the count of certain message / code in kibana data table. In the buckets section, I select aggregation as "Terms" and select the property I would like to count in "Field". It works fine for some terms but not the others.
It is okay for "fields.MemberType.raw"
No result found if I select "fields.RestApiRequestAndResponse.Response.Message".
I can find both fields in discover.
Have no idea why no result found ... Any idea or how can I trace what happened? Thanks in advance
finally found the cause of the issue.. my colleague has update the template of the log... the field "fields.RestApiRequestAndResponse.Response" has map to "nested". :(
Related
I am trying to create one query in the Kibana search bar to retrieve some specific documents.
The goal is to get the documents that either have the field "myDate" before 2019-10-08 or "myDate" does not exist.
I have documents that meet one or the other condition.
I started by creating this query :
myDate:<=2019-10-08 OR NOT _exists_:myDate
But no documents were returned.
Since it did not work, I tried some other ways i found online :
myDate:<=2019-10-08 OR NOT (_exists_:myDate)
myDate:<=2019-10-08 OR !(_exists_:myDate)
myDate:<=2019-10-08 OR NOT (myDate:*)
But still, no results.
When I use either "part" of the "OR" condition, it works perfectly : I get either the documents who have myDate<=2019-10-08 or the ones that do not have a "myDate" field filled.
But when I try with both conditions, I get no document.
I have to use only the search bar to find these documents, neither an elasticsearch rest query nor by using kibana filters.
Thank you for your help :)
Below query works. Use Inspect button in kibana to see what query is actually being fired and make sure you are using correct index pattern as well.
(myDate:<=2019-12-31) OR (NOT _exists_:myDate)
Take a look at Query DSL documentation for Boolean operators for more better understanding with different use cases
I just started working with graylog and I have some issues.
Can I write a query that will bring me logs with unique identifier?
For examples I have logs with op_id and loan_amt and I want to get sum of loan_amt from all logs. Here comes the problem : some logs may share same op_id and my sum will not be correct because will add plenty times the loan_amt from logs with same op_id
Can you help me, please?
If I understand correctly you will need to further narrow down your search criteria to filter out duplicate log entries.
You can use the GrayLog search query language to do this.
Try to find fields where duplicate logs differentiate from each other and then create a filter to exclude one from your results.
For example something like this:
source:hostname.that.logs.loans_amt AND LoggerName:your.logger.that.logs.loan_amt
I am trying to create a Date Histogram and aggregate a particular field to find the maximum value which is of long type in mapping from my ealsticsearch, but i get the result in floating point number,
for example :
Instead of getting 31032832 am getting 3.1032832E7
However am able to get 31032832 properly when i query my elasticsearch index through chrome plugin sense.
I found out what was the issue! it was giving me double value after aggregation because of this:
while accessing i called myResult.getMax().longValue() which solved my problem.
I have just updated a website, the update adds new fields to elasticsearch.
In my dev environment, it all works fine. but on the live site, the new fields are not being found.
Eg. I have added a new field with the value : 1
However, when adding a filtered query of
{"field":1}
It does not find any matching results.
When I look in the documents, I can see docs with the field set to 1
Would the reason for this be that the new field was added after the mappings was set? I am not all that familiar with elasticsearch, So I am not really sure where to start looking to fix it.
Any help would be appreciated.
Update:
querying from URL shows nothing either
_search/?pretty=true&size=50&q=field1:*
however there is another field that was added at the same time which I can search on.
I can see field1 in the result set but it just wont allow me to search on it.
Only difference i see in the mapping is that the one that is working is set to type:long whereas the one not working is set as type:string
Is it a length issue on the ngram? what was your "min_gram" settings?
When you check on your index settings like this:
GET <host>/<index_name>/_settings
Does it work when you filter for a two digit field?
Are all the field values one digit?
It's OK to add a field after the mapping was set. ElasticSearch will guess the mapping for you. (in fact, it's one of their selling features --- no need to define the mapping, just throw the data at it)
There are a few things that can go wrong:
Verify that data is actually in the index. To do that, just navigate to the _search url with no parameters, you should see the field if it is indexed.
Look at your mapping. Could it be that the field is explicitly set not to be indexed?
Another possibility is that your query is wrong (but that is unlikely, since you're saying it works in the development environment)
I need to facet inside n documents which are selected like
... ORDER BY something DESC LIMIT 100
Is that possible with Solr? How?
this is a total hack, but here goes...
do your initial query, and get your results back.
construct a new query, like so:
http://localhost:8080/solr/select/?q=id%3A123+OR+id%3A456...(keep OR-ing them up)...&facet=true&facet.field=something
where you concatenate all of your ids to a new query using OR. then, when you facet on your field, the facet summary will only apply to the results.
AFAIK no, that's not supported / implemented. Facets aren't really meant to be "stats" but a guidance to the end-user. Picture yourself browsing a faceted interface and seeing facets change whenever you change sort order or paging. Faceted browsing would be useless if it worked like that.
I think this would be a nice feature for the StatsComponent though.
I think this is possible with results grouping (now in trunk!):
http://wiki.apache.org/solr/FieldCollapsing
... the only problem is that you can set only one 'facet.field' (i.e. group.field)
But the great thing is that you get scored facets!