I'm trying to gather weekly statistics from Sentry, using the search feature.
I want to find new errors in the last week. Which I think I can do with age:
is:unresolved age:-1w
But how do I search for issues that were resolved in the last week? I can't find any search tag for this. I can't even find a resolved timestamp anywhere when viewing an individual issue.
Related
I am using an elastic search for search purpose. But recently I observer that some random error while adding data into elastic search:
version conflict, required seqNo [113789], primary term [19]. current document has seqNo [113797] and primary term [19]
The above type error comes randomly and I am not able to add/update data in elastic search.
Can you please help to understand:
What is the root cause of this issue?
How I can reproduce this issue? as this coming randomly need to know the basic step to reproduce this issue
What is the solution for this? How I can solve this issue?
This error happened during an update of the document at the same time others updated it. Check the parallelism.
When this process read the document it had the version number 113789 and when ES received the update did not match with the version number (current version 113797). It causes the version conflict.
Is there a way to update documents something similar to UpdateByQuery, but in bulks and without getting them.
According to the documentation we are unable to set a size for UpdateByQuery requests.
I.e Update 5 documents at a time and not all at once.
One solution that seems obvious is to GET 5 documents, and then UPDATE them.
I'm trying to come up with a way where I dont have to do a GET request for every update.
You can set the batch size on UpdateByQueryRequest with setBatchSize as in this page from the docs.
https://www.elastic.co/guide/en/elasticsearch/client/java-rest/master/java-rest-high-document-update-by-query.html
Now that's based on the latest version of the Java client. If you are using a different client or version, it may not be present. Hope that helps.
Here is the problem
I have about 1 million record in indexes. There is a property aging in the documents which increase daily. Every night scheduler runs and it calculates the aging from current date and created date in the document and update the index.
The problem is as data is increasing the bulk update is leading to GC overhead limit exceeded. So what I did is added some pause in each update, but still no help.
Now I am thinking and researching of using groovy script with 'update_with_query'.
I want to ask it there any other way to maintain age. e.g in jira everyday overdue date is increased or I have to fetch visit and update documents
EveryTime bulk request is run I can see elastic search throttling ' now throttling indexing: numMergesInFlight=5, maxNumMerges=4'. I have read about this but not sure what to do. I think there should be another approach to calculate aging but not sure, because as data will increase this problem is going to persist
IN the end I want a query like give me all docs whose aging is 100 or give me all documents whose aging > 100
The answer was simple. I was thinking other way around.
if a query is get all docs where aging is > 2. It means I need to get all docs who were created before two days. Simple convert '2' to date from current date and use range operation and it should solve the problem
Is it possible to give different time periods or different search terms to each Visualization in a Kibana Dashboard?
Currently - no.
This is on the list of enhancements that the 'elastic' team will implement soon, but doesn't have any due date yet.
You could follow the open issue here: https://github.com/elastic/kibana/issues/3578
I think i've understood your question.
Lets supose this is yout data whitin elasticSearch:
timestamp level message
19:05:15 error connection failed
19:06:30 debug connection succesfull
You can reflect your percentajes of each level in differente time periods (10% of debug, 20% of errors, 14% of info and so on). For instance you can design a chart for the last 1 hour and other one for the last day in the same dashboard, so you don't need to manipulate the date picker in de header.
First you have to make a query to filter your data by the timestamp
(ex. last day):
#timestamp:[now-1d TO now]
Second, you need to save this search, and name it.
Finally, design whatever visualization you need based on this
search, and the results will be bound to it.
Repeat with different time periods.
Hope this helps. Bye.
We are using the ELK for log aggregation. Is it possible to search for events that occured during a particular time range. Lets say I want to see all exceptions that occurred between 10am and 11am in last month.
Is it possible to extract the time part from #timestamp and do a range search on that somehow (similiar to date() in SQL)?
Thanks to Magnus who pointed me to looking at scripted fields. Take a look at:
https://www.elastic.co/blog/kibana-4-beta-3-now-more-filtery
or
https://www.elastic.co/guide/en/elasticsearch/reference/1.3/search-request-script-fields.html
Unfortunately you can not use these scripted fields in queries but only in visualisations.
So I resorted to a workaround and use logstashs drop filter to remove the events I don't want to show up in Kibana in the first-place. That is not perfect for obvious reasons but it does the job.