using search_after
i have more than 100,000 documents on elasticsearch and i need to get previous page and forward page
but my problem is i cannot get previous page with search_After. is it possible to use search_After with from to size to pagination more than 10,000 documents? or do i other options?
i tried using search_after java code and sort fileds are userId and date(HH:mm:ss.SSS) but when i click previous button or jump to page 1 to page 5, its not showing proper values
Related
I'm trying to figure out how to accomplish pagination with a multi match query using elasticsearch.
The scroll and search_after APIs seem like they won't work. scroll isn't meant for real time user requests as per documentation. search_after requires some unique field per id and requires you to sort on that field as per documentation but when using a multi-match query you're basically sorting by the score.
So, the only thing I've thought of so far is to do the following:
Send back last document id + score and use the score as the sort field. But, this could potentially return duplicate documents if other documents were added in between two queries.
If you want to paginate the first option is to use from and size parameter in your query. The documentation here
Pagination of results can be done by using the from and size
parameters. The from parameter defines the offset from the first
result you want to fetch. The size parameter allows you to configure
the maximum amount of hits to be returned.
Though from and size can be set as request parameters, they can also
be set within the search body. from defaults to 0, and size defaults
to 10.
Note that from + size can not be more than the index.max_result_window
index setting which defaults to 10,000. See the Scroll or Search After
API for more efficient ways to do deep scrolling.
If you don't need to paginate over 10k results it's your best choice. The max_result_window can be modified, but the performance will decrease as the selected page number will increase.
But of course if some documents are added during your user pagination they will be added and your pagination can be slightly inaccurate.
I am trying to make an Instagram like Explore page using Elasticsearch. The contents are scored based on time as well as number of likes. Since, the content likes are frequently updated, pagination is difficult using From/Size and Search After. Suppose, I fetched first 10 posts using From 0, Size 10. Another 10 posts scored more likes by the time I'm trying to fetch the second page in pagination. Now, I have the same posts that I fetched in first pagination at positions 10 to 20. This will create lot of duplicate in my explore page.
I am more concerned about avoiding duplicates in pagination than missing some content, because if the user refresh explore page, the top contents will be displayed again. The best way I think is to fetch all posts below a particular score. Is there anything like a max_score api. If not, how can i solve this problem?
I am trying to fetch more than 10000 doc with jest client.
I used scroll feature and use a query size of 50, but my program goes into an infinite loop and in every iteration returns the same 50 doc results.
I guess it is problem with scroll id which I am not passing can some body help.
Below is the call to be made to retrieve the first 50 records :
POST <host_name>:<port_num>/<index_name>/_search?scroll=1m&size=50
As shown above, the size is mentioned as 50 and scroll is 1m, this means that the scroll api will retrieve 50 records per hit and this scroll is available for 1 minute. Also, this api returns a scroll id, which should be used for further retrieval of records. Please find the sample below:
POST <host_name>:<port_num>/_search?scroll=1m&scroll_id=<scroll_id>
Note : For further scroll api calls, index name need not be mentioned. Only the scroll_id and scroll time is sufficient.
For more information, please refer to the elastic search documentation on scroll api : https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html
I am new to Kibana and using it for visualising the data present in Elastic Search.
I am trying to create dynamic dashboard i.e. by using saved search indexes having field values as variable.
What I want
Want to use place holders in the query which can be populated from URL parameters and then search results rendered in dashboard.
So that user can search results by providing some input instead of fixed query.
Can it be done in Kibana? If not, is there any better visualisation tool other than Kibana to serve this purpose.
In the Dashboard View there is actually a searchbar where you can just fire normal Matchqueries and its easy to filter i.e:
Create a table with terms aggregation for one of the fields a user might be interested in.
Click on one of the Terms in the Dashboard
A filter can be seen under the searchbar and all elements in the dashboard will be filtered with it.
If you have line charts users can zoom into the charts to see only information of the zoomed in timeframe
Barcharts are interactive like tables
Play around a bit. Kibana is very powerful you just have to find the right visualizations.
I want to get back the title of the first result of each page in a search so that I can display them on the pagination ui. Is there an easy way to do this with ElasticSearch?
You could do the search with "fields" : [] to retrieve a list of all matching document IDs [1], and then do a multi-get request to retrieve the documents including whatever information you need to display them [2].
http://www.elasticsearch.org/guide/reference/api/search/fields.html
http://www.elasticsearch.org/guide/reference/api/multi-get.html