I'm trying to fetch limited amound of search result but still use of pagination. When I use next line in my repository class I get 100 result on the page.
Page<Media> findTop100ByOrderByViewCount(Pageable pageable);
If I use next line instead, then I get expected result with according to my pageable which is 10 item on each page.
Page<Media> findAllByOrderByViewCount(Pageable pageable);
I want to limit the search result. If there are 1000s of search result, I don't want to have 100s of paginated result. I want to limit my paginated limit to have max 10 page.
How can I achieve that in Spring boot? I'm using version of 2.2.2.RELEASE.
There is no shortcut for this in Spring Data.
You can wrap the call to the repository and replace the Page you receive with a modified one, which limits the total amount to 10*pagesize and removes the next value for page number 10. And probably skips the call to the database completely if the pagenumber is greater 10.
Related
I am working on a project that uses Scopus API to get document names or journal names under different scenarios. I am using ScopusSearch API (https://dev.elsevier.com/documentation/ScopusSearchAPI.wadl) and SerialTitle API (https://dev.elsevier.com/documentation/SerialTitleAPI.wadl) for the purpose.
However, the total number of documents I am able to retrieve using these API's is very few. I want to increase the number of documents being fetched. Now, I've been through the documentation of these API's a several times but I am confused with the use of start parameter and the cursor parameter.
Take for example, ScopusSearch API, under its query params section:
start parameter
cursor parameter
Can someone please help me understand the difference between these two? And more specifically when to use the start and when to use the cursor parameter?
If you use pybliometrics, as your tag suggests, then you don't need to care about this.
The basic idea behind this pagination (that's what you're after) is:
Run a query with unlimited number of results with cursor set to "*"
Set start to 0 and get the first count results
Set start to start+count+1 and get the next count results
Repeat step 3 until all results are fetched
I have 100 rows of data in DynamoDB and a api with path api/get/{number}
Now when I say number=1 api should return me first 10 values. when I say number=2 it should return next 10 values. I did something like this with query, lastEvaluatedKey and sort by on createdOn . Now the use case is if the user passes number=10 after number=2 the lastEvaluatedKey is still that of page 2 and the result would be data of page 3. How can I get data directly. Also if the user goes from number=3 to number=1 still the data will not be of page 1.
I am using this to make API call based of pagination on HTML.
I am using java 1.8 and aws-java-sdk-dynamodb.
Non-sequential pagination in DynamoDB is tough - you have to design your data model around it, if it's an operation that needs to be efficient at all times. For a recommendation in your specific case I'd need more details about the data and access patterns.
In general you have the option of setting the ExclusiveStartKey attribute in the query call, which is similar to an offset in relational databases, but only similar and not identical. The ExclusiveStartKey is the key after which the query will continue, meaning data from your table and not just a number.
That means you usually can't guess it, unless it's a sequential number - which isn't ideal.
For sequential pagination, i.e. the user goes from page 1 to page 2, page 2 to page 3 etc. you can pass that along in the request as a token, but that won't work if the user moves in the other direction page 3 to page 2 or just randomly navigates to page 14.
In your case you only have a limited amount of data - 100 items, so my solution for your specific case would be to query all items and limit the amount of items in the response to n * 10, where n is the result page. Then you return the last 10 items from that result to your client.
This is a solution that would get expensive at scale (time + cost) though, fortunately not many people will use the pagination to go to page 7 or 8 though (you could bury a body on page 2 of the google search results).
Yan Cui has written an interesting post on this problem on Hackernoon, you might want to check it out.
I have a model called DemoModel and contains 1000 records in DB. So i am paginating using paginator in Django(assume that per page 15 records, so i have 67 pages).
So i want to get the records of 3,4 and 5 pages and i have to append the records into list.
So can i get the objects_list based on page range or anything else i want to do?
Example:
records.page(1)
Here i am getting only one page records at a time, but how can i get multiple page records i.e; from fist page to third page
Assuming you are asking about the API request to get the paginated resources, and you are using the default pagination class: rest_framework.pagination.LimitOffsetPagination, then you can make an request as such:
https://api.example.org/accounts/?limit=30&offset=15
which in turns give you the 2nd and 3rd "page".
The limit indicates the maximum number of items to return, and is equivalent to the page_size in other styles. The offset indicates the starting position of the query in relation to the complete set of unpaginated items. doc link
Given I have a simple query:
List<Customer> findByEntity(String entity);
This query returns 7k records in 700ms.
Page<Customer> findByEntity(String entity, Pageable pageable);
this query returns 10 records in 1080ms. I am aware of the additional count query for pagination, but still something seems off. Also one strange thing I've noticed is that if I increase page size from 10 to 1900, response time is exactly the same around 1080 ms.
Any suggestions?
It might indeed be the count query that's expensive here. If you insist on knowing about the total number of elements matching in the collection there's unfortunately no way around that additional query. However there are two possibilities to avoid more of the overhead if you're able to sacrifice on information returned:
Using Slice as return type — Slice doesn't expose a method to find out about the total number of elements but it allows you to find out about whether a next slice is available. We avoid the count query here by reading one more element than requested and using its (non-)presence as indicator of the availability of a next slice.
Using List as return type — That will simply apply the pagination parameters to the query and return the window of elements selected. However it leaves you with no information about whether subsequent data is available.
Method with pagination runs two query:
1) select count(e.id) from Entity e //to get number of total records
2) select e from Entity e limit 10 [offset 10] //'offset 10' is used for next pages
The first query runs slow on 7k records, IMHO.
Upcoming release Ingalis of Spring Data will use improved algorithm for paginated queries (more info).
Any suggestions?
I think using a paginated query with 7k records it's useless. You should limit it.
I want to get first 20 records, I have response time as 200
After some time (By calling the same service) I want another 20 records.
By each hit I want to get 20 records. How can I implement this?
I am using Spring ,hibernate and angular as front-end.
Please provide a solution .
Thanks in advance.
Use spring-data-rest, you should be able to expose your hibernate entity to the user in a RESTful way. Using the auto generated end points you should be able to perform POST/PUT/GET/DELETE. When you expose the entity, the pagination is by default available for you.
By making use of spring-data-rest your scenario could be solved by giving a page size in the (GET) REST response
Example:
For instance assume that you have 200 user records in your DB and you want server 20 records per request,then the GET REST URL will be look like this:
http://localhost:8080/users?page=1&size=5
There are 2 key information to be noted:
page - the page number to access (0 indexed, defaults to 0).
size - the page size requested (defaults to 20).
So to get first 20 records, user will issue a request like:
http://localhost:8080/users or http://localhost:8080/users?page=0&size=20
To access next 20 items, change the page number alone: http://localhost:8080/users?page=1
Since default size is 20, in you case you can omit that; but if you decide to modify the size, say 25 or 30, then you should be able to supply that as part of size param.