Dynamic navigation shows incorrect result count - google-search-appliance

When we filter result by Duplicate Snippet filter (filter=p) we receive incorrect result count in dynamic navigation.
E.g.
Picture with issue
Actual result count = 5
Result are shown for end-user = 3 (Two pages was filtered)
Dynamic navigation by content format(Web Pages) = 5
As I know GSA has some problems with determinating actual count for a large number of results.
Is it possible to set filtered result count in dynamic navigation (Web Pages)?
GSA System Version: 7.2.0.G.252

Related

Calculate total items per page (pagination logic)

I need to check the total items of every page on pagination.
For example, I have 165 posts and a pagination with 4 links:
1,2,3,4
How can I discover the total items per page based on the number of pages generated?
I don't know what you're using for pagination, but the count for each page is generally set by a variable, then the remainder fills up the last page. It doesn't just output a random amount of the total on each page.
If using Kaminari, for example, there is a variable config.default_per_page.
Something like this should provide the answer.
full_pages = total_posts / default_per_page
spill_over = total_posts - (default_per_page * full_pages)

REST API - Retrieve previous query in dynamoDB

I have 100 rows of data in DynamoDB and a api with path api/get/{number}
Now when I say number=1 api should return me first 10 values. when I say number=2 it should return next 10 values. I did something like this with query, lastEvaluatedKey and sort by on createdOn . Now the use case is if the user passes number=10 after number=2 the lastEvaluatedKey is still that of page 2 and the result would be data of page 3. How can I get data directly. Also if the user goes from number=3 to number=1 still the data will not be of page 1.
I am using this to make API call based of pagination on HTML.
I am using java 1.8 and aws-java-sdk-dynamodb.
Non-sequential pagination in DynamoDB is tough - you have to design your data model around it, if it's an operation that needs to be efficient at all times. For a recommendation in your specific case I'd need more details about the data and access patterns.
In general you have the option of setting the ExclusiveStartKey attribute in the query call, which is similar to an offset in relational databases, but only similar and not identical. The ExclusiveStartKey is the key after which the query will continue, meaning data from your table and not just a number.
That means you usually can't guess it, unless it's a sequential number - which isn't ideal.
For sequential pagination, i.e. the user goes from page 1 to page 2, page 2 to page 3 etc. you can pass that along in the request as a token, but that won't work if the user moves in the other direction page 3 to page 2 or just randomly navigates to page 14.
In your case you only have a limited amount of data - 100 items, so my solution for your specific case would be to query all items and limit the amount of items in the response to n * 10, where n is the result page. Then you return the last 10 items from that result to your client.
This is a solution that would get expensive at scale (time + cost) though, fortunately not many people will use the pagination to go to page 7 or 8 though (you could bury a body on page 2 of the google search results).
Yan Cui has written an interesting post on this problem on Hackernoon, you might want to check it out.

How to combine multiple page records?

I have a model called DemoModel and contains 1000 records in DB. So i am paginating using paginator in Django(assume that per page 15 records, so i have 67 pages).
So i want to get the records of 3,4 and 5 pages and i have to append the records into list.
So can i get the objects_list based on page range or anything else i want to do?
Example:
records.page(1)
Here i am getting only one page records at a time, but how can i get multiple page records i.e; from fist page to third page
Assuming you are asking about the API request to get the paginated resources, and you are using the default pagination class: rest_framework.pagination.LimitOffsetPagination, then you can make an request as such:
https://api.example.org/accounts/?limit=30&offset=15
which in turns give you the 2nd and 3rd "page".
The limit indicates the maximum number of items to return, and is equivalent to the page_size in other styles. The offset indicates the starting position of the query in relation to the complete set of unpaginated items. doc link

How to deal with intermediate additions while paginating a list with Spring Data MongoDB?

I'm trying to create a backend in Spring Data Mongodb. I have the following code which works and I have used the built in methods by extending my repo with the MongoRepository class:
#RequestMapping(value="/nextpost", method=RequestMethod.GET)
public List getNextPosts(#RequestParam int next) {
Pageable pageable = new PageRequest(next, 5, new Sort(new Sort.Order(Direction.DESC, "id")));
page = repo.findAll(pageable);
return page.getContent();
}
The above code will return the page as per the page number inserted into the "next" variable.
My android frontend however allows for things to be added and deleted from the database and this causes problems with this pagination method. Lets take an example:
When my android frontend starts up, it loads the first 5 items by
calling "getNextPosts" with next = 0.
My android frontend also keeps track of the page it is on and
increments it when the user wants to see more items.
Now, we immediately add 5 more items.
When I swipe up to fetch the next 5 items, it calls the
"getNextPosts" method passing the the "next page" value = 1. The app
will load the
same 5 items originally displayed when the app was started as the 5 "NEW" items I have added just pushed the 5 "OLD" items down in
the database.
Therefore on the app, we see 15 items comprising of:
5 "NEW" + 5 "OLD" + 5 "OLD"
So if I gave numbers to all my items on my android ListView, I would see:
15
14
13
12
11
// the above would be the new items added
10
9
8
7
6
//the above would be the original items on page 0
10
9
8
7
6
//the above would be still be the original items but now we are on page 1
Does anyone know how one can solve this issue so that when I swipe up, the items would be:
15
14
13
12
11
// the above would be the new items added
10
9
8
7
6
5
//the above would be the original items on page 0
4
3
2
1
0
//the above would be on page 1
tl;dr
That's the nature of the beast. Pagination in Spring Data is defined as retrieving a part of the result set at the time of querying. Especially for remote communication, that kind of statelessness is usually the best tradeoff between keeping state, keeping connections open, scalability etc.
Details
The only way to avoid this would be to capture the state of the database at the time of the first access and only work on that. You can actually build this by retrieving all items and page through the data locally.
Of course hardly anyone does this as it easily gets out of hand for larger data volumes. Also, this would bring up other problems like: when do you actually want to see the items introduced in the meantime? So the definition of "correct content" when paginating a list is not distinct.
Mitigation strategies
If applicable to your scenario you could try to apply a sorting that guarantees new items to be added at the very end and thus basically making this an append-only list. This would naturally sort the most recent items last though, which is contrary to what's needed often times.
If you use the pagination to work down a list of items and process all of them, another approach is to keep track of the identifiers of the items you already have processed. In your particular scenario, you'd be able to detect that the items have already been processed and go on with the next page. This of course only makes sense if you read and process faster than someone else manipulates the list in the backend.
Another solution could be to store an insert timestamp into the db for each entry. This enables you to create deterministic pagination queries:
The moment you initialize pagination (querying first page) you restrict items to have an insert timestamp lower equals than now(). You have to save now() as the pagination timestamp for querying more pages in the future. Since newly added items all get an insert timestamp greater than the pagination timestamp those items won't affect existing paginations.
Please keep in mind that new items won't show until you re-initialize pagination by refreshing the pagination timestmap. But you can simply check for the existence of new items by counting the number of items with an insertion timestamp greater than the pagination timestamp and in this case show a refresh button or something like that.

Does a report's maximum row count affect the style of pagination in Oracle Application Express (APEX) 3.1?

I am trying to make a report use a drop-down list (select list) for pagination on a report however I have found that if the maximum row count for the report is set to a value above 8000 that APEX will use a pagination style of next and previous links instead. Is this a known bug of Oracle APEX 3.1?
EDIT: Pagination works as expected when the maximum row count is set to a value below 8000.
The root cause of the problem was that the number of rows for each page in the report was statically assigned in a named list of value (VALUE) and tied to the page item.
I then removed that item reference and the select list was used for the pagination scheme!
I have pictures of the pagination scheme settings but my reputation is still too low to post them.

Resources