Im trying to implement a search in spring boot. Since the search params is dynamic, i had to go for specification api. My requirement is to search for orders given certain params and sort those orders by creation date. Since data can be large, the api should also support for pagination.
Below is the code snipped of the predicates.
public search(OrderSearchCriteria searchCriteria, Integer pageNo, Integer pageSize) {
Pageable pageRequest = (!ObjectUtils.isEmpty(pageNo) && !ObjectUtils.isEmpty(pageSize))
? PageRequest.of(pageNo, pageSize)
: Pageable.unpaged();
Page<Order> ordersPage = this.dao.findAll(((Specification<Order>)(root, query, criteriaBuilder) -> {
// my search params, its more than what is shown here
Long id = searchCriteria.getId();
Priority priority = searchCriteria.getPriority();
List<Predicate> predicates = new ArrayList<>();
if(!ObjectUtils.isEmpty(id))
predicates.add(criteriaBuilder.and(criteriaBuilder.equal(root.get("id"), id)));
if(!ObjectUtils.isEmpty(priority))
predicates.add(criteriaBuilder.and(criteriaBuilder.equal(root.get("priority"), priority)));
// This line is causing the issue
query.orderBy(criteriaBuilder.desc(root.get("creationDate")));
return criteriaBuilder.and(predicates.toArray(new Predicate[0]));
}), pageRequest);
The issue is, if i add this sort by line query.orderBy(criteriaBuilder.desc(root.get("creationDate")))
Pagination is not working. page 0 and page 1 both shows same result(same order) given size as 1. But page 2 shows different result as expected. But if i remove the sort by line shown above, the code is working as expected. How do i support both pagination and sorting without having these issues? i tried applying sort in pagerequest as well. But same issue PageRequest.of(pageNo, size, sort.by(DESC, "creationDate")). Appreciate help.
What do you mean by "it shows same result"? There might be multiple entries that have the same creation date and if you omit a sort, Spring Data will by default sort by id to provide a consistent result.
Related
I need to apply pagination in a spring boot project.
I apply pagination in 2 queries. Each of them gives me data from different tables. Now, some of these records are identical in the two tables hence need to be removed.
At the end, the number of entries that I need to send will be reduced, thereby ruining the initial pagination applied. How do I go about this? What should be my approach?
Here I take 2 lists from 2 jpa calls(highRiskCust and amlPositiveCust) that will apply pagination, then remove the duplicacy and return the final result (tempReport).
`
List<L1ComplianceResponseDTO> highRiskCust = customerKyc.findAllHighRiskL1Returned(startDateTime, endDateTime,agentIds);
List<L1ComplianceResponseDTO> amlPositiveCust = customerKyc.findAllAmlPositiveL1Returned(startDateTime, endDateTime,agentIds);
List<L1ComplianceResponseDTO> tempReport = new ArrayList<>();
tempReport.addAll(amlPositiveCust);
tempReport.addAll(highRiskCust);
tempReport = tempReport.stream().filter(distinctByKey(p -> p.getKycTicketId()))
.collect(Collectors.toList());
`
In order to have pagination working, you need to do it with a unique request.
Fondammently this request should use UNION.
Since JPA does not support UNION, either you do a native query or you change you query logic using outer joins.
Good Morning.
I'm starting to learn some mongo right now.
I'm facing this problem right now, and i'm start to think if this is the best approach to resolve this "task", or if is bettert to turn around and write another way to solve this "problem".
My goal is to iterate a simple map of values (key) and vector\array (values)
My test map will be recived by a rest layer.
{
"1":["1","2","3"]
}
now after some logic, i need to use the Dao in order to look into db.
The Key will be "realm", the value inside vector are "castle".
Every Realm have some castle and every castle have some "rules".
I need to find every rules for each avaible combination of realm-castle.
AccessLevel is a pojo labeled by #Document annotation and it will have various params, such as castle and realm (both simple int)
So the idea will be to iterate a map and write a long query for every combination of key-value.
public AccessLevel searchAccessLevel(Map<String,Integer[]> request){
Query q = new Query();
Criteria c = new Criteria();
request.forEach((k,v)-> {
for (int i: Arrays.asList(v)
) {
q.addCriteria(c.andOperator(
Criteria.where("realm").is(k),
Criteria.where("castle").is(v))
);
}
});
List<AccessLevel> response=db.find(q,AccessLevel.class);
for (AccessLevel x: response
) {
System.out.println(x.toString());
}
As you can see i'm facing an error concerning $and.
Due to limitations of the org.bson.Document, you can't add a second '$and' expression specified as [...]
it seems mongo can't handle various $and, something i'm pretty used to abuse over sql
select * from a where id =1 and id=2 and id=3 and id=4
(not the best, sincei can use IN(), but sql allow me)
So, the point is: mongo can actualy work in this way and i need to dig more into the problem, or i need to do another approach, like using criterion.in(), and make N interrogation via mongotemplate one for every key in my Map?
in our Spring application we have a table that contains a lot of "Payment" record. Now we need a query that pages the results sorted from the one with the largest total to the smallest, we are facing an error because sometimes the same record is contained in two successive pages.
We are creating a PageRequest passed to the repository. Here our implementation:
Repository:
public interface StagingPaymentEntityRepository extends JpaRepository<StagingPaymentEntity, Long> {
Page<StagingPaymentEntity> findAllByStatusAndCreatedDateLessThanEqualAndOperationTypeOrderByEffectivePaymentDesc(String status, Timestamp batchStartTimestamp, String operationType, Pageable pageable);
}
public class BatchThreadReiteroStorni extends ThreadAbstract<StagingPaymentEntity> {
PageRequest pageRequest = PageRequest.of (index, 170);
Page<StagingPaymentEntity> records = ((StagingPaymentEntityRepository) repository).findAllByStatusAndCreatedDateLessThanEqualAndOperationTypeOrderByEffectivePaymentDesc("REITERO", batchStartTimestamp, "STORNO", pageRequest) ;
}
where index is the index of the page we are requesting.
There is a way to understand why it is happening ? Thank for support
This can have multiple reasons.
Non deterministic ordering: If the ordering you are using isn't deterministic, i.e. there are rows that might com in any order that order might change between selects resulting in items getting skipped or returned multiple times. Fix: add the primary key as a last column to the ordering.
If you change the entities in a way that affects the ordering, or another process does that you might end up with items getting processed multiple times.
In this scenario I see a couple of approaches:
do value based pagination. I.e. don't select pages but select the next N rows after .
Instead of paging use a Stream this allows to use a single select but still processing the results an element at a time. You might have to flush and evict entities and I'm not 100% sure that works, but certainly worth a try.
Finally you can mark all all rows that you want to process in a separate column, then select N marked entities and unmark them once they are processed.
I have created two entities in JPA, Listing and ItemType - these exist in a many-to-many relationship (Hibernate auto-generates a junction table). I'm trying to find the best way to create a query which accepts a dynamic list of item type Strings and returns the IDs of all listings which match the specified item types, but I am a recent initiate in JPA.
At present I'm using JpaRepository to create relatively simple queries. I've been trying to do this using CriteriaQuery but some close-but-not-quite answers I've read elsewhere seem to suggest that because this is in Spring, this may not be the best approach and that I should be handling this using the JpaRepository implementation itself. Does that seem reasonable?
I have a query which doesn't feel a million miles away (based on Baeldung's example and my reading on WikiBooks) but for starters I'm getting a Raw Type warning on the Join, not to mention that I'm unsure if this will run and I'm sure there's a better way of going about this.
public List<ListingDTO> getListingsByItemType(List<String> itemTypes) {
List<ListingDTO> listings = new ArrayList<>();
CriteriaQuery<Listing> criteriaQuery = criteriaBuilder.createQuery(Listing.class);
Root<Listing> listing = criteriaQuery.from(Listing.class);
//Here Be Warnings. This should be Join<?,?> but what goes in the diamond?
Join itemtype = listing.join("itemtype", JoinType.LEFT);
In<String> inClause = criteriaBuilder.in(itemtype.get("name"));
for (String itemType : itemTypes) {
inClause.value(itemType);
}
criteriaQuery.select(listing).where(inClause);
TypedQuery<Listing> query = entityManager.createQuery(criteriaQuery);
List<Listing> results = query.getResultList();
for (Listing result : results) {
listings.add(convertListingToDto(result));
}
return listings;
}
I'm trying to understand how best to pass in a dynamic list of names (the field in ItemType) and return a list of unique ids (the PK in Listing) where there is a row which matches in the junction table. Please let me know if I can provide any further information or assistance - I've gotten the sense that JPA and its handling of dynamic queries like this is part of its bread and butter!
The criteria API is useful when you need to dynamically create a query based on various... criteria.
All you need here is a static JPQL query:
select distinct listing from Listing listing
join listing.itemTypes itemType
where itemType.name in :itemTypes
Since you're using Spring-data-jpa, you just need to define a method and annotate it with #Query in your repository interface:
#Query("<the above query>")
List<Listing> findByItemTypes(List<String> itemTypes)
I am integrating FOSElasticaBundle in my Symfony 2.3 project and I need to sort the results by their price property.
Here is my code:
$finder = $this->container->get('fos_elastica.finder.website.product');
$fieldTerms = new \Elastica\Query\Terms();
$fieldTerms->setTerms('taxon_ids', $taxon_ids_array);
$boolQuery->addMust($fieldTerms);
$resultSet = $finder->find($boolQuery);
How I can do this?
Thanks
Try create a \Elastica\Query object which also contains the sorting information, then send this to the finder:
$finder = $this->container->get('fos_elastica.finder.website.product');
$fieldTerms = new \Elastica\Query\Terms();
$fieldTerms->setTerms('taxon_ids', $taxon_ids_array);
$boolQuery->addMust($fieldTerms);
$finalQuery = new \Elastica\Query($boolQuery);
$finalQuery->setSort(array('price' => array('order' => 'asc')));
$resultSet = $finder->find($finalQuery);
Have a look at the elasticsearch docs on the sort parameter to see how to use it properly.
NOTE: \Elastica\Query is quite different to \Elastica\Query\AbstractQuery, the first encapsulates everything you could send to the _search API endpoint (facets, sorting, explain, etc...) The AbstractQuery represents a base type for each of the individual query types (range, fuzzy, terms, etc...).