I am stuck with integrating pagination in Spring boot project.
Service Impl class
#Override
public Page<OnlineBuyProductReportItemDTO> generateOnlineBuyProductReport(String fromDate, String toDate, int pageNum, String sortField, String sortDir) {
Pageable pageable = PageRequest.of(pageNum - 1, 10,
sortDir.equals("asc") ? Sort.by(sortField).ascending()
: Sort.by(sortField).descending()
);
List<PurchaseOrder> purchaseOrderList = purchaseOrderRepository.searchPurchaseOrdersForOnlineBuyProductReport(
DateUtil.convertToDate(fromDate), DateUtil.convertToDate(toDate));
OnlineBuyProductReportItemDTO item = null;
List<OnlineBuyProductReportItemDTO> itemList = new ArrayList<OnlineBuyProductReportItemDTO>();
if(purchaseOrderList != null && purchaseOrderList.size() > 0) {
for (PurchaseOrder purchaseOrder : purchaseOrderList) {
item = new OnlineBuyProductReportItemDTO();
item.setOrderNumber(purchaseOrder.getOrderNumber());
item.setCreatedDate(DateUtil.convertDatetoStringSwissDate(purchaseOrder.getCreationTime()));
Address address = addressRepository.getAddressById(purchaseOrder.getBillingAddressId());
item.setCustomerName(address.getFullName());
item.setContactNumber(address.getTelephone());
List<OrderCart> orderCartList = orderCartRepository.getOrderCartByPurchaseOrderId(purchaseOrder.getId());
List<String> buyProductNameList = new ArrayList<String>();
if(orderCartList != null && orderCartList.size() > 0) {
for (OrderCart orderCart : orderCartList) {
buyProductNameList.add(orderCart.getProductName());
}
}
item.setProductName(buyProductNameList);
item.setPrice(purchaseOrder.getTotalPrice());
item.setStatus(purchaseOrder.getDeliveryStatus().getName());
itemList.add(item);
}
}
return new PageImpl<OnlineBuyProductReportItemDTO>(itemList, pageable,itemList.size());
}
Here I have fetch data from 3 database tables and after data processing I have added to these data to DTO List.
But, when after integrated with the front end, pagination options are displayed but did't work as expected.
All the pages containing same data set and also sorting options are not working as expected.
It seems this pageable part is not applying to the data processing.
Pageable pageable = PageRequest.of(pageNum - 1, 10,
sortDir.equals("asc") ? Sort.by(sortField).ascending()
: Sort.by(sortField).descending()
);
Is there any way to do this
Related
I have a Service entity that has a field Jobs which has Job entity in a ManyToMany relationship.
Filtering, sorting and pagination works perfectly, but the only problem is sorting by Jobs.
When I send a request to /serviceslist?sort=jobs&sortDir=asc I get duplicate results for each Service that has multiple jobs connected to it. To be more specific, if a Service has 4 jobs, the service gets returned 4 times and so on.
I think I should use groupBy, but how would I implement groupBy in my service below?
Service:
public Page<com.bitweb.syda.data.entity.service.Service> getServicesList(ServiceListRequest request, Pageable pageable) {
Specification<com.bitweb.syda.data.entity.service.Service> spec = where(null);
if (request.getSearch() != null) spec = spec.and(search(request.getSearch()));
if (request.getName() != null) spec = spec.and(name(request.getName()));
if (request.getJobs() != null) spec = spec.and(hasJobs(request.getJobs()));
if (request.getNeedsPatrol() != null) spec = spec.and(needsPatrol(request.getNeedsPatrol()));
return serviceRepository.findAll(spec, pageable);
}
Controller:
#RequestMapping(path = "/serviceslist", method = RequestMethod.GET)
public Page<ServiceResponse> getServicesList(
#RequestParam(defaultValue = "0") Integer page,
#RequestParam(defaultValue = "10") Integer size,
#RequestParam(required = false) String search,
#RequestParam(required = false) String name,
#RequestParam(required = false) String jobs,
#RequestParam(required = false) Boolean needsPatrol,
#RequestParam(defaultValue = "createTime") String sort,
#RequestParam(defaultValue = "asc") String sortDir
) {
ServiceListRequest request = new ServiceListRequest(search, name, jobs, needsPatrol);
Sort.Direction direction;
if (sortDir.equals("asc")) {
direction = Sort.Direction.ASC;
} else {
direction = Sort.Direction.DESC;
}
return serviceService.getServicesList(request, of(page, size, direction, sort))
.map(ServiceResponse::new);
}
Here are some examples what I've tried already:
public Page<com.bitweb.syda.data.entity.service.Service> getServicesList(ServiceListRequest request, Pageable pageable) {
Specification<com.bitweb.syda.data.entity.service.Service> spec = (root, query, builder) -> {
//you can do any check here if you want with the join and check all the search parameters here if you want
//Join<Object, Object> jobs = root.join("jobs");
// also set query to get distinct values
query.distinct(true);
return null;
};
if (request.getSearch() != null) spec = spec.and(search(request.getSearch()));
if (request.getName() != null) spec = spec.and(name(request.getName()));
if (request.getJobs() != null) spec = spec.and(hasJobs(request.getJobs()));
if (request.getNeedsPatrol() != null) spec = spec.and(needsPatrol(request.getNeedsPatrol()));
return serviceRepository.findAll(spec, pageable);
}
After trying this everything works as expected, but sorting by jobs gives me this error:
PSQLException: ERROR: for SELECT DISTINCT, ORDER BY expressions must appear in select list
Position: 652
I'm not very proficient with SQL and Criteria API and I don't know what I could do to fix this.
When using query.groupBy(root); instead of distinct, I get this error:
PSQLException: ERROR: column "job4_.id" must appear in the GROUP BY clause or be used in an aggregate function
Position: 780
I have a ldap method that returns all users that are in it (almost 1300 users) and I want to return them by page, similar to what PagingAndSortingRepository does in Springboot:
If I have this endpoint ( users/?page=0&size=1 )and I wnat to return on page 0 just 1 entry.
Is there any way to do that?
Currently I have this but it doesn´t work:
SearchRequest searchRequest = new SearchRequest(ldapConfig.getBaseDn(), SearchScope.SUB,
Filter.createEqualityFilter("objectClass", "person"));
ASN1OctetString resumeCookie = null;
while (true) {
searchRequest.setControls(new SimplePagedResultsControl(pageable.getPageSize(), resumeCookie));
SearchResult searchResult = ldapConnection.search(searchRequest);
numSearches++;
totalEntriesReturned += searchResult.getEntryCount();
for (SearchResultEntry e : searchResult.getSearchEntries()) {
String[] completeDN = UaaUtils.searchCnInDn(e.getDN());
String[] username = completeDN[0].split("=");
UserEntity u = new UserEntity(username[1]);
list.add(u);
System.out.println("TESTE");
}
SimplePagedResultsControl responseControl = SimplePagedResultsControl.get(searchResult);
if (responseControl.moreResultsToReturn()) {
// The resume cookie can be included in the simple paged results
// control included in the next search to get the next page of results.
System.out.println("Antes "+resumeCookie);
resumeCookie = responseControl.getCookie();
System.out.println("Depois "+resumeCookie);
} else {
break;
}
Page<UserEntity> newPage = new PageImpl<>(list, pageable, totalEntriesReturned);
System.out.println("content " + newPage.getContent());
System.out.println("total elements " + newPage.getTotalElements());
System.out.println(totalEntriesReturned);
}
I'm unsure if this is the proper way, but here's how I went about it:
public PaginatedLookup getAll(String page, String perPage) {
PagedResultsCookie cookie = null;
List<LdapUser> results;
try {
if ( page != null ) {
cookie = new PagedResultsCookie(Hex.decode(page));
} // end if
Integer pageSize = perPage != null ? Integer.parseInt(perPage) : PROCESSOR_PAGE_SIZE;
PagedResultsDirContextProcessor processor = new PagedResultsDirContextProcessor(pageSize, cookie);
LdapName base = LdapUtils.emptyLdapName();
SearchControls sc = new SearchControls();
sc.setSearchScope(SearchControls.SUBTREE_SCOPE);
sc.setTimeLimit(THREE_SECONDS);
sc.setCountLimit(pageSize);
sc.setReturningAttributes(new String[]{"cn", "title"});
results = ldapTemplate.search(base, filter.encode(), sc, new PersonAttributesMapper(), processor);
cookie = processor.getCookie();
} catch ( Exception e ) {
log.error(e.getMessage());
return null;
} // end try-catch
String nextPage = null;
if ( cookie != null && cookie.getCookie() != null ) {
nextPage = new String(Hex.encode(cookie.getCookie()));
} // end if
return new PaginatedLookup(nextPage, results);
}
The main issue I kept on hitting was trying to get the cookie as something that could be sent to the client, which is where my Hex.decode and Hex.encode came in handy.
PersonAttributesMapper is a private mapper that I have to make the fields more human readable, and PaginatedLookup is a custom class I use for API responses.
I am new to ElasticsearchTemplate. I want to get 1000 documents from Elasticsearch based on my query.
I have used QueryBuilder to create my query , and it is working perfectly.
I have gone through the following links , which states that it is possible to achieve big data sets using scan and scroll.
link one
link two
I am trying to implement this functionality in the following section of code, which I have copy pasted from one of the link , mentioned above.
But I am getting following error :
The type ResultsMapper is not generic; it cannot be parameterized with arguments <myInputDto>.
MyInputDto is a class with #Document annotation in my project.
End of the day , I just want to retrieve 1000 documents from Elasticsearch.
I tried to find size parameter but I think it is not supported.
String scrollId = esTemplate.scan(searchQuery, 1000, false);
List<MyInputDto> sampleEntities = new ArrayList<MyInputDto>();
boolean hasRecords = true;
while (hasRecords) {
Page<MyInputDto> page = esTemplate.scroll(scrollId, 5000L,
new ResultsMapper<MyInputDto>() {
#Override
public Page<MyInputDto> mapResults(SearchResponse response) {
List<MyInputDto> chunk = new ArrayList<MyInputDto>();
for (SearchHit searchHit : response.getHits()) {
if (response.getHits().getHits().length <= 0) {
return null;
}
MyInputDto user = new MyInputDto();
user.setId(searchHit.getId());
user.setMessage((String) searchHit.getSource().get("message"));
chunk.add(user);
}
return new PageImpl<MyInputDto>(chunk);
}
});
if (page != null) {
sampleEntities.addAll(page.getContent());
hasRecords = page.hasNextPage();
} else {
hasRecords = false;
}
}
What is the issue here ?
Is there any other alternative to achieve this?
I will be thankful if somebody could tell me how this ( code ) is working in the back end.
Solution 1
If you want to use ElasticsearchTemplate, it would be much simpler and readable to use CriteriaQuery, as it allows to set the page size with setPageable method. With scrolling, you can get next sets of data:
CriteriaQuery criteriaQuery = new CriteriaQuery(Criteria.where("productName").is("something"));
criteriaQuery.addIndices("prods");
criteriaQuery.addTypes("prod");
criteriaQuery.setPageable(PageRequest.of(0, 1000));
ScrolledPage<TestDto> scroll = (ScrolledPage<TestDto>) esTemplate.startScroll(3000, criteriaQuery, TestDto.class);
while (scroll.hasContent()) {
LOG.info("Next page with 1000 elem: " + scroll.getContent());
scroll = (ScrolledPage<TestDto>) esTemplate.continueScroll(scroll.getScrollId(), 3000, TestDto.class);
}
esTemplate.clearScroll(scroll.getScrollId());
Solution 2
If you'd like to use org.elasticsearch.client.Client instead of ElasticsearchTemplate, then SearchResponse allows to set the number of search hits to return:
QueryBuilder prodBuilder = ...;
SearchResponse scrollResp = client.
prepareSearch("prods")
.setScroll(new TimeValue(60000))
.setSize(1000)
.setTypes("prod")
.setQuery(prodBuilder)
.execute().actionGet();
ObjectMapper mapper = new ObjectMapper();
List<TestDto> products = new ArrayList<>();
try {
do {
for (SearchHit hit : scrollResp.getHits().getHits()) {
products.add(mapper.readValue(hit.getSourceAsString(), TestDto.class));
}
LOG.info("Next page with 1000 elem: " + products);
products.clear();
scrollResp = client.prepareSearchScroll(scrollResp.getScrollId())
.setScroll(new TimeValue(60000))
.execute()
.actionGet();
} while (scrollResp.getHits().getHits().length != 0);
} catch (IOException e) {
LOG.error("Exception while executing query {}", e);
}
I'm having some trouble doing aggregation on a particular column in HBase.
This is the snippet of code I tried:
Configuration config = HBaseConfiguration.create();
AggregationClient aggregationClient = new AggregationClient(config);
Scan scan = new Scan();
scan.addColumn(Bytes.toBytes("drs"), Bytes.toBytes("count"));
ColumnInterpreter<Long, Long> ci = new LongColumnInterpreter();
Long sum = aggregationClient.sum(Bytes.toBytes("DEMO_CALCULATIONS"), ci , scan);
System.out.println(sum);
sum returns a value of null.
The aggregationClient API works fine if I do a rowcount.
I was trying to follow the directions in http://michaelmorello.blogspot.in/2012/01/row-count-hbase-aggregation-example.html
Could there be a problem with me using a LongColumnInterpreter when the 'count' field was an int? What am I missing in here?
You can only use long(8bytes) to do sum with default setting.
Cause in the code of AggregateImplementation's getSum method, it handle all the returned KeyValue as long.
List<KeyValue> results = new ArrayList<KeyValue>();
try {
boolean hasMoreRows = false;
do {
hasMoreRows = scanner.next(results);
for (KeyValue kv : results) {
temp = ci.getValue(colFamily, qualifier, kv);
if (temp != null)
sumVal = ci.add(sumVal, ci.castToReturnType(temp));
}
results.clear();
} while (hasMoreRows);
} finally {
scanner.close();
}
and in LongColumnInterpreter
public Long getValue(byte[] colFamily, byte[] colQualifier, KeyValue kv)
throws IOException {
if (kv == null || kv.getValueLength() != Bytes.SIZEOF_LONG)
return null;
return Bytes.toLong(kv.getBuffer(), kv.getValueOffset());
}
I'm using MVC3 ajaxgrid scaffolding with EF4.1 code first and i've this error:
Cannot implicitly convert type 'System.Linq.IQueryable' to 'System.Data.Objects.ObjectQuery'
The code with the error in, is autogenerated:
public ActionResult GridData(int start = 0, int itemsPerPage = 20, string orderBy = "UserID", bool desc = false)
{
Response.AppendHeader("X-Total-Row-Count", repository.Users.Count().ToString());
ObjectQuery<User> users = (repository as IObjectContextAdapter).ObjectContext.CreateObjectSet<User>();
users = repository.Users.Include(u => u.Role); //ERROR HERE
users = users.OrderBy("it." + orderBy + (desc ? " desc" : ""));
return PartialView(users.Skip(start).Take(itemsPerPage));
}
This is the Users repository method and the Roles Foreign Key
public IQueryable<Entities.User> Users
{
get { return context.Users; }
}
public IQueryable<Entities.Role>Roles
{
get { return context.Roles; }
}
How can i resolve the conversion?
Get rid of the Lambda and use the related object:
var users = repository.Users.Include("Role"); //ERROR HERE
Assuming the entity User has a navigational property Role.
The reason is clear:
You have users variable with ObjectQuery<User> type then you assign that variable result of a query which is IQueryable<User>.
UPDATE:
Try the code below:
public ActionResult GridData(int start = 0, int itemsPerPage = 20, string orderBy = "UserID", bool desc = false)
{
Response.AppendHeader("X-Total-Row-Count", repository.Users.Count().ToString());
//ObjectQuery<User> users = (repository as IObjectContextAdapter).ObjectContext.CreateObjectSet<User>();
var users = repository.Users.Include(u => u.Role); //ERROR HERE
users = users.OrderBy("it." + orderBy + (desc ? " desc" : ""));
return PartialView(users.Skip(start).Take(itemsPerPage));
}