I am using Spring Data JPA and I have a repository which extends PagingAndSortingRepository. My issue is that I have a query in which returns all of the results instead of the desired page size (100). I can't seem to find an issue that is wrong. Can anyone assist please?
#Test
public void testFindPageByStartAndEndDate() {
Timestamp endDate = Timestamp.valueOf("2017-06-14 09:18:42");
Timestamp startDate = Timestamp.valueOf("2017-05-19 01:31:23");
PageRequest pageRequest1 = new PageRequest(0, 100, Sort.Direction.ASC, "orderDate");
Page<Order> page1Orders = orderRepository.findPageByStartAndEndDate(startDate, endDate, pageRequest1);
assertThat(page1Orders.getTotalElements(), greaterThan(0L));
//
//
// Test Fails Here
// Expected: a value less than <101L>
// but: <139L> was greater than <101L>
//
//
assertThat(page1Orders.getTotalElements(), lessThan(101L));
}
This is the query that I am using.
#Query("SELECT o FROM Order o WHERE o.orderDate >= ?1 AND o.orderDate <= ?2")
Page<Order> findPageByStartAndEndDate(#Param("startDate") Timestamp startDate,
#Param("endDate") Timestamp endDate,
Pageable pageable);
getTotalElements() returns the count without pagination.
You get the number of elements present in the slice with getNumberOfElements().
See the implementation of Page as a reference here.
Related
I am trying to sort my table's content on the backend side, so I am sending org.springframework.data.domain.Pageable object to controller. It arrives correctly, but at the repository I am getting org.hibernate.hql.internal.ast.InvalidPathException. Somehow the field name I would use for sorting gets an org. package name infront of the filed name.
The Pageable object logged in the controller:
Page request [number: 0, size 10, sort: referenzNumber: DESC]
Exception in repository:
Invalid path: 'org.referenzNumber'","logger_name":"org.hibernate.hql.internal.ast.ErrorTracker","thread_name":"http-nio-8080-exec-2","level":"ERROR","level_value":40000,"stack_trace":"org.hibernate.hql.internal.ast.InvalidPathException: Invalid path: 'org.referenzNumber'\n\tat org.hibernate.hql.internal.ast.util.LiteralProcessor.lookupConstant(LiteralProcessor.java:111)
My controller endpoint:
#GetMapping(value = "/get-orders", params = { "page", "size" }, produces = { MediaType.APPLICATION_JSON_VALUE })
public ResponseEntity<PagedModel<KryptoOrder>> getOrders(
#ApiParam(name = "searchrequest", required = true) #Validated final OrderSearchRequest orderSearchRequest,
#PageableDefault(size = 500) final Pageable pageable, final BindingResult bindingResult,
final PagedResourcesAssembler<OrderVo> pagedResourcesAssembler) {
if (bindingResult.hasErrors()) {
return ResponseEntity.badRequest().build();
}
PagedModel<Order> orderPage = PagedModel.empty();
try {
var orderVoPage = orderPort.processOrderSearch(resourceMapper.toOrderSearchRequestVo(orderSearchRequest), pageable);
orderPage = pagedResourcesAssembler.toModel(orderVoPage, orderAssembler);
} catch (MissingRequiredField m) {
log.warn(RESPONSE_MISSING_REQUIRED_FIELD, m);
return ResponseEntity.badRequest().build();
}
return ResponseEntity.ok(orderPage);
}
the repository:
#Repository
public interface OrderRepository extends JpaRepository<Order, UUID> {
static final String SEARCH_ORDER = "SELECT o" //
+ " FROM Order o " //
+ " WHERE (cast(:partnerernumber as org.hibernate.type.IntegerType) is null or o.tradeBasis.account.retailpartner.partnerbank.partnerernumber = :partnerernumber)"
+ " and (cast(:accountnumber as org.hibernate.type.BigDecimalType) is null or o.tradeBasis.account.accountnumber = :accountnumber)"
+ " and (cast(:orderReference as org.hibernate.type.LongType) is null or o.tradeBasis.referenceNumber = :orderReference)"
+ " and (cast(:orderReferenceExtern as org.hibernate.type.StringType) is null or o.tradeBasis.kundenreferenceExternesFrontend = :orderReferenceExtern)"
+ " and (cast(:dateFrom as org.hibernate.type.DateType) is null or o.tradeBasis.timestamp > :dateFrom) "
+ " and (cast(:dateTo as org.hibernate.type.DateType) is null or o.tradeBasis.timestamp < :dateTo) ";
#Query(SEARCH_ORDER)
Page<Order> searchOrder(#Param("partnerernumber") Integer partnerernumber,
#Param("accountnumber") BigDecimal accountnumber, #Param("orderReference") Long orderReference,
#Param("orderReferenceExtern") String orderReferenceExtern, #Param("dateFrom") LocalDateTime dateFrom,
#Param("dateTo") LocalDateTime dateTo, Pageable pageable);
}
Update:
I removed the parameters from the sql query, and put them back one by one to see where it goes sideways. It seems as soon as the dates are involved the wierd "org." appears too.
Update 2:
If I change cast(:dateTo as org.hibernate.type.DateType) to cast(:dateFrom as date) then it appends the filed name with date. instead of org..
Thanks in advance for the help
My guess is, Spring Data is confused by the query you are using and can't properly append the order by clause to it. I would recommend you to use a Specification instead for your various filters. That will not only improve the performance of your queries because the database can better optimize queries, but will also make use of the JPA Criteria API behind the scenes, which requires no work from Spring Data to apply an order by specification.
Since your entity Order is named as the order by clause of HQL/SQL, my guess is that Spring Data tries to do something stupid with the string to determine the alias of the root entity.
I'm setting up a method to filter entries by the "lastupdated" column. I'm trying to filter entries of which the lastupdated value is between a given startTime and endTime.
I'm using the simplified code below:
public List<SomeEntity> getItemsByLastUpdated() {
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery<SomeEntity> criteriaQuery = cb.createQuery(SomeEntity.class);
var root = criteriaQuery.from(SomeEntity.class);
var predicates = new ArrayList<>();
var startTime = Instant.now();
var endTime = Instant.now().plus(5, MINUTES);
predicates.add(cb.greaterThanOrEqualTo(root.get("lastupdated"), startTime));
predicates.add(cb.lessThan(root.get("lastupdated"), endTime));
criteriaQuery.where(predicates.toArray(new Predicate[0]));
return entityManager.createQuery(criteriaQuery).getResultList();
}
For some reason, the criteria for datetime fields are not (correctly) being applied: All the entities are being returned instead of only the items within the startTime-endTime range.
I'm guessing that I need to explicitly state that the "lastupdated" field is a datetime field; when I set startTime to Instant.MAX, I receive an error that seems to hint on a date filter instead of datetime:
Invalid value for EpochDay (valid values -365243219162 - 365241780471): 365241780838
Would anyone know how I can filter on a datetime field being between two given java Instants?
PS. I'm aware of using derived queries such as findByXGreaterThanOrEqualToAndXLessThan(Instant instant1, Instant instant2); but since there are several other criterias which I have not included, this option is not feasible.
I think you could easily do the same using EntityManager and JPQL instead of Criteria API:
public List<SomeEntity> getItemsByLastUpdated() {
var query = "select e from SomeEntity e
where e.lastupdated >= :from
and e.lastupdated < :to
and your_other_criteria";
return entityManager.createQuery(query)
.setParameter("from", startTime)
.setParameter("to", endTime)
.getResultList();
}
It took me quite some time to figure it out; turned out that the lack of filtering didn't have anything to do with the CriteriaBuilder setup.
I had created a test to verify the method, and prepared some entities before running the test. However, the "lastupdated" field in the entity class was annotated with #UpdateTimestamp, which would result in all the saved entities having pretty much the same 'lastupdated' value.
I'm struggling to trying the pagination feature, as described in the reference document.
This is my table schema:
CREATE TABLE cities
(
id int PRIMARY KEY,
name varchar(255),
pref_id int
);
Repository:
public interface CityRepository extends CrudRepository<CityEntity, Integer> {
Page<CityEntity> findAll(Pageable pageable);
// get all cities in the prefecture
Page<CityEntity> findByPrefId(Integer prefId, Pageable pageable);
}
Test code:
Page<CityEntity> allCities = repository.findAll(PageRequest.of(0, 10));
Page<CityEntity> cities = repository.findByPrefId(1, PageRequest.of(0, 10));
findAll works well, but findByPrefId throws the following error:
Incorrect result size: expected 1, actual 10
org.springframework.dao.IncorrectResultSizeDataAccessException: Incorrect result size: expected 1, actual 10
at org.springframework.dao.support.DataAccessUtils.nullableSingleResult(DataAccessUtils.java:100)
at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.queryForObject(NamedParameterJdbcTemplate.java:237)
at org.springframework.data.jdbc.repository.query.AbstractJdbcQuery.lambda$singleObjectQuery$1(AbstractJdbcQuery.java:115)
at org.springframework.data.jdbc.repository.query.PartTreeJdbcQuery.execute(PartTreeJdbcQuery.java:98)
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor$QueryMethodInvoker.invoke(QueryExecutorMethodInterceptor.java:195)
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:152)
at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:130)
...
If I change the method signature into List<CityEntity> findByPrefId(Integer prefId, Pageable pageable), it works.
Am I missing something? I'm using the latest version of spring-data-jdbc (2.0.2.RELEASE).
I don't know about the technicality, but this is what I learned from experience.
In your case, if the total number of cities is lesser than the pageable.getPageSize(), then your repository will return a List<>.
But if total number of cities is bigger than the pageable.getPageSize() then your repository will return a Page<>.
Knowing that, this is what I did to work around it.
Long amount = repository.countByPrefId(prefId);
if(pagination.getPageSize()>amount ) {
List<CityEntity> list = repository.findByPrefId(prefId);
} else {
Page<CityEntity> pages = repository.findByPrefId(person, PageRequest.of(0, 10));
}
This also means that in your repository you'll have two differents methods, one with Pageable as a parameter and one with only PrefId as a parameter.
I believe the accepted answer is referring to Spring Data JPA which does work by returning pages based on a count query derived from the custom query OR manually set via countQuery, no reason for the if/else.
However this flat out does not work in Spring Data JDBC.
https://jira.spring.io/browse/DATAJDBC-554
Workaround provided in link but for reference:
interface FooRepository extends PagingAndSortingRepository<FooEntity, Long> {
List<FooEntity> findAllByBar(String bar, Pageable pageable);
Long countAllByBar(String bar);
}
And then combining those 2 queries like this:
List<FooEntity> fooList = repository.findAllByBar("...", pageable);
Long fooTotalCount = repository.countAllByBar("...");
Page<FooEntity> fooPage = PageableExecutionUtils.getPage(fooList, pageable, () -> fooTotalCount);
Is it possible to fetch data in user defined ranges [int starting record -int last record]?
In my case user will define in query String in which range he wants to fetch data.
I have tried something like this
Pageable pageable = new PageRequest(0, 10);
Page<Project> list = projectRepository.findAll(spec, pageable);
Where spec is my defined specification but unfortunately this do not help.
May be I am doing something wrong here.
I have seen other spring jpa provided methods but nothing are of much help.
user can enter something like this localhost:8080/Section/employee? range{"columnName":name,"from":6,"to":20}
So this says to fetch employee data and it will fetch the first 15 records (sorted by columnName ) does not matter as of now.
If you can suggest me something better that would be great.if you think I have not provided enough information please let me know, I will provide required information.
Update :I do not want to use native or Create query statements (until I don't have any other option).
May be something like this:
Pageable pageable = new PageRequest(0, 10);
Page<Project> list = projectRepository.findAll(spec, new pageable(int startIndex,int endIndex){
// here my logic.
});
If you have better options, you can suggest me that as well.
Thanks.
Your approach didn't work, because new PageRequest(0, 10); doens't do what you think. As stated in docs, the input arguments are page and size, not limit and offset.
As far as I know (and somebody correct me if I'm wrong), there is no "out of the box" support for what you need in default SrpingData repositories. But you can create custom implementation of Pagable, that will take limit/offset parameters. Here is basic example - Spring data Pageable and LIMIT/OFFSET
We can do this with Pagination and by setting the database table column name, value & row counts as below:
#Transactional(readOnly=true)
public List<String> queryEmployeeDetails(String columnName,String columnData, int startRecord, int endRecord) {
Query query = sessionFactory.getCurrentSession().createQuery(" from Employee emp where emp.col= :"+columnName);
query.setParameter(columnName, columnData);
query.setFirstResult(startRecord);
query.setMaxResults(endRecord);
List<String> list = (List<String>)query.list();
return list;
}
If I am understanding your problem correctly, you want your repository to allow user to
Provide criteria for query (through Specification)
Provide column to sort
Provide the range of result to retrieve.
If my understanding is correctly, then:
In order to achieve 1., you can make use of JpaSpecificationExecutor from Spring Data JPA, which allow you to pass in Specificiation for query.
Both 2 and 3 is achievable in JpaSpecificationExecutor by use of Pagable. Pageable allow you to provide the starting index, number of record, and sorting columns for your query. You will need to implement your range-based Pageable. PageRequest is a good reference on what you can implement (or you can extend it I believe).
So i got this working as one of the answer suggested ,i implemented my own Pageable and overrided getPagesize(),getOffset(),getSort() thats it.(In my case i did not need more)
public Range(int startIndex, int endIndex, String sortBy) {
this.startIndex = startIndex;
this.endIndex = endIndex;
this.sortBy = sortBy;
}
#Override
public int getPageSize() {
if (endIndex == 0)
return 0;
return endIndex - startIndex;
}
#Override
public int getOffset() {
// TODO Auto-generated method stub
return startIndex;
}
#Override
public Sort getSort() {
// TODO Auto-generated method stub
if (sortBy != null && !sortBy.equalsIgnoreCase(""))
return new Sort(Direction.ASC, sortBy);
else
return new Sort(Direction.ASC, "id");
}
where startIndex ,endIndex are starting and last index of record.
to access it :
repository.findAll(spec,new Range(0,20,"id");
There is no offset parameter you can simply pass. However there is a very simple solution for this:
int pageNumber = Math.floor(offset / limit) + ( offset % limit );
PageRequest pReq = PageRequest.of(pageNumber, limit);
The client just have to keep track on the offset instead of page number. By this I mean your controller would receive the offset instead of the page number.
Hope this helps!
I have this request working good with queryDSL :
Iterable<AO> query_result = new JPAQuery(entityManager).from(ao)
.leftJoin( ao.lots , lot )
.leftJoin( ao.acs , ac )
.where(where).distinct()
.list(ao);
But what is its equivalent if we use it with spring data jpa
ao_respository.findAll(Predicate arg0, Pageable arg1);
Because i want to return a Page and just with querydsl it doesn't implement Page without spring data jpa.
I try to put my where in Predicate arg0 but i got this exception
Undeclared path 'lot '. Add this path as a source to the query to be able to reference it
where lot is declared as QLot lot = QLot.lot;
I created my own Page class and executed the query like this:
JPAQuery query = new JPAQuery(entityManager).from(ao)
.leftJoin( .. ).fetch()
.leftJoin( .. ).fetch()
...
.where(where)
MaPage<AO> page = new MaPage<AO>();
page.number = pageNumber+1;
page.content = query.offset(pageNumber*pageSize).limit(pageSize).list(ao);
page.totalResult = query.count();
My Page class:
public class MaPage<T> {
public List<T> content;
public int number;
public Long totalResult;
public Long totalPages;
...
}
It works but I got this warning
nov. 21, 2014 6:48:54 AM
org.hibernate.hql.internal.ast.QueryTranslatorImpl list WARN:
HHH000104: firstResult/maxResults specified with collection fetch;
applying in memory!
Returning a Page:
JPAQuery query =
...
.orderBy(getOrderSpecifiers(pageable, MyEntity.class))
.limit(pageable.getPageSize())
.offset(pageable.getOffset());
long total = query.fetchCount();
List<MyEntity> content = query.fetch();
return new PageImpl<>(content, pageable, total);
And I created this function to get OrderSpecifier:
private OrderSpecifier[] getOrderSpecifiers(#NotNull Pageable pageable, #NotNull Class klass) {
// orderVariable must match the variable of FROM
String className = klass.getSimpleName();
final String orderVariable = String.valueOf(Character.toLowerCase(className.charAt(0))).concat(className.substring(1));
return pageable.getSort().stream()
.map(order -> new OrderSpecifier(
Order.valueOf(order.getDirection().toString()),
new PathBuilder(klass, orderVariable).get(order.getProperty()))
)
.toArray(OrderSpecifier[]::new);
}
If you have a working, complex query in querydsl and you want to use springdata pagination, you have to:
make your querydsl/repository method return Page<T>
Page<YourEntity> yourSelect(Pageable aPageable)
use querydsl offset and limit to page your result set
List<YourEntity> theResultList = jpaQueryFactory
.select(<whatever complext jpaquery you like>)
.offset(aPageable.getOffset())
.limit(aPageable.getPageSize())
.fetch();
provide a LongSuplier counting all available results with respect to your query and use PageableExecutionUtils to return the result as Page
final long theCount = jpaQueryFactory
.selectFrom(<your select to count all results>)
.fetchCount();
return PageableExecutionUtils.getPage(theResultList, aPageable, () -> theCount);