My problem is the following,
There are two entity classes, let's call them Entity1 and Entity2 with One-to-Many relationship in between, i.e. one Entity1 contains multiple Entity2s, and Entity2 may have only one Entity1:
#Entity
#Table(name = "entity1")
public class Entity1 {
int x;
int y;
...
#LazyCollection(LazyCollectionOption.TRUE)
#OneToMany(mappedBy = "e1", cascade = CascadeType.ALL, orphanRemoval = true)
private Set<Entity2> entity2s = new HashSet<>();
}
#Entity
#Table(name = "entity2")
public class Entity2 {
int a;
int b;
...
#ManyToOne
#JoinColumn(name = "entity1_id")
#JsonBackReference
private Entity1 e1;
}
Now I would like to issue a query for retrieving Entity2s with WHERE conditions for both Entity2 and its corresponding Entity1:
#Query("SELECT " +
" e2 " +
"FROM " +
" Entity2 e2 " +
"WHERE " +
" e2.a = '<val1>' AND e2.b = '<val2>' AND e2.e1.x = '<val3>' AND e2.e1.y ='<val4>'")
List<Entity2> findMyEntity2s(
#Param...,
#Param...,
);
So the problem with this approach is that, it indeed gets desired Entity2s by cross joining entity1 and entity2 tables with specified WHERE conditions BUT it fetches e1s for each of those Entity2s in the result with a separate query.
So for example if the result of join is 5 Entity2s, there will be 5 additional queries to entity1 table.
I tried to set #ManyToOne in Entity2 as #ManyToOne(fetch = FetchType.LAZY) but it didn't help. I guess that's expected because LAZY would simply postpone the retrieval of e1s but wouldn't eliminate it completely.
Next, I read about #EntityGraph, and added it to Entity2:
#Entity
#Table(name = "entity2")
#NamedEntityGraph(name = "graph.entity2.entity1",
attributeNodes = { #NamedAttributeNode("e1") })
public class Entity2 {
int a;
int b;
...
#ManyToOne
#JoinColumn(name = "entity1_id")
#JsonBackReference
private Entity1 e1;
}
and in the repository, I added it as:
#EntityGraph(value = "graph.entity2.entity1")
#Query("SELECT " +
" e2 " +
"FROM " +
" Entity2 e2 " +
"WHERE " +
" e2.a = '<val1>' AND e2.b = '<val2>' AND e2.e1.x = '<val3>' AND e2.e1.y ='<val4>'")
List<Entity2> findMyEntity2s(
#Param...,
#Param...,
);
In this case, the separate SQL queries disappear, EntityGraph does left join and its result contains columns from both entity1 and entity2, BUT because the conditions for e2.e1 are still in WHERE clause, it adds ONE MORE unnecessary cross join with entity1 table (e2.e1 conditions are checked in that cross join).
I couldn't find a way to get rid of that extra cross join, so now I'm using the following query:
#EntityGraph(value = "graph.entity2.entity1")
#Query("SELECT " +
" e2 " +
"FROM " +
" Entity2 e2 " +
"WHERE " +
" e2.a = '<val1>' AND e2.b = '<val2>'")
List<Entity2> findMyEntity2s(
#Param...,
#Param...,
);
So basically I get Entity2s and in the application I filter out based on conditions of Entity1 (e2.e1.x = '<val3>' AND e2.e1.y ='<val4>').
Is there a way to make it work with a single join only, for both entity's conditions, not only Entity2 conditions? The way I'm doing it now, does not seem correct and efficient to me, and I feel there's a way to do that using repository method only, without involving the app. Would appreciate any help on this
UPD. Read about nativeQuery option (nativeQuery = true) for #Query annotation, which allows specifying a raw query and thus bypassing entity-based query, but the query still fetches many-to-one e1 field, using entity1_id (entity graph was disabled). I tried to enable entity graph but it dropped exception stating that entity graph cannot be used with native query, which is expected
This is the classic n + 1 query problem.
You can read the detail here: https://vladmihalcea.com/n-plus-1-query-problem/
In your query, append:
LEFT JOIN FETCH e2.e1 e2e1
This will fetch e1 with e2 in the first and single query.
Don't forget; always use FetchType.LAZY and fetch your entities with JOIN FETCH. Otherwise, you will get into a big mass while the scope of the project enlarges.
In addition, why do you use Jaxson annotations in your Entity classes? Use entities for only DAO access and map them to another DTOs to use elsewhere.
I have to extract data for statistic purpose. I've created a native query and used #SqlResultSetMapping to map the resultset to an object.
Hibernate needs to declare this class (Elaboration) as #Entity BUT IS NOT A TABLE, and I don't want a table because I have only to extract data on the fly when needed.
The code works fine but the gitlab pipeline fails during validation with
schemaManagementException: Schema-validation: missing table [elaboration].
Here my code so far:
SqlResultSetMapping(name="ValueMapping",
classes={
#ConstructorResult(
targetClass=Elaboration.class,
columns={
#ColumnResult(name="areadesc", type=String.class),
#ColumnResult(name="subsectordesc", type=String.class),
#ColumnResult(name="eurovalue", type=BigDecimal.class),
#ColumnResult(name="eurotch", type=BigDecimal.class),
}
)
})
#Entity
public class Elaboration{
#Id
private Long id;
private String areadesc;
private String subsectordesc;
private Integer dossiercount;
private BigDecimal eurovalue;
private BigDecimal eurotch;
....
and the custom query:
String statisticValueQuery = "select a.mdescr as areadesc, s.mdescr as subsectordesc, sum(euro_value) as eurovalue,
sum(euro_value_tch) as eurotch " +
"from dossier d " +
"join dossier_document dd on d.id = dd.dossier_id " +
"join dossier_country dc on d.id = dc.dossier_id " +
"join country c on dc.country_id = c.id " +
"join area a on c.area_id = a.id " +
"join dossier_subsector ds on d.id = ds.dossier_id " +
"join subsector s on ds.subsector_id = s.id " +
"where dd.document_id = :document " +
"and d.submission_date >= :startdate and d.submission_date <= :enddate " +
"group by s.id, a.id;";
public List<Elaboration> getValueElaboration(ElaborationRequestDTO elaborationRequestDTO){
Query resultMapping = em.createNativeQuery(statisticValueQuery, "ValueMapping");
resultMapping.setParameter("startdate", elaborationRequestDTO.getElaborateFromEquals());
resultMapping.setParameter("enddate", elaborationRequestDTO.getElaborateToEquals());
resultMapping.setParameter("document", elaborationRequestDTO.getDocumentIdEquals());
return resultMapping.getResultList();
Is there a way to pass the validation test?
Thanks
This is wrong statement.
Hibernate needs to declare this class (Elaboration) as #Entity
You should just put your #SqlResultSetMapping declaration above some #Entity but it can be some other entity not related to the Elaboration.
#SqlResultSetMapping(name="ValueMapping",
classes={
#ConstructorResult(
targetClass=Elaboration.class,
columns={
#ColumnResult(name="areadesc", type=String.class),
#ColumnResult(name="subsectordesc", type=String.class),
#ColumnResult(name="eurovalue", type=BigDecimal.class),
#ColumnResult(name="eurotch", type=BigDecimal.class),
}
)
})
#Entity
public class SomeEntity {
}
And if Elaboration is not an entity you should not annotate it as such.
I am a JPA newbie and wanted to have a JPA native query for a single table (below) which I would like to fetch in my #Entity based class called TestRequest. It has a column 'RequestTime' that is fetched with DAYNAME() and then with DATEDIFF() functions.
SELECT TestRequest.Id AS Id
, TestRequest.RequestTime AS RequestTime
, DAYNAME(TestRequest.RequestTime) AS RequestDay
, TestRequest.StatusMessage AS StatusMessage
, DATEDIFF(CURDATE(), TestRequest.RequestTime) AS HowLongAgo
FROM TestRequest
LEFT JOIN TestRun
ON TestRequest.TestRunId = TestRun.Id
WHERE Requestor = '[NAME]'
ORDER BY Id DESC
Is there any way in which the column (fetched second time as HowLongAgo) be set into a property which is not mapped to a table column within the TestRequest class? Are there any field level annotations for this?
You need to use Interface-based projections:
You will need to create an interface that define the getters for each field in your projection like:
public interface RequestJoinRunProjection {
int getId();
LocalDate getRequestTime();
String getMessage();
String getRequestDay();
Long getHowLongAgo();
}
Then you define a method on your Repository that has the native query you want to run:
public interface TestRequestRepository extends CrudRepository<TestRequest, Long> {
// Any other custom method for TestRequest entity
#Query(value = "SELECT trq.Id AS id " +
" , trq.RequestTime AS requestTime " +
" , DAYNAME(trq.RequestTime) AS requestDay " +
" , trq.StatusMessage AS statusMessage " +
" , DATEDIFF(YEAR, CURDATE(), trq.RequestTime) AS howLongAgo " +
"FROM TestRequest trq " +
" LEFT JOIN TestRun tr " +
" ON trq.TestRunId = tr.Id " +
"WHERE Requestor = ?1 ORDER BY Id DESC"), nativeQuery = true)
List<RequestJoinRunProjection> findTestSumary(String name);
}
Notice query must be native since you are using database functions, also the column names must match the setters of your projection interface(following bean rules), so use AS in order to change the names in your query.
I strongly suggest you test your query on h2 before injecting into #Query annotation. DATEDIFF requires 3 parameters.
In a web project, using latest spring-data (1.10.2) with a MySQL 5.6 database, I'm trying to use a native query with pagination but I'm experiencing an org.springframework.data.jpa.repository.query.InvalidJpaQueryMethodException at startup.
UPDATE: 20180306 This issue is now fixed in Spring 2.0.4 For those still interested or stuck with older versions check the related answers and comments for workarounds.
According to Example 50 at Using #Query from spring-data documentation this is possible specifying the query itself and a countQuery, like this:
public interface UserRepository extends JpaRepository<User, Long> {
#Query(value = "SELECT * FROM USERS WHERE LASTNAME = ?1",
countQuery = "SELECT count(*) FROM USERS WHERE LASTNAME = ?1",
nativeQuery = true)
Page<User> findByLastname(String lastname, Pageable pageable);
}
Out of curiosity, In NativeJpaQuery class I can see that it contains the following code to check if it's a valid jpa query:
public NativeJpaQuery(JpaQueryMethod method, EntityManager em, String queryString, EvaluationContextProvider evaluationContextProvider, SpelExpressionParser parser) {
super(method, em, queryString, evaluationContextProvider, parser);
JpaParameters parameters = method.getParameters();
boolean hasPagingOrSortingParameter = parameters.hasPageableParameter() || parameters.hasSortParameter();
boolean containsPageableOrSortInQueryExpression = queryString.contains("#pageable") || queryString.contains("#sort");
if(hasPagingOrSortingParameter && !containsPageableOrSortInQueryExpression) {
throw new InvalidJpaQueryMethodException("Cannot use native queries with dynamic sorting and/or pagination in method " + method);
}
}
My query contains a Pageable parameter, so hasPagingOrSortingParameter is true, but it's also looking for a #pageable or #sort sequence inside the queryString, which I do not provide.
I've tried adding #pageable (it's a comment) at the end of my query, which makes validation to pass but then, it fails at execution saying that the query expects one additional parameter: 3 instead of 2.
Funny thing is that, if I manually change containsPageableOrSortInQueryExpression from false to true while running, the query works fine so I don't know why it's checking for that string to be at my queryString and I don't know how to provide it.
Any help would be much appreciated.
Update 01/30/2018
It seems that developers at spring-data project are working on a fix for this issue with a PR by Jens Schauder
My apologies in advance, this is pretty much summing up the original question and the comment from Janar, however...
I run into the same problem: I found the Example 50 of Spring Data as the solution for my need of having a native query with pagination but Spring was complaining on startup that I could not use pagination with native queries.
I just wanted to report that I managed to run successfully the native query I needed, using pagination, with the following code:
#Query(value="SELECT a.* "
+ "FROM author a left outer join mappable_natural_person p on a.id = p.provenance_id "
+ "WHERE p.update_time is null OR (p.provenance_name='biblio_db' and a.update_time>p.update_time)"
+ "ORDER BY a.id \n#pageable\n",
/*countQuery="SELECT count(a.*) "
+ "FROM author a left outer join mappable_natural_person p on a.id = p.provenance_id "
+ "WHERE p.update_time is null OR (p.provenance_name='biblio_db' and a.update_time>p.update_time) \n#pageable\n",*/
nativeQuery=true)
public List<Author> findAuthorsUpdatedAndNew(Pageable pageable);
The countQuery (that is commented out in the code block) is needed to use Page<Author>
as the return type of the query, the newlines around the "#pageable" comment are needed to avoid the runtime error on the number of expected parameters (workaround of the workaround). I hope this bug will be fixed soon...
This is a hack for program using Spring Data JPA before Version 2.0.4.
Code has worked with PostgreSQL and MySQL :
public interface UserRepository extends JpaRepository<User, Long> {
#Query(value = "SELECT * FROM USERS WHERE LASTNAME = ?1 ORDER BY ?#{#pageable}",
countQuery = "SELECT count(*) FROM USERS WHERE LASTNAME = ?1",
nativeQuery = true)
Page<User> findByLastname(String lastname, Pageable pageable);
}
ORDER BY ?#{#pageable} is for Pageable.
countQuery is for Page<User>.
Just for the record, using H2 as testing database, and MySQL at runtime, this approach works (example is newest object in group):
#Query(value = "SELECT t.* FROM t LEFT JOIN t AS t_newer " +
"ON t.object_id = t_newer.object_id AND t.id < t_newer.id AND o_newer.user_id IN (:user_ids) " +
"WHERE t_newer.id IS NULL AND t.user_id IN (:user_ids) " +
"ORDER BY t.id DESC \n-- #pageable\n",
countQuery = "SELECT COUNT(1) FROM t WHERE t.user_id IN (:user_ids) GROUP BY t.object_id, t.user_id",
nativeQuery = true)
Page<T> findByUserIdInGroupByObjectId(#Param("user_ids") Set<Integer> userIds, Pageable pageable);
Spring Data JPA 1.10.5, H2 1.4.194, MySQL Community Server 5.7.11-log (innodb_version 5.7.11).
I am adding this answer just as a placeholder for those users who are using more recent versions of Spring Boot. On Spring Boot 2.4.3, I observed that none of the workaround were necessary, and the following code worked straight out of the box for me:
public interface UserRepository extends JpaRepository<User, Long> {
#Query(value="SELECT * FROM USERS WHERE LASTNAME = ?1", nativeQuery=true)
Page<User> findByLastname(String lastname, Pageable pageable);
}
The countQuery definition was not necessary, and a call to Page#getTotalElements() in fact already was returning the correct count, as returned by JPA's own internal count query.
The above code is extremely powerful, offering pagination made via a native query, yet which return results into actual Java entities (rather than the ugly and bulky List<Object[]>, which sometimes is necessary).
Try this:
public interface UserRepository extends JpaRepository<User, Long> {
#Query(value = "SELECT * FROM USERS WHERE LASTNAME = ?1 ORDER BY /*#pageable*/",
countQuery = "SELECT count(*) FROM USERS WHERE LASTNAME = ?1",
nativeQuery = true)
Page<User> findByLastname(String lastname, Pageable pageable);
}
("/* */" for Oracle notation)
I have exact same symptom like #Lasneyx. My workaround for Postgres native query
#Query(value = "select * from users where user_type in (:userTypes) and user_context='abc'--#pageable\n", nativeQuery = true)
List<User> getUsersByTypes(#Param("userTypes") List<String> userTypes, Pageable pageable);
I use oracle database and I did not get the result but an error with generated comma which d-man speak about above.
Then my solution was:
Pageable pageable = new PageRequest(current, rowCount);
As you can see without order by when create Pagable.
And the method in the DAO:
public interface UserRepository extends JpaRepository<User, Long> {
#Query(value = "SELECT * FROM USERS WHERE LASTNAME = ?1 /*#pageable*/ ORDER BY LASTNAME",
countQuery = "SELECT count(*) FROM USERS WHERE LASTNAME = ?1",
nativeQuery = true)
Page<User> findByLastname(String lastname, Pageable pageable);
}
I could successfully integrate Pagination in
spring-data-jpa-2.1.6
as follows.
#Query(
value = “SELECT * FROM Users”,
countQuery = “SELECT count(*) FROM Users”,
nativeQuery = true)
Page<User> findAllUsersWithPagination(Pageable pageable);
Both the following approaches work fine with MySQL for paginating native query. They doesn't work with H2 though. It will complain the sql syntax error.
ORDER BY ?#{#pageable}
ORDER BY a.id \n#pageable\n
Using "ORDER BY id DESC \n-- #pageable\n "
instead of "ORDER BY id \n#pageable\n" worked for me with MS SQL SERVER
This worked for me (I am using Postgres) in Groovy:
#RestResource(path="namespaceAndNameAndRawStateContainsMostRecentVersion", rel="namespaceAndNameAndRawStateContainsMostRecentVersion")
#Query(nativeQuery=true,
countQuery="""
SELECT COUNT(1)
FROM
(
SELECT
ROW_NUMBER() OVER (
PARTITION BY name, provider_id, state
ORDER BY version DESC) version_partition,
*
FROM mydb.mytable
WHERE
(name ILIKE ('%' || :name || '%') OR (:name = '')) AND
(namespace ILIKE ('%' || :namespace || '%') OR (:namespace = '')) AND
(state = :state OR (:state = ''))
) t
WHERE version_partition = 1
""",
value="""
SELECT id, version, state, name, internal_name, namespace, provider_id, config, create_date, update_date
FROM
(
SELECT
ROW_NUMBER() OVER (
PARTITION BY name, provider_id, state
ORDER BY version DESC) version_partition,
*
FROM mydb.mytable
WHERE
(name ILIKE ('%' || :name || '%') OR (:name = '')) AND
(namespace ILIKE ('%' || :namespace || '%') OR (:namespace = '')) AND
(state = :state OR (:state = ''))
) t
WHERE version_partition = 1
/*#{#pageable}*/
""")
public Page<Entity> findByNamespaceContainsAndNameContainsAndRawStateContainsMostRecentVersion(#Param("namespace")String namespace, #Param("name")String name, #Param("state")String state, Pageable pageable)
The key here was to use: /*#{#pageable}*/
It allows me to do sorting and pagination.
You can test it by using something like this: http://localhost:8080/api/v1/entities/search/namespaceAndNameAndRawStateContainsMostRecentVersion?namespace=&name=&state=published&page=0&size=3&sort=name,desc
Watch out for this issue: Spring Pageable does not translate #Column name
Create your custom repository:
public interface ProductsCustomRepository extends JpaRepository<ProductResultEntity, Long> {
#Query(
value = "select tableA.id, tableB.bank_name from tableA join tableB on tableA.id = tableB.a_id where tableA.id = :id
and (:fieldX is null or tableA.fieldX LIKE :fieldX)",
countQuery = "select count(*) from tableA join tableB on tableA.id = tableB.a_id where tableA.id = :id
and (:fieldX is null or tableA.fieldX LIKE :fieldX)",
nativeQuery = true
)
Page<ProductResultEntity> search(#Param("id") Long aId,
#Param("fieldX") String keyword, Pageable pageable
);
}
Create View as query of:
create view zzz as select * from tableA join tableB on tableA.id = tableB.a_id
Generate ProductResultEntity from that zzz view (and delete view zzz when have done)
Test call function and enjoy it:
productsRepository.search("123", "%BANK%", PageRequest.of(0, 5, Sort.by(Sort.Direction.ASC, "id")));
Entity:
#Entity
public class ProductResultEntity {
private Long id;
private String bank;
#Id
#Column(name = "id", nullable = false)
public long getId() {
return id;
}
public void setId(long id) {
this.id = id;
}
#Column(name = "bank_name", nullable = false)
public String getBank() {
return bank;
}
public void setBank(String bank) {
this.bank = bank;
}
}
It does work as below:
public interface UserRepository extends JpaRepository<User, Long> {
#Query(value = "select * from (select (#rowid\\:=#rowid+1) as RN, u.* from USERS u, (SELECT #rowid\\:=0) as init where LASTNAME = ?1) as total"+
"where RN between ?#{#pageable.offset-1} and ?#{#pageable.offset + #pageable.pageSize}",
countQuery = "SELECT count(*) FROM USERS WHERE LASTNAME = ?1",
nativeQuery = true)
Page<User> findByLastname(String lastname, Pageable pageable);
}
For me below worked in MS SQL
#Query(value="SELECT * FROM ABC r where r.type in :type ORDER BY RAND() \n-- #pageable\n ",nativeQuery = true)
List<ABC> findByBinUseFAndRgtnType(#Param("type") List<Byte>type,Pageable pageable);
I'm using the code below. working
#Query(value = "select * from user usr" +
"left join apl apl on usr.user_id = apl.id" +
"left join lang on lang.role_id = usr.role_id" +
"where apl.scr_name like %:scrname% and apl.uname like %:uname and usr.role_id in :roleIds ORDER BY ?#{#pageable}",
countQuery = "select count(*) from user usr" +
"left join apl apl on usr.user_id = apl.id" +
"left join lang on lang.role_id = usr.role_id" +
"where apl.scr_name like %:scrname% and apl.uname like %:uname and usr.role_id in :roleIds",
nativeQuery = true)
Page<AplUserEntity> searchUser(#Param("scrname") String scrname,#Param("uname") String uname,#Param("roleIds") List<Long> roleIds,Pageable pageable);
Removing \n#pageable\n from both query and count query worked for me.
Springboot version : 2.1.5.RELEASE
DB : Mysql
You can use below code for h2 and MySQl
#Query(value = "SELECT req.CREATED_AT createdAt, req.CREATED_BY createdBy,req.APP_ID appId,req.NOTE_ID noteId,req.MODEL model FROM SUMBITED_REQUESTS req inner join NOTE note where req.NOTE_ID=note.ID and note.CREATED_BY= :userId "
,
countQuery = "SELECT count(*) FROM SUMBITED_REQUESTS req inner join NOTE note WHERE req.NOTE_ID=note.ID and note.CREATED_BY=:userId",
nativeQuery = true)
Page<UserRequestsDataMapper> getAllRequestForCreator(#Param("userId") String userId,Pageable pageable);
You can achieve it by using following code,
#Query(value = "SELECT * FROM users u WHERE ORDER BY ?#{#pageable}", nativeQuery = true)
List<User> getUsers(String name, Pageable pageable);
Simply use ORDER BY ?#{#pageable} and pass page request to your method.
Enjoy!
#Query(value = "select " +
//"row_number() over (order by ba.order_num asc) as id, " +
"row_number() over () as id, " +
"count(ba.order_num),sum(ba.order_qty) as sumqty, " +
"ba.order_num, " +
"md.dpp_code,md.dpp_name, " +
"from biz_arrangement ba " +
"left join mst_dpp md on ba.dpp_code = md.dpp_code " +
"where 1 = 1 " +
"AND (:#{#flilter.customerCodeListCheck}='' OR ba.customer_code IN (:#{#flilter.customerCodeList})) " +
"AND (:#{#flilter.customerNameListCheck}='' OR ba.customer_name IN (:#{#flilter.customerNameList})) " +
"group by " +
"ba.order_num, " +
"md.dpp_code,md.dpp_name ",
countQuery = "select " +
"count ( " +
"distinct ( " +
"ba.order_num, " +
"md.dpp_code,md.dpp_name) " +
")" +
"from biz_arrangement ba " +
"left join mst_dpp md on ba.dpp_code = md.dpp_code " +
"where 1 = 1 " +
"AND (:#{#flilter.customerCodeListCheck}='' OR ba.customer_code IN (:#{#flilter.customerCodeList})) " +
"AND (:#{#flilter.customerNameListCheck}='' OR ba.customer_name IN (:#{#flilter.customerNameList})) ",
nativeQuery = true)
Page<Map<String, Object>> nativeQueryDynamicPageAndSort(#Param("flilter") Flilter flilter, Pageable pageable);
no need to add ?#{#pageable},
the problem I got is when I use
row_number() over (order by ba.order_num asc) as id,
the input sort won't work
when I change to
row_number() over () as id,
the dynamic input sort and pagination are both okay!
This is a group by query with a row id.
Replacing /#pageable/ with ?#{#pageable} allow to do pagination.
Adding PageableDefault allow you to set size of page Elements.