Requirement
I have a simple OneToMany Relationship between two entities. I want to use JPA property expression to find Parent Entity with a condition which matches all children entities.
Parent
#Entity
public class PcSigningStatus {
#OneToMany(cascade = CascadeType.ALL, mappedBy = "signingStatus")
private List<PcSigningProcessEvent> signingProcessEvents = new ArrayList<>();
Child
#Entity
public class PcSigningProcessEvent {
#Enumerated(EnumType.STRING)
private ProcessEventType phase;
#ManyToOne
#JoinColumn(name = "SIGNING_STATUS_ID")
private PcSigningStatus signingStatus;
}
Parent Repository
public interface SigningStatusRepo extends CrudRepository<PcSigningStatus, Long> {
PcSigningStatus[] findBySigningProcessEvents_PhaseNot(ProcessEventType phase);
}
Issue
Suppose I only have 1 parent entity in database with 2 child entities containing different value of phase. The above query returns the same parent multiple times depending on the number of children it does not match the value of phase in. I need help in finding a way that Parent entity is returned only if the phase is not present in all the children entities
Maybe you should use #Query for this?
Hard to understand your question :) But you can try add distinct flag using Distinct mentioned in https://docs.spring.io/spring-data/jpa/docs/current/reference/html/#repositories.query-methods.query-creation
Edited: or you can use Query something like
SELECT * FROM PcSigningProcessEvent WHERE PcSigningProcessEvent.id NOT IN (SELECT SIGNING_STATUS_ID FROM PcSigningProcessEvent WHERE phase = ?)
So I found no way of doing this with property expression, and had to use Native Query instead.
Soltution
#Query(value = "SELECT ss.* FROM PARENT_TABLE ss\n" +
"left join CHILD_TABLE spe\n" +
"ON ss.ID= spe.SIGNING_STATUS_ID\n" +
"and spe.phase IN ('Value1', 'Value2')\n" +
"where spe.ID is null", nativeQuery = true)
PARENT_ENTITY[] customQuery();
This will return all parent entities whose children's property phase don't have any one of the value 'Value1' or 'Value2' or both.
Related
My problem is the following,
There are two entity classes, let's call them Entity1 and Entity2 with One-to-Many relationship in between, i.e. one Entity1 contains multiple Entity2s, and Entity2 may have only one Entity1:
#Entity
#Table(name = "entity1")
public class Entity1 {
int x;
int y;
...
#LazyCollection(LazyCollectionOption.TRUE)
#OneToMany(mappedBy = "e1", cascade = CascadeType.ALL, orphanRemoval = true)
private Set<Entity2> entity2s = new HashSet<>();
}
#Entity
#Table(name = "entity2")
public class Entity2 {
int a;
int b;
...
#ManyToOne
#JoinColumn(name = "entity1_id")
#JsonBackReference
private Entity1 e1;
}
Now I would like to issue a query for retrieving Entity2s with WHERE conditions for both Entity2 and its corresponding Entity1:
#Query("SELECT " +
" e2 " +
"FROM " +
" Entity2 e2 " +
"WHERE " +
" e2.a = '<val1>' AND e2.b = '<val2>' AND e2.e1.x = '<val3>' AND e2.e1.y ='<val4>'")
List<Entity2> findMyEntity2s(
#Param...,
#Param...,
);
So the problem with this approach is that, it indeed gets desired Entity2s by cross joining entity1 and entity2 tables with specified WHERE conditions BUT it fetches e1s for each of those Entity2s in the result with a separate query.
So for example if the result of join is 5 Entity2s, there will be 5 additional queries to entity1 table.
I tried to set #ManyToOne in Entity2 as #ManyToOne(fetch = FetchType.LAZY) but it didn't help. I guess that's expected because LAZY would simply postpone the retrieval of e1s but wouldn't eliminate it completely.
Next, I read about #EntityGraph, and added it to Entity2:
#Entity
#Table(name = "entity2")
#NamedEntityGraph(name = "graph.entity2.entity1",
attributeNodes = { #NamedAttributeNode("e1") })
public class Entity2 {
int a;
int b;
...
#ManyToOne
#JoinColumn(name = "entity1_id")
#JsonBackReference
private Entity1 e1;
}
and in the repository, I added it as:
#EntityGraph(value = "graph.entity2.entity1")
#Query("SELECT " +
" e2 " +
"FROM " +
" Entity2 e2 " +
"WHERE " +
" e2.a = '<val1>' AND e2.b = '<val2>' AND e2.e1.x = '<val3>' AND e2.e1.y ='<val4>'")
List<Entity2> findMyEntity2s(
#Param...,
#Param...,
);
In this case, the separate SQL queries disappear, EntityGraph does left join and its result contains columns from both entity1 and entity2, BUT because the conditions for e2.e1 are still in WHERE clause, it adds ONE MORE unnecessary cross join with entity1 table (e2.e1 conditions are checked in that cross join).
I couldn't find a way to get rid of that extra cross join, so now I'm using the following query:
#EntityGraph(value = "graph.entity2.entity1")
#Query("SELECT " +
" e2 " +
"FROM " +
" Entity2 e2 " +
"WHERE " +
" e2.a = '<val1>' AND e2.b = '<val2>'")
List<Entity2> findMyEntity2s(
#Param...,
#Param...,
);
So basically I get Entity2s and in the application I filter out based on conditions of Entity1 (e2.e1.x = '<val3>' AND e2.e1.y ='<val4>').
Is there a way to make it work with a single join only, for both entity's conditions, not only Entity2 conditions? The way I'm doing it now, does not seem correct and efficient to me, and I feel there's a way to do that using repository method only, without involving the app. Would appreciate any help on this
UPD. Read about nativeQuery option (nativeQuery = true) for #Query annotation, which allows specifying a raw query and thus bypassing entity-based query, but the query still fetches many-to-one e1 field, using entity1_id (entity graph was disabled). I tried to enable entity graph but it dropped exception stating that entity graph cannot be used with native query, which is expected
This is the classic n + 1 query problem.
You can read the detail here: https://vladmihalcea.com/n-plus-1-query-problem/
In your query, append:
LEFT JOIN FETCH e2.e1 e2e1
This will fetch e1 with e2 in the first and single query.
Don't forget; always use FetchType.LAZY and fetch your entities with JOIN FETCH. Otherwise, you will get into a big mass while the scope of the project enlarges.
In addition, why do you use Jaxson annotations in your Entity classes? Use entities for only DAO access and map them to another DTOs to use elsewhere.
I need to retrieve a list of Category from the DB on the basis of value of column called owner. Here is my Category -
#Entity
#Table(name = "categories")
class Category(#Column(name = "category_id", nullable = false)
#Id #GeneratedValue(strategyGenerationType.AUTO)
var id: Long = 0,
#Column(name = "category_owner", nullable = false)
#field:NotNull(message = "Please assign an owner")
var owner: Long?,
#Column(name = "category_name", nullable = false)
#field:NotEmpty(message = "Please assign a name")
var name: String?)
Here is my interface which defines the function findByOwner -
interface CategoryRepository: JpaRepository<Category, Long> {
fun findByOwner(categoryOwner: Long): List<Category>
}
However, when I call the method, I get no response. I have made sure that the DB has correct data and I'm providing the correct owner Id. Have even invalidated the cache etc. What could be going wrong?
EDIT:
After spring.jpa.show-sql=true -
findAll()
Hibernate: select category0_.category_id as category1_0_, category0_.category_name as category2_0_, category0_.category_owner as category3_0_ from categories category0_
findByOwner()
Hibernate: select category0_.category_id as category1_0_, category0_.category_name as category2_0_, category0_.category_owner as category3_0_ from categories category0_ where category0_.category_owner=?
EDIT 2:
Turns out that my implementation was fine all along. The bug was in my service.
Create your named method according with the name of the column.
fun findByCategoryOwner(categoryOwner: Long): List<Category>
Or use #Query
#Query("SELECT * FROM categories WHERE category_owner = ?1", nativeQuery = true)
fun findByOwner(cateogryOwner: Long): List<Category
Can you put a breakpoint in org.springframework.data.jpa.repository.query.JpaQueryExecution class and when you execute findByOwner, it will come here.
When it reaches this breakpoint, select the query.createQuery(accessor).getResultList() and evaluate to see what value is returned by hibernate for spring-data-jpa to use
This post should help you. It appears to be happeing because of the parameter name mismatch.
Use camelCase to name your variables in Entity class then jpa will auto recognise the column name
findByCategoryOwner(String categoryOwner)
If you still wish to have underscore in your column names then try this
findByCategory_Owner(String categoryOwner)
I haven't tried the second option though
At least in java you need to provide the id in the method name:
**fun findByOwner_Id(categoryOwner: Long): List<Category>**
So change it from findByOwner -> findByOwnerId.
I've an entity defined like below. If I use save() Hibernate does not create a new index for newly created entity. Updating/modifying an existing entity works well and as expected.
I'm using kotling with spring boot 2.
#Entity(name = "shipment")
#Indexed
data class Shipment(
#Id #GeneratedValue(strategy = GenerationType.IDENTITY) val id: Long = -1,
#JoinColumn(name = "user") #ManyToOne() var user: User?,
#IndexedEmbedded
#JoinColumn(name = "sender") #ManyToOne(cascade = [CascadeType.ALL]) val sender: Contact,
#IndexedEmbedded
#JoinColumn(name = "sender_information") #ManyToOne(cascade = [CascadeType.ALL]) val senderInformation: ShipmentInformation,
) {}
Save function, I'm using this same function to update my entity and index is updated if index exists.
#Transactional
fun save(user: User, shipment: Shipment): Shipment {
shipment.user = user;
return this.shipmentRepository.save(shipment)
}
application.properties
spring.jpa.properties.hibernate.search.default.directory_provider=filesystem
spring.jpa.properties.hibernate.search.default.indexBase=./lucene/
spring.jpa.open-in-view=false
If I restart the server, indexing manually works too.
#Transactional
override fun onApplicationEvent(event: ApplicationReadyEvent) {
val fullTextEntityManager: FullTextEntityManager = Search.getFullTextEntityManager(entityManager)
fullTextEntityManager.createIndexer().purgeAllOnStart(true)
fullTextEntityManager.createIndexer().optimizeAfterPurge(true)
fullTextEntityManager.createIndexer().batchSizeToLoadObjects(15)
fullTextEntityManager.createIndexer().cacheMode(CacheMode.IGNORE)
fullTextEntityManager.createIndexer().threadsToLoadObjects(2)
fullTextEntityManager.createIndexer().typesToIndexInParallel(2)
fullTextEntityManager.createIndexer().startAndWait()
return
}
I tried to force to use JPA transaction manager but It did not help me.
#Bean(name = arrayOf("transactionManager"))
#Primary
fun transactionManager(#Autowired entityManagerFactory: EntityManagerFactory): org.springframework.orm.jpa.JpaTransactionManager {
return JpaTransactionManager(entityManagerFactory)
}
Update
I think I found why I don't get the results of newly inserted entities.
My search query has a condition on "pid" field which is declared:
#Field(index = Index.YES, analyze = Analyze.NO, store = Store.NO)
#SortableField
#Column(name = "id", updatable = false, insertable = false)
#JsonIgnore
#NumericField val pid: Long,
and query:
query.must(queryBuilder.keyword().onField("customer.pid").matching(user.customer.id.toString()).createQuery())
pid is not stored and so newly inserted values are not visible. Can this be the cause?
BTW: How can I query/search by nested indexed document id? In my case it is customer.id which is DocumentId. I've tried to change the query like below but don't get any result, should I create a new field to query?
query.must(queryBuilder.keyword().onField("customer.id").matching(user.customer.id.toString()).createQuery())
Update 2
I found a solution and now getting the newly inserted datas too. There was an error with definition of "pid" field and I've defined my Fields as below and it works as expected.
#Fields(
Field(name = "pid", index = Index.YES, analyze = Analyze.YES, store = Store.NO)
)
#SortableField(forField = "pid")
#Id #GeneratedValue(strategy = GenerationType.IDENTITY) val id: Long?,
Can we search and sort by id in an easy way or is it the best practice? I know that we should use native JPA functions to get results by id but in my case I need to search by an embedded id to restrict search results. (depends on role of user) so therefore it is not an option for me.
And I don't understand why manual indexing works...
BTW: How can I query/search by nested indexed document id? In my case it is customer.id which is DocumentId. I've tried to change the query like below but don't get any result, should I create a new field to query?
Normally you don't need to create a separate field if all you want is to perform an exact match.
Can we search and sort by id in an easy way
Searching, yes, at least in Hibernate Search 5.
Sorting, no: you need a dedicated field.
or is it the best practice?
The best practice is to declare a field alongside your #DocumentId if you need anything more complex than an exact match on the ID.
I know that we should use native JPA functions to get results by id
I'm not sure I understand what you mean by "native JPA functions".
but in my case I need to search by an embedded id to restrict search results. (depends on role of user)
Yes, this should work. That is, it should work if the id is properly populated.
And I don't understand why manual indexing works...
Neither do I, but I suppose the explanation lies in the "error in the definition of "pid" field". Maybe the ID wasn't populated properly in some cases, leading to the entity being considered as deleted by Hibernate Search?
If you need me to give you a definitive answer, the best way to get it would be to create a reproducer. You can use this as a template: https://github.com/hibernate/hibernate-test-case-templates/tree/master/search
This looks odd:
#Id #GeneratedValue(strategy = GenerationType.IDENTITY) val id: Long = -1,
I'd expect a nullable long, initialized to null (or whatever is the Kotlin equivalent).
I'm not sure this is the problem, but I imagine it could be, as a non-null ID is generally only expected from an already persisted entity.
Other than that, I think you're on the right track: if mass indexing works but not automatic indexing, it may have something to do with your changes not being executed in database transactions.
I have a case statement in my Native query where I am attempting to override a field in my entity.
SELECT i.id, i.ONE_TO_ONE_ID, i.ANOTHER, CASE(WHEN condition THEN 'YES' WHEN another_condition THEN 'NO' ELSE 'MAYBE' END) as word ....
I am using this with JpaRepository as a native query, with pagination.
When I run the native query against my db directly, the result set looks as though I expect.
| id_value | MAPPED_ENTITY_ID_value | another value | word_value (YES) |
When I run the native query from my JpaRepository, everything works there, except word is always null. I cant' seem to figure out how to map the additional String word result to a field in my Entity.
Is there a way to get this to map? Or will I have to create an entire #SqlResultSetMapping() for all of my fields coupled with a native query? (hoping not)
UPDATE: 1
I was generalizing above. Here is my Query.
#Query(
name = "listPagedMapping",
value = "SELECT DISTINCT i.ID, i.INSTANCE_ID, i.REGION, i.CNAME_STACK_ID, i.INSTANCE_STATE, i.IP_ADDRESS, i.EC2_ROLE_NAME, i.INSTANCE_OWNER, i.IS_MASTER, i.EC2_MASTER_ID, i.CNAME, i.EC2_START_TIMESTAMP, i.PRIVATE_DNS, i.INSTANCE_NAME, i.AUTO_TERMINATE, i.AUTO_TERMINATE_DATE, i.TERMINATION_ZONE, i.ADMIN_GROUP_AD_LDAP_ID, i.USER_GROUP_AD_LDAP_ID, (CASE WHEN i.INSTANCE_OWNER=:username THEN 'OWNER' WHEN i.ADMIN_GROUP_AD_LDAP_ID IN (g.AD_LDAP_ID) THEN 'ADMIN' WHEN i.USER_GROUP_AD_LDAP_ID IN (g.AD_LDAP_ID) THEN 'USER' END) as PERMISSION FROM USER u, USER_ACCESS_GROUPS g, EC2_PROVISIONING i WHERE i.INSTANCE_OWNER=:username and i.INSTANCE_STATE in (:instanceStates) or u.username=:username and i.INSTANCE_STATE in (:instanceStates) and g.USER_ID=u.USER_ID and (i.ADMIN_GROUP_AD_LDAP_ID IN (g.AD_LDAP_ID) or i.USER_GROUP_AD_LDAP_ID IN (g.AD_LDAP_ID))",
countQuery = "SELECT count(*) FROM (SELECT DISTINCT i.* FROM USER u, USER_ACCESS_GROUPS g, EC2_PROVISIONING i WHERE i.INSTANCE_OWNER=:username and i.INSTANCE_STATE in (:instanceStates) or u.username=:username and i.INSTANCE_STATE in (:instanceStates) and g.USER_ID=u.USER_ID and (i.ADMIN_GROUP_AD_LDAP_ID IN (g.AD_LDAP_ID) or i.USER_GROUP_AD_LDAP_ID IN (g.AD_LDAP_ID))) as ug",
nativeQuery = true)
Page<Ec2Instance> findAllByPermissionUserAdminOrOwnerAndInstanceStateIn(
#Param("username")final String username,
#Param("instanceStates") final Set<String> instanceStates,
final Pageable pageable);
}
Obviously a bit more complex.
I can get it to map to the entity field with using a named query, but then I loose all the default mappings:
#JsonInclude(JsonInclude.Include.NON_NULL)
#SuppressWarnings("unused")
#Data
#AllArgsConstructor
#NoArgsConstructor
#EqualsAndHashCode(exclude={"masterNode", "workers", "associatedBuckets"})
#Entity
#Table(name = "EC2_PROVISIONING")
#SqlResultSetMapping(
name="listPagedMapping",
columns = {
#ColumnResult(name = "permission", type = String.class)
}
)
#NamedNativeQuery(
name = "listAccessibleInstances",
query = ACCESSIBLE_QUERY,
resultSetMapping = "listPagedMapping"
)
public class Ec2Instance {
....
private String permission;
#column(name = "INSTANCE_ID")
private String instanceId;
#ManyToOne
#JoinColumn(name = "EC2_MASTER_ID")
private Ec2Instance masterNode;
#Setter(AccessLevel.NONE)
#ManyToMany(fetch = FetchType.EAGER)
#JoinTable(name = "WORKER_EC2_NODES", joinColumns = { #JoinColumn(name = "EC2_MASTER_ID") }, inverseJoinColumns = {
#JoinColumn(name = "ID") })
private Set<Ec2Instance> workers = new HashSet<>();
... More fields ..
}
I guess, I am hoping there is a way to provide a single mapping on-top of the default mapping that is done by ORM. The above code results in only a pageable of Content PERMISSION, rather than the whole entity + permission.
UPDATE: 2
Ok, so I am getting closer... Seems by removing the #ColumnResult I do get the default mapping, plus the PERMISSION field mapped over! Looks like this:
#SqlResultSetMapping(
name="listPagedMapping"
)
The last issue is it does not accept my CountQuery, and causes my tests to fail whenever a Pagination Query results with multiple pages. Looks like Spring try's to come up with its own CountQuery, which is not correct.
UPDATE: 3
To finish this off, looks like I can provide the Count Query as described here: Spring Data - Why it's not possible to have paging with native query
I will give this a go and update back.
I never got this to work quite how I wanted. I am sure I could by mapping my entire entity, but, that would have been painstaking. I ended up solving this by using NamedNativeQueries, with mapping for the additional Column as a result of my Case statement. My entity class is now annotated like:
#JsonInclude(JsonInclude.Include.NON_NULL)
#SuppressWarnings("unused")
#Data
#AllArgsConstructor
#NoArgsConstructor
#EqualsAndHashCode(callSuper = false)
#Entity
#Table(name = "EC2_PROVISIONING")
#SqlResultSetMappings({
#SqlResultSetMapping(
name = "listPagedMapping",
entities = {
#EntityResult(
entityClass = Ec2Instance.class
)
},
columns = {#ColumnResult(name = "permission", type = String.class)}
),
#SqlResultSetMapping(name = "listPagedMapping.count", columns = #ColumnResult(name = "cnt"))
})
#NamedNativeQueries({
#NamedNativeQuery(
name = "Ec2Instance.listAccessibleInstances",
query = ACCESSIBLE_QUERY,
resultClass = Ec2Instance.class,
resultSetMapping = "listPagedMapping"
),
#NamedNativeQuery(
name = "Ec2Instance.listAccessibleInstances.count",
resultSetMapping = "listPagedMapping.count",
query = ACCESSIBLE_QUERY_COUNT
)
})
We also dont need the permission field in this entity anymore. I removed that.
Then in my Repository:
Page<Object[]> listAccessibleInstances(
#Param("username")final String username,
#Param("instanceStates") final Set<String> instanceStates,
final Pageable pageable);
Thats it! Now the result of my case statement is returned with each entity.
Object[0] = original, default mapped entity.
Object[1] = permission
Current project runs on Spring + Openjpa + Roo. I have an entity like this
public class Zoo{
....
#OneToMany(mappedBy="zoo", fetch = FetchType.LAZY, cascade = CascadeType.REMOVE)
private List<Elephant> elephants;
#OneToMany(mappedBy="zoo", fetch = FetchType.LAZY, cascade = CascadeType.REMOVE)
private List<Jaguar> jaguars;
#OneToMany(mappedBy="zoo", fetch = FetchType.LAZY, cascade = CascadeType.REMOVE)
private List<Tiger> tigers;
....
}
Then I have a simple UI page just trying to update the Zoo name, however from SQL trace log after the simple query
SELECT t0.id, t0.name
FROM Zoo t0
WHERE t0.id = ?
there are a query like this
SELECT * FROM Zoo, Tiger, TigerProduct, TigerFood, FoodSupplier, SupplierContacts...
and a hundreds queries like this:
SELECT * FROM TigerProduct where tiger.id =: id_1
.....
SELECT * FROM TigerProduct where tiger.id =: id_n
....
....
SELECT * FROM TigerFood where tiger.id =: id_1
....
SELECT * FROM TigerFood where tiger.id =: id_n
And same to Jaguar and Elephant as well. This makes this simple action really slow when there is large amount of data resides in the database.
The java code for the first query and the ones after is pretty simple:
public static Zoo findZoo(Long id) {
if (id == null) return null;
return entityManager().find(Zoo.class, id);
}
from above it looks like the default FetchType.Lazy on #OneToMany relation is not so lazy at all that JPA tries to pull all data on the chain.
So what's going on and how to clear this situation? I only prefer to have the first query and that's it
FetchType.Lazy is only a hint, and not a requirement, as the documentation says. So you cannot rely on this behavior, you can only hope that your JPA provider respects your hint. Also JPA does not forces a way how the JPQL queries or entitymanager calls are converted to SQL code, so it is somehow our duty to select a JPA provider+version that knows how to do things better (as we define what better means). This was probably a decision that should encourage the competition between JPA providers.