We have a spring boot application where we are trying to save information related to notifications received from third-party systems. The information is saved only when there exists a subscription for a specific kind of notification.
In short, we have three different entities present.
1) A subscription entity which is a combination of
a) a subscriber or User
b) a subject on which the subscription is taken
c) type of notification on which a subscriber took the subscription with respect to a subject.
2) A subscriber entity
3) a subject entity
While implementing a save subscription scenario in spring data jpa we are facing an issue related to the duplication of data.
The way the functionality is envisioned we get the information related to both subject, subscriber, and type as part of the notification subscription post message.
{
"notificationTypeCodes" : [""],
"subjectId" : "Person1",
"subscriberId" : "USER1"
}
we then save this information in three different tables 1) subscription table
2) subjectId 3) subscriberId
The subscription table contains the relationship columns containing combination of id wrt ( notificationTypeCodes,subjectId,subscriberId).
The data wrt these ids are placed in their respective tables.
we are populating all three tables in the same transaction using the cascading approach. In order to prevent duplicate values being entered in any of the tables, we tried doing a getsubscritopn before save and set up a condition statement stating that it subscription does not exist them only save the subscription
subscription object = fetchsubscription();
if(subscription is not present)
{
save subscritopm)
}
else{
response stating subscritpion exists
}
But in case of multithreading scenarios, our check is falling as multiple threads are entering the if block before the subscription is saved and duplicate entries are getting crated in all three tables.
we cannot create constraints on a table as we can have scenarios where
different users(subscribers) can subscribe on same subjects and if we have constraints on a subject table this valid scenario might be rolled back.
Is there a way that we can handle the duplicate scenario the DB level. where we can throw a unique constraint exception. if we are storing a duplicate subscription.
Note: we don't have any constraints on the table as of now.
2) we don't want to use synchronized block in the service class
It would be great if someone can provide some insite into the same.
Thanks
in advance
I have now tried to explain the same problem with books publisher and bookspublisher entity
**************** bookspublisher entity************
package com.hellokoding.jpa.model;
import javax.persistence.*;
#Entity
#Table(name = "book_publisher")
public class BookPublisher
{
#Id
private int id;
public BookPublisher() {
}
public Book getBook()
{
return book;
}
public void setBook( Book aBook )
{
book = aBook;
}
public int getId()
{
return id;
}
public void setId( int aId )
{
id = aId;
}
public Publisher getPublisher()
{
return publisher;
}
public void setPublisher( Publisher aPublisher )
{
publisher = aPublisher;
}
#OneToOne(cascade = {CascadeType.PERSIST, CascadeType.MERGE}, fetch = FetchType.EAGER)
#JoinColumn(name = "book_id")
private Book book;
#OneToOne(cascade = {CascadeType.PERSIST, CascadeType.MERGE}, fetch = FetchType.EAGER)
#JoinColumn(name = "publisher_id")
private Publisher publisher;
}
Service to persists bookspublisher data.
The issue is when I am persisting the BookPublisher entity in multithreading environment duplicate data is getting created in both books and publisher tables.
The use case states that only the very first time when a new book or publisher comes it should be persisted in rest of the calls no duplicate entries should be allowed in the tables.
#Service
public class BookPublisherService
{
#Autowired
private BookPublisherRepository bookpublisherRepository;
#Transactional
public void saveSubscription( BookPublisherRequest aRequest )
{
String bookName= aRequest.getBook();
String publisher= aRequest.getPublisher();
BookPublisher subscription = new BookPublisher();
subscription.setBook( new Book( bookName) );
subscription.setPublisher( new Publisher( publisher ) );
bookpublisherRepository.save( subscription );
}
}
Is it possible that if book and publisher values already exists then the same id's are provided to the publisherbook entity and by having unique constrain on that single table(publisherbook ) i can throw publishbook already exists message.
These two statements seem contradictory:
we cannot create constraints on a table as we can have scenarios where
different users(subscribers) can subscribe on same subjects and if we have constraints on a subject table this valid scenario might be rolled back.
and
Is there a way that we can handle the duplicate scenario the DB level. where we can throw a unique constraint exception. if we are storing a duplicate subscription.
It sounds like you need a unique constraint on your Subscription entity to prevent the duplicates. i.e.
#Table(name = "subscriptions", uniqueConstraints = #UniqueConstraint(name = "uq_sub", columnNames = {"notificationTypeCode", "subscriberId", "subjectId"}))
Related
I am exploring possible ideas when it comes to designing the one-to-one and one-to-many relationships while using Spring Data R2DBC.
As Spring Data R2DBC still do not support relationships natively there is still a need to handle those on our own (unlike Spring Data JDBC).
What I would imagine that when it comes to one-to-one mapping, the implementation could look like this:
#Table("account")
public class Account {
#Id
private Long id;
#Transient // one-to-one
private Address address;
}
#Table("address")
public class Address {
#Id
private Integer id;
}
while the database schema would be defined as follows:
--address
CREATE TABLE address
(
id SERIAL PRIMARY KEY
)
--account
CREATE TABLE account
(
id SERIAL PRIMARY KEY,
address_id INTEGER REFERENCES address(id)
)
As the Account object is my aggregate root what I would imagine is that I am supposed to load the Address object with it following the advice of Jens Schaduer:
An aggregate is a cluster of objects that form a unit, which should
always be consistent. Also, it should always get persisted (and
loaded) together.
source: Spring Data JDBC, References, and Aggregates
This leads me to thinking that in case of one-to-one relationships like this one I in fact should have my Account entity defined like this:
#Table("account")
public class Account {
#Id
private Long id;
#Transient // one-to-one
private Address address;
#Column("address_id")
private Integer addressId;
}
and later on to recreate the full Account aggregate entity with an Address I would write something like:
#Service
public class AccountServiceImpl implements AccountService {
private final AccountRepository accountRepository;
private final AddressRepository addressRepository;
public AccountServiceImpl(AccountRepository accountRepository,
AddressRepository addressRepository) {
this.accountRepository = accountRepository;
this.addressRepository = addressRepository;
}
#Override
public Mono<Account> loadAccount(Integer id) {
return accountRepository.getAccountById(id)
.flatMap(account ->
Mono.just(account)
.zipWith(addressRepository.getAddressByAccountId(account.getAddressId()))
.map(result -> {
result.getT1().setAddress(result.getT2());
return result.getT1();
})
);
}
}
If that is not the case, how else should I handle one-to-one relationships while using Spring Data R2DBC?
I think your approach is reasonable. There are just a couple of nitpicks:
Do you need the flatMap -> Mono.just ? Can't you just use map directly?
I wouldn't consider this a service, but a repository (it's just not implemented by Spring Data directly.
You might be able to that code in a after load callback.
I have User nodes and having contact relationships with each other. user1 has two contacts user2 and user3. Now I want to delete user2 from contacts. Following is a code snippet about contact relationships.
#Node
public class User{
...Id and different properties
#Relationship(type = "CONTACT")
public Set<User> contacts = new HashSet<>();
}
Now when I delete a relationship and save back the node it shows the following message:
WARN 38461 --- [nio-8080-exec-2] o.s.d.n.c.m.DefaultNeo4jIsNewStrategy : Instances of class com.talkkia.api.entity.User with an assigned id will always be treated as new without version property!
Code for deleting relationship is here:
#Transactional
#Override
public String deleteContact(String mobile1, String mobile2) {
Optional<User> user1 = userRepository.findOneByMobile(mobile1);
Optional<User> user2 = userRepository.findOneByMobile(mobile2);
if(user1.get().getContacts().contains(user2.get())){
user1.get().getContacts().remove(user2.get());
System.out.println(user1.get().getContacts());
userRepository.save(user1.get());
return user2.get().getName() + " has been deleted from your contact.";
}
return user2.get().getName() + " can't be deleted from contact.";
}
The warning message is exactly what leads into the situation you are faced with.
Because you are using an assigned, manual provided id, Spring Data Neo4j cannot differentiate if the object is new or not.
The solution is to provide a #Version field in the entity (like #Version Long version).
From this SDN can derive the information if the element comes from the database or was newly created.
As a consequence of the wrongly assumed new state of the entity, no relationships will be deleted because it is not necessary from the persistence logic.
#Query("MATCH(user1:User {mobile: $mobile1})-[c:CONTACT]-(user2:User {mobile:$mobile2}) DETACH DELETE c")
public String deleteContact(String mobile1, String mobile2);
Have found the above solution very easy and good in performance I think. #meistermeier point is also valid and considerable.
I have an entity like below. When I need to list comment size of company I'm calling totalComments() method. For this does hibernate go to the database and fetch entire comment data or just querying with count(*)? If hibernate fetch entire comment what is the efficient way for getting comment size?
#Entity
#Table(name = "companies")
public class Company extends ItemEntity {
#OneToMany(fetch = FetchType.LAZY)
#JoinTable(name="companies_comments",
joinColumns=#JoinColumn(name="company_id"),
inverseJoinColumns=#JoinColumn(name="comment_id"))
private Set<Comment> comments = new HashSet<>();
public void addComment(Comment comment) {
this.comments.add(comment);
}
public int totalComments() {
return this.comments.size();
}
}
You should drop the own method counter and create a specific (business) query to retrieve the size of the list, such as
public long getCommentsCount(Company c) {
String query = "SELECT COUNT(cm) FROM Company AS c JOIN c.comments AS cm WHERE c = :company";
return entityManager.createQuery(q, Long.class).setParameter("company", c).getSingleResult();
}
Some persistence provider may optimize performance when this kind of query is loaded as a #NamedQuery on entity, or when using CriteriaQuery API.
Depending on your database, you may need to change the return class to Number.class and convert to long.
If you want to tune even more your performance, use createNativeQuery method and write your own pure SQL, but keep in mind that changes on db schema requires to review theses queries.
I found the answer. If we don't adjust for getting collection size of entity hibernate loads every comment. We can solve this performance issue in two ways.
We can use #LazyCollection(LazyCollectionOption.EXTRA) like below. By LazyCollectionOption.EXTRA .size() and .contains() won't initialize the whole collection.
#OneToMany(fetch = FetchType.LAZY)
#LazyCollection(LazyCollectionOption.EXTRA)
#JoinTable(name="companies_comments",
joinColumns=#JoinColumn(name="company_id"),
inverseJoinColumns=#JoinColumn(name="comment_id"))
private Set<Comment> comments = new HashSet<>();
Or we can use #Formula annotation.
#Formula(SELECT COUNT(*) FROM companies_comments cc WHERE cc.company_id = id)
private int numberOfComments;
Edit after 8 months: For simplicity and performance perspective, we should create a JPA Query Method like below.
#Repository
public interface CommentRepository extends JpaRepository<Comment, Long> {
int countAllByCompany(Company company);
}
We should never use getComments().size() for this purpose, because this way all comments are loaded into memory and this may be cause performance issues.
It is also true when adding comments to the collection. We shouldn't use getComments().add(newComment). When we have OneToMany relation, all we have to do is set the company field of the comment like as newComment.setCompany(company), and perform the persist operation. Therefore, it is recommended to define OneToMany relationships bidirectional.
I am trying to populate my database, and during this, I create a associations between employees and teams. I'm trying to access the team members on a team via a hibernate OneToMany reference and doing lazy loading. This works great in my app, except when I call it right after the entities are created.
I'm fairly certain this is a hibernate transaction or caching issue, as all of the rows exist in the database, but I'm struggling to understand what the issue is. I have tried converting all of the saves in the populateTeamsAndEmployees function to saveAndFlush, but that did not help.
How can I make team.getTeamMembers() work every time it is called?
Note: This only happens when I call the seedData() function via RequestMapping, but it doesn't happen when I call seedData() with a #Scheduled annotation
Function with Problem
void seedData() {
populateTeamsAndEmployees()
otherBusinessLogic()
}
public void otherBusinessLogic() {
List<Team> teams = teamRepository.findAll();
teams.foreach(team -> {
/*
works great every time, returns all team members
*/
List<Employee> teamMembers = employeeRepository.findEmployeeByTeamId(team.getId());
/*
this returns null when ran immediately after populateTeamsAndEmployees(),
but returns the same as the line above all other times
*/
List<Employee> thisDoesntWork = team.getTeamMembers();
});
}
public void populateTeamsAndEmployees() {
List<EmployeeFromOtherSystem> employeesToConvert = otherSystemRepo.findAll();
employeesToConvert.foreach(otherSysEmployee -> {
employeeRepository.save(otherSysEmployee.toEmployee());
});
}
Entities:
class Team {
#OneToMany(mappedBy = "team", fetch = FetchType.LAZY)
private List<Employee> teamMembers;
}
class Employee {
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "TEAM_ID")
private Team team;
}
So the problem was that JPA was caching the entity right after I created it.
I was able to fix it by calling em.clear()
See below for how to access EntityManager with a JpaRepository:
https://dzone.com/articles/accessing-the-entitymanager-from-spring-data-jpa
And see below for why to use .clear()
EntityManager refresh
I am using Spring Data JPA in a Spring Boot Application, with MYSQL. There I am saving a list of entities with unique constraint over a field. Out of the list of entities, there is one entity that will throw DataIntegrityViolationException due to the unique constraint. I noticed that none of the entities get persisted in that case, even those that does not violate the unique constraint.
What should be the ideal approach in this case so that those entities which do not violate the unique get persisted ?
Of course I can iterate the list and save them one by one. In fact that is what SimpleJpaRepository is doing underneath.
#Transactional
public <S extends T> List<S> save(Iterable<S> entities) {
List<S> result = new ArrayList<S>();
if (entities == null) {
return result;
}
for (S entity : entities) {
result.add(save(entity));
}
return result;
}
My code - Entity :
#Entity
#Table(uniqueConstraints = #UniqueConstraint(columnNames = { "name" }, name = "uq_name"))
public class SampleContent {
#Id
#GeneratedValue
private Long id;
private String name;
//getter setters
}
Repository :
public interface SampleContentRepository extends JpaRepository<SampleContent, Serializable>{
}
JUnit test :
#Test
public void testCreate(){
List<SampleContent> sampleContentList = new ArrayList<>();
SampleContent sampleContent1 = new SampleContent();
sampleContent1.setName("dd");
SampleContent sampleContent2 = new SampleContent();
sampleContent2.setName("Roy");
SampleContent sampleContent3 = new SampleContent();
sampleContent3.setName("xx");
sampleContentList.add(sampleContent1);
sampleContentList.add(sampleContent2);
sampleContentList.add(sampleContent3);
try{
this.sampleContentRepository.save(sampleContentList);
}catch(DataIntegrityViolationException e){
System.err.println("constraint violation!");
}
}
There is an entity with name "Roy" already present in the table. So, the entire transaction fails and #Transactional rolls back.
I think you can use next steps:
Load existing entities from DB into Set
Override equals and hashCode methods based on name
call Set::addAll you antities (or just add them one by one)
save that Set to DB
Maybe it's suboptimal because forces you to make select * query. But I think it's much more effective then saving entities one by one to DB.
Accotding to this article you can use name as your business key, which has lots of benefits.