Spring Boot Unit Test Unique Constraint ignored - spring

I'm experiencing a different behaviour of my application REST controller between unit testing and running it normally. I have added a #Column(unique = true) on one of my fields to constrain a unique number, somehow it's being ignored in the unit test and I cannot understand why. I'm showing an abbreviated example below, basically adding a vehicle with same number twice to a shop:
#Test
public void CreateVehicle_SameNR_NOK() {
Shop shop = shopController.createVehicle(0L,1,new Vehicle("AAA",
"111222"));
Assert.assertEquals(shop.getVehicles().size(),1);
shopController.createVehicle(0L,1,new Vehicle("AAA",
"111222"));
Assert.assertEquals(shop.getVehicles().size(),1); //This fails.
}
#Entity
public class Vehicle {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE)
private long id;
#NotNull
private String name;
#NotNull
#Column(unique = true) //ignored in unit test?
private String nr;
protected Vehicle() {}
public Vehicle(String name, String nr) {
this.name = name;
this.nr = nr;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getNr() {
return nr;
}
public void setNr(String nr) {
this.nr = nr;
}
}
application.properties
spring.datasource.url=jdbc:h2:mem:test_mem;DB_CLOSE_ON_EXIT=FALSE
spring.datasource.username=sa
spring.datasource.password=
spring.datasource.driver-class-name=org.h2.Driver
logging.level.org.springframework.web=DEBUG
debug=true
ShopController extract
#RequestMapping(value ="{id}/shop/{shopId}/vehicle", method = RequestMethod.POST)
public Shop createShopVehicle(#PathVariable Long id, #PathVariable Long shopId, #RequestBody Vehicle vehicle) {
Shop shop = shopRepository.findOne(shopId);
shop.getVehicles().add(vehicle);
shopRepository.save(shop);
return shop;
}

I had this same issue in Spring Boot using JPA and Postgres.
Once I changed my statements from:
repository.save(entity)
to:
repository.saveAndFlush(entity)
it started throwing a DataIntegrityViolationException in the tests due to a violated unique constraint.
Since you're doing code-first I'll assume you're having it create the tables automatically for you. Verify in the created schema that the unique constraint is there after auto-creation of your tables.
Also for your code I'd recommend catching and re-throwing a 409 conflict.. ex:
try {
shopRepository.saveAndFlush(shop)
} catch (div: DataIntegrityViolationException) {
throw ResponseStatusException(org.springframework.http.HttpStatus.CONFLICT, "already exists")
}

Related

Problem when attempting a saveAndFlush commit (JPA ) when primary key is auto-generated from postGres trigger

I am using spring JPA to attempt to write records to a postGres DB. At the time of the commit, I am getting the following error:
Caused by: org.postgresql.util.PSQLException: ERROR: null value in column "col_id" violates not-null constraint
Detail: Failing row contains (null, null, null, null, null)
I have the following repository interface:
public interface MyRepo extends JpaRepository <MyModel, String> {
}
, the following model class:
#Entity
#Validated
#Table(name = "my_table", schema="common")
public class MyModel {
#Id
#Column(name = "col_id")
private String id;
#Column(name = "second_col")
private String secCol;
#Column(name = "third_col")
private String thirdCol;
#Column(name = "fourth_col")
private String fourthCol;
#Column(name = "fifth_col")
private String fifthCol;
public MyModel() {
}
public MyModel(String id, String secCol, String thirdCol, String fourthCol, String fifthCol) {
this.id = id;
this.secCol = secCol;
this.thirdCol = thirdCol;
this.fourthCol = fourthCol;
this.fifthCol = fifthCol;
}
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getSecCol() {
return secCol;
}
public void setSecCol(String secCol) {
this.secCol = secCol;
}
public String getThirdCol() {
return thirdCol;
}
public void setThirdCol(String thirdCol) {
this.thirdCol = thirdCol;
}
public String getFourthCol() {
return fourthCol;
}
public void setFourthCol(String fourthCol) {
this.fourthCol = fourthCol;
}
public String getFifthCol() {
return fifthCol;
}
public void setFifthCol(String fifthCol) {
this.fifthCol = fifthCol;
}
}
, and the relevant part of the service class:
public MyModel myModel (MyModel myModel) {
MyModel mm = null;
try {
mm = myRepo.saveAndFlush(myModel);
} catch ( Exception e) {
e.printStackTrace();
}
return mm;
}
UPDATE:
I finally realized that my problem is due to a database trigger that auto-generates primary key against a complex business rule. Also, I assume I might need to use a custom save method rather than the default repo.saveAndFlush? I would be grateful for any ideas given new information. Thanks!
I reproduced the exact same code in a test project with Postgres and it worked well for me. You are absolutely correct that the values of the model class are not populated. You must share your controller also. It may really help me to help you if I can get a look where your service is being called from. Only that will help me to deduce why your model values are being passed as null in the service call.

JHipster - Insert in the database with the GET method

I have to create an application with Jhipster but i never use it before.
When a user send a GET request to the address http://localhost:8080/api/newmesure/{mac-address}/{value}
I want to insert a new mesure in my database.
First i created 3 entity "Plantes", "Capteurs" and "Mesures" with this format :
Image here : https://i.stack.imgur.com/zJqia.png (I'm not allowed to post)
I activated the JPA Filtering to create a #Query to insert data in my database but i read that was not possible.
In /src/main/java/com/mycompany/myapp/web/rest/MesuresRessources.java :
/**
* REST controller for managing {#link com.mycompany.myapp.domain.Mesures}.
*/
#RestController
#RequestMapping("/api")
public class MesuresResource {
private final Logger log = LoggerFactory.getLogger(MesuresResource.class);
private static final String ENTITY_NAME = "mesures";
#Value("${jhipster.clientApp.name}")
private String applicationName;
private final MesuresService mesuresService;
private final MesuresQueryService mesuresQueryService;
public MesuresResource(MesuresService mesuresService, MesuresQueryService mesuresQueryService) {
this.mesuresService = mesuresService;
this.mesuresQueryService = mesuresQueryService;
}
#GetMapping("/newMesure/{mac}/{value}")
public String newMesure(#PathVariable String mac,#PathVariable int value) {
log.debug("Adresse MAC : "+mac);
log.debug("Valeur : "+value);
#Query("SELECT valeur FROM Mesures WHERE id = 1") //not working
Mesures getValeur(); //not working
return "Mesure ajoutée";
}
}
In /src/main/java/com/mycompany/myapp/domain/Mesures.java :
/**
* A Mesures.
*/
#Entity
#Table(name = "mesures")
public class Mesures implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(name = "valeur")
private Integer valeur;
#ManyToOne(optional = false)
#NotNull
#JsonIgnoreProperties("macs")
private Capteurs mac;
// jhipster-needle-entity-add-field - JHipster will add fields here, do not remove
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public Integer getValeur() {
return valeur;
}
public Mesures valeur(Integer valeur) {
this.valeur = valeur;
return this;
}
public void setValeur(Integer valeur) {
this.valeur = valeur;
}
public Capteurs getMac() {
return mac;
}
public Mesures mac(Capteurs capteurs) {
this.mac = capteurs;
return this;
}
public void setMac(Capteurs capteurs) {
this.mac = capteurs;
}
// jhipster-needle-entity-add-getters-setters - JHipster will add getters and setters here, do not remove
#Override
public boolean equals(Object o) {
if (this == o) {
return true;
}
if (!(o instanceof Mesures)) {
return false;
}
return id != null && id.equals(((Mesures) o).id);
}
#Override
public int hashCode() {
return 31;
}
#Override
public String toString() {
return "Mesures{" +
"id=" + getId() +
", valeur=" + getValeur() +
"}";
}
}
Louan
Learning java with JHipster is probably not a wise idea, it uses a very rich technology stack which might lose you unless you invest enough time to learn the basics.
There are many things wrong in your code and approach:
You can't use #Query annotation inside the body of method a of your REST controller, it must be used in your #Repository interface, this code can't compile. See https://www.baeldung.com/spring-data-jpa-query for a quick introduction
JPA filtering is not related to inserting into database
In HTTP/REST, GET method is supposed to be idempotent. For making changes in your database you should use POST or PUT methods. See What is idempotency in HTTP methods?
Your entity naming convention is not consistent: use singular for entity classes because each entity object represents one single instance of Mesure. Here you have Plantes (plural), Capteur (singular) and Mesures (plural). For table names, JHipster uses singular but plural is quite common too because a table holds many rows. Of course, this is just a convention and you or your team may decide to apply another (like a prefix for table names) but the key point is to be consistent.

RestController: returning Resource<> makes FetchType.LAZY to behave as FetchType.EAGER

I am having below mappings as Company having 1:N relation with CompanyFunds
#Entity
public class Company{
#Id
private Integer companyId;
private String name;
#OneToMany(mappedBy = "company")
private List<CompanyFund> companyFunds;
}
#Entity
public class CompanyFunds{
#Id
private Integer fundId;
private String name;
#ManyToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "company_id")
private Company company;
}
I am using Spring-data-Jpa for my persistence layer and below are the controller and service methods:
//controller
#GetMapping(value = "/{companyId}")
public Resource<Company> find(#PathVariable Integer companyId) {
Resource<Company> companyResource = companyService.find(companyId);
return companyResource;
}
//service
public Resource<CompanyTypeOther> find(Integer companyId) {
Company company = companyRepository.findById(companyId);
return restResourceAssembler.toResource(company);
}
#Component
public class RestResourceAssembler implements ResourceAssembler<T, Resource<T>> {
private EntityLinks entityLinks;
public RestResourceAssembler(EntityLinks entityLinks) {
this.entityLinks = entityLinks;
}
#Override
public Resource<T> toResource(T entity) {
Resource<T> resource = new Resource<>(entity);
resource.add(entityLinks.linkToSingleResource(entity.getClass(), entity.getId()).withSelfRel());
return resource;
}
}
Now the weird thing is, until the return companyResource;(in controller) doesn't get executed, the companyResource contains null for companyFunds i.e the LAZY loading is working fine till that point. But the moment the return companyResource; executes, something goes inside the Spring and the Select statement for CompanyFund gets fired. I debugged the steps and below is the code(try block) responsible for this:
public class ServletInvocableHandlerMethod extends InvocableHandlerMethod {
......
public void invokeAndHandle(ServletWebRequest webRequest, ModelAndViewContainer mavContainer,
........other code
try {
this.returnValueHandlers.handleReturnValue(
returnValue, getReturnValueType(returnValue), mavContainer, webRequest);
}
catch (Exception ex) {
if (logger.isTraceEnabled()) {
logger.trace(formatErrorForReturnValue(returnValue), ex);
}
throw ex;
}
}
.....
}
There are no toString() declared in entities, also no getCompanyFund() called. Can't understand what Spring is doing with returnValue above, so that some getters(or something) are getting called.
One more thing I noticed is, this problem only occurs when the Resource<Company> is returned. If I return Company from the controller, nothing unexpected happens. Lazy loading works fine.
Since I want to lazy load the entity, a little fix/hack solved the problem for me(as of now).
#JsonIgnore
#OneToMany(mappedBy = "company")
private List<CompanyFund> companyFunds;
#JsonIgnore prevents LAZY loaded entity from being serialized. So I guess the jackson is the culprit here.
This is not a fix is my view but just a hack to do the thing. Still waiting someone from Spring team to reply.

Spring-Data-JPA atomic insert with dependencies

I want to insert Entities to a database from a scalable microservice. I tried #Lock(LockModeType.PESSIMISTIC_WRITE) to prevent from doubled entries. The problem is, I have dependencies on my Entities.
A basic example is:
TestEntity.java
public class TestEntity {
#GeneratedValue()
#Id
private Long id;
private String string;
#ManyToOne
private TestEntityParent testEntityParent;
}
TestEntityParent.java
public class TestEntityParent {
#GeneratedValue()
#Id
private Long id;
private String stringTwo;
#OneToMany(mappedBy = "testEntityParent")
private List<TestEntity> testEntities;
}
TestEnityRepository.java
public interface TestEnityRepository extends JpaRepository<TestEntity,Long> {
#Lock(LockModeType.PESSIMISTIC_WRITE)
TestEntity saveAndFlush(TestEntity testEntity);
Optional<TestEntity> findByStringAndTestEntityParentStringTwo(String string, String stringTwo);
}
TestEntityParentRepository.java
public interface TestEntityParentRepository extends JpaRepository<TestEntityParent, Long> {
#Lock(LockModeType.PESSIMISTIC_WRITE)
TestEntityParent save(TestEntityParent testEntityParent);
Optional<TestEntityParent> findByStringTwo(String stringTwo);
}
AtomicDbService.java
#Service
public class AtomicDbService {
#Autowired
TestEnityRepository testEnityRepository;
#Autowired
TestEntityParentRepository testEntityParentRepository;
#Transactional
public TestEntity atomicInsert(TestEntity testEntity) {
TestEntityParent testEntityParent = testEntityParentRepository.findByStringTwo(testEntity.getTestEntityParent().getStringTwo())
.orElse(testEntityParentRepository.save(testEntity.getTestEntityParent()));
return testEnityRepository.findByStringAndTestEntityParentStringTwo(
testEntity.getString(), testEntity.getTestEntityParent().getStringTwo()
).orElse(testEnityRepository
.save(
TestEntity.builder()
.string(testEntity.getString())
.testEntityParent(testEntityParent)
.build()
)
);
}
}
My test case:
#Test
#Transactional
public void testAtomicInsert(){
TestEntityParent testEntityParent = TestEntityParent.builder().stringTwo("testTwo").build();
TestEntity testEntity = TestEntity.builder().string("test").testEntityParent(testEntityParent).build();
atomicDbService.atomicInsert(testEntity);
System.out.println(testEnityRepository.findAll());
atomicDbService.atomicInsert(testEntity);
System.out.println(testEnityRepository.findAll());
atomicDbService.atomicInsert(testEntity);
System.out.println(testEnityRepository.findAll());
System.out.println(testEnityRepository.findAll());
}
I get the following answer:
[TestEntity(id=2, string=test, testEntityParent=TestEntityParent(id=1, stringTwo=testTwo, testEntities=null))]
[TestEntity(id=2, string=test, testEntityParent=TestEntityParent(id=1, stringTwo=testTwo, testEntities=null)), TestEntity(id=3, string=test, testEntityParent=TestEntityParent(id=1, stringTwo=testTwo, testEntities=null))]
and an error:
query did not return a unique result: 2;
Without dependencies everything works fine.
UPDATE:
Adding #Lock(LockModeType.PESSIMISTIC_WRITE) to the find method leads to
Feature not supported: "MVCC=TRUE && FOR UPDATE && JOIN"; SQL statement:
... same applies to
#Lock(LockModeType.PESSIMISTIC_WRITE)
#Query("SELECT e from TestEntity e join e.testEntityParent p where e.string = :string and p.stringTwo = :stringTwo ")
Optional<TestEntity> findWhatever(#Param("string") String string, #Param("stringTwo") String stringTwo);
... since for update is always generated.
Apparently, it was a stupid mistake I needed to replace orElse with orElseGet and a lambda and everything worked, even without all those #Lock, etc - tricks.
Still I don't understand what exactly went wrong with the transactions and why.

Spring JPA: Locking parent row when inserting one to many child record

We have two tables that have a one to many relationship. When we insert multiple records into the child table across multiple threads (more specifically across multiple REST web requests) we are running into lost update issues due to a race condition.
What we need to be able to do is have JPA recognize that the entity has been updated elsewhere prior to inserting the child record. I've tried using the #Version annotation approach but that doesn't seem to do the trick as the update/insert (I guess...) is happening on another table. I tried adding a version timestamp column on the parent table that is updated on every update but that didn't seem to do the trick either.
I think what I actually need to do is get a reference to the EntityManager directly so that I can issue a lock() command on the record prior to calling save(). I'm just too new to Spring to know if
A) that is indeed the correct approach,
B) if there is a better/easier way to do what we are trying to accomplish, and
C) how to actually do that.
Also, I am aware of the #OneToMany annotation but that didn't seem to do anything.
I've truncated the code below for brevity and I also created a trimmed down version of the code that demonstrates the problem and will hopefully make it easier to see what I am trying to do. In the test if you change the thread pool number to 1 you can see the test pass.
Engagement class:
#Entity
public class Engagement implements Serializable {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id")
private Long id;
#ElementCollection(fetch = EAGER)
private List<String> assignedUsers;
#Version
private Long version;
private LocalDateTime updatedOn;
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public Long getVersion(){return version;}
public void setVersion(Long version){this.version = version;}
public LocalDateTime getUpdatedOn(){
return updatedOn;
}
public void setUpdatedOn(LocalDateTime updatedOn) {
this.updatedOn = updatedOn;
}
public List<String> getAssignedUsers() {
return assignedUsers;
}
public void setAssignedUsers(List<String> assignedUsers) {
this.assignedUsers = assignedUsers;
}
public Engagement() {
}
}
User class:
public final class User {
private final String name;
private final String email;
private final String userId;
private final List<Engagement> engagements;
#ConstructorProperties({"roles", "name", "email", "userId", "engagements"})
User(String name, String email, String userId, List<Engagement> engagements) {
this.name = name;
this.email = email;
this.userId = userId;
this.engagements = engagements;
}
public static User.UserBuilder builder() {
return new User.UserBuilder();
}
public String getName() {
return this.name;
}
public String getEmail() {
return this.email;
}
public String getUserId() {
return this.userId;
}
public List<Engagement> getEngagements() {
return this.engagements;
}
public static final class UserBuilder {
private String name;
private String email;
private String userId;
private List<Engagement> engagements;
UserBuilder() {
}
public User.UserBuilder name(String name) {
this.name = name;
return this;
}
public User.UserBuilder email(String email) {
this.email = email;
return this;
}
public User.UserBuilder userId(String userId) {
this.userId = userId;
return this;
}
public User.UserBuilder engagements(List<Engagement> engagements) {
this.engagements = engagements;
return this;
}
public User build() {
return new User(this.name, this.email, this.userId, this.engagements);
}
public String toString() {
return "User.UserBuilder(name=" + this.name + ", email=" + this.email + ", userId=" + this.userId + ", engagements=" + this.engagements + ")";
}
}
}
Thread test:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest
public class EngagementTest {
#Mock
UsersAuthService usersService;
#Autowired
EngagementsRepository engagementsRepository;
UsersAuthService authService;
#Before
public void init() {
MockitoAnnotations.initMocks(this);
authService = new UsersAuthServiceImpl(usersService, engagementsRepository);
}
#Test
public void addingMultipleUsersAtOnceSucceeds() throws InterruptedException {
Long engagementId = 1L;
String userId1 = "user1";
String userId2 = "user2";
String userId3 = "user3";
String userId4 = "user4";
String userId5 = "user5";
String auth = "asdf";
User adminUser = User.builder()
.userId("adminUser")
.email("user#user.com")
.name("Admin User")
.build();
Engagement engagement = new Engagement();
engagement.setAssignedUsers(new ArrayList<>());
engagement.getAssignedUsers().add(adminUser.getUserId());
engagementsRepository.save(engagement);
ExecutorService executorService = Executors.newFixedThreadPool(5);//change this to 1 to see the test pass
List<Callable<Engagement>> callableList = Arrays.asList(
addUserThread(engagementId, userId1, auth, adminUser),
addUserThread(engagementId, userId2, auth, adminUser),
addUserThread(engagementId, userId3, auth, adminUser),
addUserThread(engagementId, userId4, auth, adminUser),
addUserThread(engagementId, userId5, auth, adminUser));
executorService.invokeAll(callableList);
Engagement after = engagementsRepository.findById(engagementId);
assertEquals(6, after.getAssignedUsers().size());
}
private Callable<Engagement> addUserThread(Long engagementId, String userId1, String auth, User adminUser) {
return () -> authService.addUserTo(engagementId, userId1, auth, adminUser);
}
}
What's happening here is that you submit the callbacks for execution but never actually wait for their completion before checking the result. You need to use the List<Future<Engagement>> to actually wait for the results to complete before proceeding.
Something like this would do the trick:
executorService.invokeAll(callableList).forEach(it -> {
try {
it.get(500, TimeUnit.MILLISECONDS);
} catch (InterruptedException | ExecutionException | TimeoutException e) {
e.printStackTrace();
}
});
Note that this is not a proper way to deal with the exception case but it causes the code to wait for completion. If you have that in place you see the threads properly rejecting some of the updates with an ObjectOptimisticLockingFailureException:
java.util.concurrent.ExecutionException: org.springframework.orm.ObjectOptimisticLockingFailureException: Object of class [com.example.racecondition.engagement.Engagement] with identifier [1]: optimistic locking failed; nested exception is org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) : [com.example.racecondition.engagement.Engagement#1]
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at com.example.racecondition.EngagementTest.lambda$0(EngagementTest.java:68)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at com.example.racecondition.EngagementTest.addingMultipleUsersAtOnceSucceeds(EngagementTest.java:66)
What's weird about the test case beyond that is that UsersAuthServiceImpl carries an #Transactional but the test case manually instantiates that class, so that there's no transactional proxy in place already. This causes the calls to findById(…) and save(…) from within addToUser(…) to run in two transactions. Tweaking that doesn't change the output though.
I think what I actually need to do is get a reference to the EntityManager directly so that I can issue a lock() command on the record prior to calling save(). I'm just too new to Spring to know if
A) that is indeed the correct approach,
If I understand you correctly you want to basically force a version increment on an entity so that if multiple threads do that one fails.
You can indeed achieve that by locking the entity in question using LockModeType.PESSIMISTIC_FORCE_INCREMENT or LockModeType.OPTIMISTIC_FORCE_INCREMENT.
B) if there is a better/easier way to do what we are trying to accomplish, and
C) how to actually do that.
With Spring Data probably the best way to do that is using the #Lock annotation on the method you use to load the entity.

Resources