I have a very basic Controller (Rest) -> Service -> Repository call as given below in Spring Boot. I was debugging the time taken by each step, it looks like there a significant percentage of time is taken by Controller to call the Service method. In the below example -
Total time: 60ms
Time taken by Controller to call Service method (marked: //POINT B):
30ms
Time taken by Service to call Repository method (marked: //POINT C):
1ms
Time taken by Repository to return data from DB: 20ms
......
My question/doubt is on the 30ms taken by the first step. There is no application logic involved between POINT A and B. Is the 30ms because of any initialization that Spring has to do each time Controller tries to call the Service? 30ms is not long, but since it was taking a significant amount of time of the overall processing, I was wondering if I was doing any Spring configuration wrong.
Controller
#Autowired
ContactService contactService;
#RequestMapping(value = "/contact/{contactId}", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
public ResponseEntity<ContactResponse> fetchContactById(#PathVariable Integer contactId) throws Exception {
ContactResponse contact = new ContactResponse();
//POINT A
contact.setContact(contactService.findContactById(contactId));
.......
return new ResponseEntity<ContactResponse>(contact, HttpStatus.OK);
}
Service class
#Autowired
ContactRepository contactRepository;
#Service
public class ContactService {
//POINT B
public ContactDTO findContactById(Integer contactId) throws Exception {
return contactRepository.findContactById(contactId);
}
}
Repository
#Repository
public class ContactRepository {
//POINT C
public ContactDTO findContactById(Integer contactId) {
.......
return contactDTO;
}
}
First of all, I would recommend to perform diagnostics using profiler (IntelliJ has build-in one, jprofiler is better but it costs, it has trial period though).
However, based on your laconic diagnostics, I would say that it is related to autocommit settings: by default autocommit is enabled, so, when transaction starts Hibernate tries to disable autocommit via acquiring connection from connection pool and issuing java.sql.Connection.setAutoCommit(false), which takes time.
Related
I am trying to write my first integration test with spring boot application. So to write proper test cases I am using #Transactional provided by spring so that state of my in memory data base remains same before and after execution the test.
Thing is #Transactional is working as expected my test execution starts it inserts some data for testing into db and when execution finish it rollbacks the changes done by it but the problem is when the test execution goes to the code I am testing it tries to fetch data persisted by test but does not find any data there. I've shared my code for better understanding.
#RunWith(SpringRunner.class)
#SpringBootTest
#Transactional
public class MyFirstControllerTest {
#Autowired
private TestRestTemplate restTemplate;
#Autowired
private MyFirstRepository myFirstRepo;
#Test
public void testMethod() {
//insert some data.
Dummy data = new DummyData();
myFirstRepo.save(data);
//hits the rest route to get data.
ResponseEntity<String> response =
this.restTemplate.getForEntity("/dummys", String.class); // did not find any data in db.
myFirstRepo.findAll(); //working - fetched the data inserted in test.
}
}
Below is my controller.
#Controller
#Validated
public class MyFirstController {
#Autowired
private MyFirstRepository myFirstRepo;
#GetMapping("/dummys")
public ResponseEntity<List<DummyDataDto>> getDummyData() {
List<DummyData> data = myFirstRepo.findAll(); //does not finds any data
//convert list of dummy data to list of dummydata dto.
return new ResponseEntity<DummyDataDto>(List<DummyDataDto>, HttpStatus.OK);
}
}
This is occurring because of the isolation level of in memory database that you have been using. I assume that default isolation level of that database in Read Committed.
If you are using H2 then you may find details of isolation level here.
All you need to use Read Uncommited isolation level for integration test, if your business requirement says to do so.
In my Spring Boot project I have implemented following service method:
#Transactional
public boolean validateBoard(Board board) {
boolean result = false;
if (inProgress(board)) {
if (!canPlayWithCurrentBoard(board)) {
update(board, new Date(), Board.AFK);
throw new InvalidStateException(ErrorMessage.BOARD_TIMEOUT_REACHED);
}
if (!canSelectCards(board)) {
update(board, new Date(), Board.COMPLETED);
throw new InvalidStateException(ErrorMessage.ALL_BOARD_CARDS_ALREADY_SELECTED);
}
result = true;
}
return result;
}
Inside this method I use another service method which is called update:
#Transactional(propagation = Propagation.REQUIRES_NEW)
public Board update(Board board, Date finishedDate, Integer status) {
board.setStatus(status);
board.setFinishedDate(finishedDate);
return boardRepository.save(board);
}
I need to commit changes to database in update method independently of the owner transaction which is started in validateBoard method. Right now any changes is rolling back in case of any exception.
Even with #Transactional(propagation = Propagation.REQUIRES_NEW) it doesn't work.
How to correctly do this with Spring and allow nested transactions ?
This documentation covers your problem - https://docs.spring.io/spring-framework/docs/current/spring-framework-reference/data-access.html#transaction-declarative-annotations
In proxy mode (which is the default), only external method calls coming in through the proxy are intercepted. This means that self-invocation, in effect, a method within the target object calling another method of the target object, will not lead to an actual transaction at runtime even if the invoked method is marked with #Transactional. Also, the proxy must be fully initialized to provide the expected behaviour so you should not rely on this feature in your initialization code, i.e. #PostConstruct.
However, there is an option to switch to AspectJ mode
Using "self" inject pattern you can resolve this issue.
sample code like below:
#Service #Transactional
public class YourService {
//... your member
#Autowired
private YourService self; //inject proxy as an instance member variable ;
#Transactional(propagation= Propagation.REQUIRES_NEW)
public void methodFoo() {
//...
}
public void methodBar() {
//call self.methodFoo() rather than this.methodFoo()
self.methodFoo();
}
}
The point is using "self" rather than "this".
The basic thumb rule in terms of nested Transactions is that they are completely dependent on the underlying database, i.e. support for Nested Transactions and their handling is database dependent and varies with it.
In some databases, changes made by the nested transaction are not seen by the 'host' transaction until the nested transaction is committed. This can be achieved using Transaction isolation in #Transactional (isolation = "")
You need to identify the place in your code from where an exception is thrown, i.e. from the parent method: "validateBoard" or from the child method: "update".
Your code snippet shows that you are explicitly throwing the exceptions.
YOU MUST KNOW::
In its default configuration, Spring Frameworkâs transaction
infrastructure code only marks a transaction for rollback in the case
of runtime, unchecked exceptions; that is when the thrown exception is
an instance or subclass of RuntimeException.
But #Transactional never rolls back a transaction for any checked exception.
Thus, Spring allows you to define
Exception for which transaction should be rollbacked
Exception for which transaction shouldn't be rollbacked
Try annotating your child method: update with #Transactional(no-rollback-for="ExceptionName") or your parent method.
Your transaction annotation in update method will not be regarded by Spring transaction infrastructure if called from some method of same class. For more understanding on how Spring transaction infrastructure works please refer to this.
Your problem is a method's call from another method inside the same proxy.It's self-invocation.
In your case, you can easily fix it without moving a method inside another service (why do you need to create another service just for moving some method from one service to another just for avoid self-invocation?), just to call the second method not directly from current class, but from spring container. In this case you call proxy second method with transaction not with self-invocatio.
This principle is useful for any proxy-object when you need self-invocation, not only a transactional proxy.
#Service
class SomeService ..... {
-->> #Autorired
-->> private ApplicationContext context;
-->> //or with implementing ApplicationContextAware
#Transactional(any propagation , it's not important in this case)
public boolean methodOne(SomeObject object) {
.......
-->> here you get a proxy from context and call a method from this proxy
-->>context.getBean(SomeService.class).
methodTwo(object);
......
}
#Transactional(any propagation , it's not important in this case)public boolean
methodTwo(SomeObject object) {
.......
}
}
when you do call context.getBean(SomeService.class).methodTwo(object); container returns proxy object and on this proxy you can call methodTwo(...) with transaction.
You could create a new service (CustomTransactionalService) that will run your code in a new transaction :
#Service
public class CustomTransactionalService {
#Transactional(propagation= Propagation.REQUIRES_NEW)
public <U> U runInNewTransaction(final Supplier<U> supplier) {
return supplier.get();
}
#Transactional(propagation= Propagation.REQUIRES_NEW)
public void runInNewTransaction(final Runnable runnable) {
runnable.run();
}
}
And then :
#Service
public class YourService {
#Autowired
private CustomTransactionalService customTransactionalService;
#Transactional
public boolean validateBoard(Board board) {
// ...
}
public Board update(Board board, Date finishedDate, Integer status) {
this.customTransactionalService.runInNewTransaction(() -> {
// ...
});
}
}
I seem to have struck an issue and have no real clue on how to solve.
My current app is based on Spring Boot with JPA and the following code gets a lock when run for the second execution.
#RequestMapping(value="/", method = RequestMethod.GET)
public String index() {
repository.save(new RawData("test"));
repository.save(new RawData("test"));
// hangs when the method index() is run 2 sequentially
RawData rawData = rawDataRepository.findOne(1L);
System.out.println(rawData);
return "#: " + repository.count();
}
When run the first time all seems ok, but executing the same code 2 times gives me a lock on:
RawData rawData = rawDataRepository.findOne(1L);
Also trying to connect to the DB gives me a lock timeout when the methods hangs or waits for a timeout.
Calling the same code in Spring Service results in the same behaviour.
#Component
public class SyncService {
#Autowired
RawDataRepository rawDataRepository;
void syncWithRemote() {
// hang on this line...
RawData rawData = rawDataRepository.findOne(1L);
System.out.println(rawData);
}
}
You should use two techniques:
Use optimistic locking by using #Version field in your entities
Add transactions support by annotating your methods by #Transactional annotation. Normally you also have to annotate Configuration class by #EnableTransactionManagement but Spring Boot makes it for you
That should solve your problems
I have developed a Spring Data repository, MemberRepository interface, that extends org.springframework.data.jpa.repository.JpaRepository. MemberRepository has a method:
#Cacheable(CacheConfiguration.DATABASE_CACHE_NAME)
Member findByEmail(String email);
The result is cached by Spring cache abstraction (backed by a ConcurrentMapCache).
The issue I have is that I want to write an integration test (against hsqldb) that asserts that the result is retrieved from db the first time and from cache the second time.
I initially thought of mocking the jpa infrastructure (entity manager, etc.) and somehow assert that the entity manager is not called the second time but it seems too hard/cumbersome (see https://stackoverflow.com/a/23442457/536299).
Can someone then please provide advice as to how to test the caching behavior of a Spring Data Repository method annotated with #Cacheable?
If you want to test a technical aspect like caching, don't use a database at all. It's important to understand what you'd like to test here. You want to make sure the method invocation is avoided for the invocation with the very same arguments. The repository fronting a database is a completely orthogonal aspect to this topic.
Here's what I'd recommend:
Set up an integration test that configures declarative caching (or imports the necessary bit's and pieces from your production configuration.
Configure a mock instance of your repository.
Write a test case to set up the expected behavior of the mock, invoke the methods and verify the output accordingly.
Sample
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration
public class CachingIntegrationTest {
// Your repository interface
interface MyRepo extends Repository<Object, Long> {
#Cacheable("sample")
Object findByEmail(String email);
}
#Configuration
#EnableCaching
static class Config {
// Simulating your caching configuration
#Bean
CacheManager cacheManager() {
return new ConcurrentMapCacheManager("sample");
}
// A repository mock instead of the real proxy
#Bean
MyRepo myRepo() {
return Mockito.mock(MyRepo.class);
}
}
#Autowired CacheManager manager;
#Autowired MyRepo repo;
#Test
public void methodInvocationShouldBeCached() {
Object first = new Object();
Object second = new Object();
// Set up the mock to return *different* objects for the first and second call
Mockito.when(repo.findByEmail(Mockito.any(String.class))).thenReturn(first, second);
// First invocation returns object returned by the method
Object result = repo.findByEmail("foo");
assertThat(result, is(first));
// Second invocation should return cached value, *not* second (as set up above)
result = repo.findByEmail("foo");
assertThat(result, is(first));
// Verify repository method was invoked once
Mockito.verify(repo, Mockito.times(1)).findByEmail("foo");
assertThat(manager.getCache("sample").get("foo"), is(notNullValue()));
// Third invocation with different key is triggers the second invocation of the repo method
result = repo.findByEmail("bar");
assertThat(result, is(second));
}
}
As you can see, we do a bit of over-testing here:
The most relevant check, I think is that the second call returns the first object. That's what the caching is all about. The first two calls with the same key return the same object, whereas the third call with a different key results in the second actual invocation on the repository.
We strengthen the test case by checking that the cache actually has a value for the first key. One could even extend that to check for the actual value. On the other hand, I also think it's fine to avoid doing that as you tend to test more of the internals of the mechanism rather than the application level behavior.
Key take-aways
You don't need any infrastructure to be in place to test container behavior.
Setting a test case up is easy and straight forward.
Well-designed components let you write simple test cases and require less integration leg work for testing.
I tried testing the cache behavior in my app using Oliver's example. In my case my cache is set at the service layer and I want to verify that my repo is being called the right number of times. I'm using spock mocks instead of mockito. I spent some time trying to figure out why my tests are failing, until I realized that tests running first are populating the cache and effecting the other tests. After clearing the cache for every test they started behaving as expected.
Here's what I ended up with:
#ContextConfiguration
class FooBarServiceCacheTest extends Specification {
#TestConfiguration
#EnableCaching
static class Config {
def mockFactory = new DetachedMockFactory()
def fooBarRepository = mockFactory.Mock(FooBarRepository)
#Bean
CacheManager cacheManager() {
new ConcurrentMapCacheManager(FOOBARS)
}
#Bean
FooBarRepository fooBarRepository() {
fooBarRepository
}
#Bean
FooBarService getFooBarService() {
new FooBarService(fooBarRepository)
}
}
#Autowired
#Subject
FooBarService fooBarService
#Autowired
FooBarRepository fooBarRepository
#Autowired
CacheManager cacheManager
def "setup"(){
// we want to start each test with an new cache
cacheManager.getCache(FOOBARS).clear()
}
def "should return cached foobars "() {
given:
final foobars = [new FooBar(), new FooBar()]
when:
fooBarService.getFooBars()
fooBarService.getFooBars()
final fooBars = fooBarService.getFooBars()
then:
1 * fooBarRepository.findAll() >> foobars
}
def "should return new foobars after clearing cache"() {
given:
final foobars = [new FooBar(), new FooBar()]
when:
fooBarService.getFooBars()
fooBarService.clearCache()
final fooBars = fooBarService.getFooBars()
then:
2 * fooBarRepository.findAll() >> foobars
}
}
I get a LazyInitializationException by combining the conversionservice with a #Transactional behavior in a MVC controller.
Following fails:
#RequestMapping(value = "/{userId}", method = RequestMethod.GET)
#ResponseBody
#Transactional
public JsonUser getUser(#PathVariable("userId") User user) {
// convertToJsonUser triggers lazy loading of User.adresses
return mUserPresenter.convertToJsonUser(user);
}
... with following exception:
org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: User.adresses, could not initialize proxy - no Session
But the same code without the conversionservice succeeds:
#RequestMapping(value = "/{userId}", method = RequestMethod.GET)
#ResponseBody
#Transactional
public JsonUser getUser(#PathVariable("userId") Long userId) {
User user = mUserRepository.findOne(userId);
// convertToJsonUser triggers lazy loading of User.adresses
return mUserPresenter.convertToJsonUser(user);
}
and the same code without the transactional behavior succeeds:
#RequestMapping(value = "/{userId}", method = RequestMethod.GET)
#ResponseBody
public JsonUser getUser(#PathVariable("userId") User user) {
// changedConvertToJsonUser DOESN'T trigger lazy loading of User.adresses
return mUserPresenter.changedConvertToJsonUser(user);
}
The conversion seems to occur in its own transaction before the main transaction to be opened by the #Transactional annotation. As a result, the User loaded by the conversionmanager is not bound to the main transaction and lazy loading fails for that reason.
Is that behavior known?
How can I get rid of it?
Did I forget something in the configuration?
Thank you in advance!!!
Controller is not a perfect place to put #Transactional annotations. It would be better to add them to service methods. This may cause problems because you may have a proxy object of controller which is mixed up with proxy from transaction manager and you can not predict what the order of execution will be. So my advice is: try to add a service layer.. even a very simple put there #Transactional annotations and test the code again.
... the conversionservice is per se the way of loading entities ...
No, it's not. It's like the name says: conversion. If you load entities, then that's your choice, not Spring's.
if it is not recommanded to open a transaction in the controller-layer, then why did Spring developers have developed the conversionservice feature?
The service in ConversionService implies that we are not in the controller layer.
The conversion seems to occur in its own transaction before the main transaction to be opened by the #Transactional annotation
There is no transaction opened by #Transactional in this case, because it doesn't work.
Transactions usually work on the service layer. You could have something like:
MyUserService implements UserService {
#Override
#Transactional(readonly = true)
public User getUserById(int id) {
But that's not your problem. Your problem is that the User object is not fully initialized. You would need a transaction on convertToJsonUser.
As a side note: The people who develop Spring prefer using the ids directly and load entities in the controller, not before.