I'm working on Spring Boot Application and I use Spring Data, HikariCP and JDBC, but I have a problem.
Inside one method I get a particular User from the database using a Spring Data repository. After I get the User from the DB I use JdbcTemplate.query to get some other information from the DB with username of above User but the application freezes and after some time it throws
java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30006ms.
When I debug the code I see that jdbctemplate is using hikariCP as datasource.
This is the code I'm using:
public User getUser() {
User user = userRepository.findByUsernameAndEnabledTrue("username");
List<String> roles= getUserRoles(user.getUsername())
return user;
}
private List<String> getUserRoles(String username) {
List<String> roles = this.jdbcTemplate.query("SELECT ga.authority FROM group_authorities ga INNER JOIN group_members gm ON gm.group_id = ga.group_id INNER JOIN users u ON gm.username=u.username WHERE u.username=?;",
new Object[]{username},new ResultSetExtractor<List<String>>() {
#Override
public List<String> extractData(ResultSet resultSet) throws SQLException, DataAccessException {
List<String> roles = new ArrayList<>();
while (resultSet.next()) {
roles.add(resultSet.getString("authority"));
}
return roles;
}
});
return roles;
}
I made a research how to use those together and share same transaction or something like but unfortunately can't fix it.
Your problem seems to be that the JdbcTemplate uses a different connection than your repository. And since the connection pool makes only one connection available and that is already used by the repository, you run into the timeout.
Increasing the capacity of the connection pool would fix that immediate problem, but the repository and the JdbcTemplate would use different connections and therefore transactions, which you probably don't want.
You don't show where your JdbcTemplate gets it's connection from, but that is probably where things go wrong. To fix it get the EntityManager injected. Then get the Connection from it. How to do that is JPA implementation dependent. Here are the versions for Eclipse Link and for Hibernate. Then use that Connection to create your JdbcTemplate.
It is possible just inject DataSource or JdbcTemplate in a custom repository, for example. And if JPA and JDBC calls are inside one transaction (generated by #Transactional, for example), Spring is smart enough to use JPATransactionManager for both cases with the same transaction and connection.
https://billykorando.com/2019/05/06/jpa-or-sql-in-a-spring-boot-application-why-not-both/
Related
I am using spring-boot-starter-data-mongodb:2.2.1.RELEASE and trying to add transaction support for Mongo DB operations.
I have the account service below, where documents are inserted into two collections accounts and profiles. In case an error occurs while inserting into profile collection, the insertion into accounts should rollback. I have configured Spring transactions using MongoTransactionManager.
#Service
public class AccountService {
#Transactional
public void register(UserAccount userAccount) {
userAccount = accountRepository.save(userAccount);
UserProfile userProfile = new UserProfile(userAccountDoc.getId());
userProfile = profileRepository.save(userProfile);
}
}
Enabled Spring transaction support for MongoDB.
#Configuration
public abstract class MongoConfig extends AbstractMongoConfiguration {
#Bean
MongoTransactionManager transactionManager(MongoDbFactory dbFactory) {
return new MongoTransactionManager(dbFactory);
}
}
As per Spring reference doc https://docs.spring.io/spring-data/mongodb/docs/2.2.1.RELEASE/reference/html/#mongo.transactions that is all required to enable transactions for MongoDB. But this is not working. Insertions into accounts collection are not rolled back in case error occurs while inserting into profiles collection. Any suggestions if I am missing anything?
I would use command monitoring or examine query logs on the server side to ensure that:
session ids are sent with queries/write commands
transaction operations are performed (there is no startTransaction command but there is a commitTransaction)
I have a requirement for which I need to execute a bunch of random complex queries with multiple joins for reporting purposes. So I am planning to use entitymanager native query feature directly. I just tried and it seems to work.
#Service
public class SampleService {
#Autowired
private EntityManager entityManager;
public List<Object[]> execute(String sql){
Query query = entityManager.createNativeQuery(sql);
return query.getResultList();
}
}
This code is invoked once in every 30 seconds. Single threaded - scheduled process.
Question:
Should I be using entity manager or entity manager factory?
Should I close the connection here? or is it managed automatically?
How to reduce the DB connection pool - as it is not multi threaded app or Should I not be worried about that?
Any other suggestions!?
Should I be using entity manager or entity manager factory?
Injecting EntityManager Vs. EntityManagerFactory
EntityManager looks fine in this instance.
Should I close the connection here? or is it managed automatically?
No I dont think you need to as the manager handles this.
How to reduce the DB connection pool - as it is not multi threaded app or Should I not be worried about that?
I doubt you need concern yourself with the connection pools unless you are expecting large volumes and your application is running slowly under load. Try doing some bench marking you may have much more capacity than you need and be prematurely optimising your app.
It more likely you would you increase it number of connections rather than decrease. To increase the number of connections you do that in the application.properties (or application.yml)
Any other suggestions!?
Rather than a generic method I would consider having a separate repository class outside of the service and have that repository method do something specific. Make a method return a specific result or thing rather than pass in any sql.
As a rough outline of two seperate classes (files) something like this
#Service
public class SampleService {
#Autowired
private MyAuthorNativeRepository myAuthorNaviveRepository;
public List<Author> getAuthors(){
return myAuthorRepository.getAuthors();
}
}
#Service
public class MyAuthorNativeRepository {
#Autowired
private EntityManager entityManager;
public List<Author> getAuthors(){
Query q = entityManager.createNativeQuery("SELECT blah blah FROM Author");
List<Author> authors = new ArrayList();
for (Object[] row : q.getResultList()) {
Author author = new Author();
author.setName(row[0]);
authors.add(author);
}
return authors;
}
}
I need to create a service that can manage multiple datasources.
These datasources do not necessarily exist when the app when first running the app, actually an endpoint will create new databases, and I would like to be able to switch to them and create data.
For example, let's say that I have 3 databases, A, B and C, then I start the app, I use the endpoint that creates D, then I want to use D.
Is that possible?
I know how to switch to other datasources if those exist, but I can't see any solutions for now that would make my request possible.
Have you got any ideas?
Thanks
To implement multi-tenancy with Spring Boot we can use AbstractRoutingDataSource as base DataSource class for all 'tenant databases'.
It has one abstract method determineCurrentLookupKey that we have to override. It tells the AbstractRoutingDataSource which of the tenant datasource it has to provide at the moment to work with. Because it works in the multi-threading environment, the information of the chosen tenant should be stored in ThreadLocal variable.
The AbstractRoutingDataSource stores the info of the tenant datasources in its private Map<Object, Object> targetDataSources. The key of this map is a tenant identifier (for example the String type) and the value - the tenant datasource. To put our tenant datasources to this map we have to use its setter setTargetDataSources.
The AbstractRoutingDataSource will not work without 'default' datasource which we have to set with method setDefaultTargetDataSource(Object defaultTargetDataSource).
After we set the tenant datasources and the default one, we have to invoke method afterPropertiesSet() to tell the AbstractRoutingDataSource to update its state.
So our 'MultiTenantManager' class can be like this:
#Configuration
public class MultiTenantManager {
private final ThreadLocal<String> currentTenant = new ThreadLocal<>();
private final Map<Object, Object> tenantDataSources = new ConcurrentHashMap<>();
private final DataSourceProperties properties;
private AbstractRoutingDataSource multiTenantDataSource;
public MultiTenantManager(DataSourceProperties properties) {
this.properties = properties;
}
#Bean
public DataSource dataSource() {
multiTenantDataSource = new AbstractRoutingDataSource() {
#Override
protected Object determineCurrentLookupKey() {
return currentTenant.get();
}
};
multiTenantDataSource.setTargetDataSources(tenantDataSources);
multiTenantDataSource.setDefaultTargetDataSource(defaultDataSource());
multiTenantDataSource.afterPropertiesSet();
return multiTenantDataSource;
}
public void addTenant(String tenantId, String url, String username, String password) throws SQLException {
DataSource dataSource = DataSourceBuilder.create()
.driverClassName(properties.getDriverClassName())
.url(url)
.username(username)
.password(password)
.build();
// Check that new connection is 'live'. If not - throw exception
try(Connection c = dataSource.getConnection()) {
tenantDataSources.put(tenantId, dataSource);
multiTenantDataSource.afterPropertiesSet();
}
}
public void setCurrentTenant(String tenantId) {
currentTenant.set(tenantId);
}
private DriverManagerDataSource defaultDataSource() {
DriverManagerDataSource defaultDataSource = new DriverManagerDataSource();
defaultDataSource.setDriverClassName("org.h2.Driver");
defaultDataSource.setUrl("jdbc:h2:mem:default");
defaultDataSource.setUsername("default");
defaultDataSource.setPassword("default");
return defaultDataSource;
}
}
Brief explanation:
map tenantDataSources it's our local tenant datasource storage which we put to the setTargetDataSources setter;
DataSourceProperties properties is used to get Database Driver Class name of tenant database from the spring.datasource.driverClassName of the 'application.properties' (for example, org.postgresql.Driver);
method addTenant is used to add a new tenant and its datasource to our local tenant datasource storage. We can do this on the fly - thanks to the method afterPropertiesSet();
method setCurrentTenant(String tenantId) is used to 'switch' onto datasource of the given tenant. We can use this method, for example, in the REST controller when handling a request to work with database. The request should contain the 'tenantId', for example in the X-TenantId header, that we can retrieve and put to this method;
defaultDataSource() is build with in-memory H2 Database to avoid using the default database on the working SQL server.
Note: you must set spring.jpa.hibernate.ddl-auto parameter to none to disable the Hibernate make changes in the database schema. You have to create a schema of tenant databases beforehand.
A full example of this class and more you can find in my repo.
UPDATED
This branch demonstrates an example of using the dedicated database to store tenant DB properties instead of property files (see the question of #MarcoGustavo below).
Latest Spring Boot with JPA and Hibernate: I'm struggling to understand the relationship between transactions, the persistence context and the hibernate session and I can't easily avoid the dreaded no session lazy initialization problem.
I update a set of objects in one transaction and then I want to loop through those objects processing them each in a separate transaction - seems straightforward.
public void control() {
List<> entities = getEntitiesToProcess();
for (Entity entity : entities) {
processEntity(entity.getId());
}
}
#Transactional(value=TxType.REQUIRES_NEW)
public List<Entity> getEntitiesToProcess() {
List<Entity> entities = entityRepository.findAll();
for (Entity entity : entities) {
// Update a few properties
}
return entities;
}
#Transactional(value=TxType.REQUIRES_NEW)
public void processEntity(String id) {
Entity entity = entityRepository.getOne(id);
entity.getLazyInitialisedListOfObjects(); // throws LazyInitializationException: could not initialize proxy - no Session
}
However, I get a problem because (I think) the same hibernate session is being used for both transactions. When I call entityRepository.getOne(id) in the 2nd transaction, I can see in the debugger that I am returned exactly the same object that was returned by findAll() in the 1st transaction without a DB access. If I understand this correctly, it's the hibernate cache doing this? If I then call a method on my object that requires a lazy evaluation, I get a "no session" error. I thought the cache and the session were linked so that's my first confusion.
If I drop all the #Transactional annotations or if I put a #Transactional on the control method it all runs fine, but the database commit isn't done until the control method completes which is obviously not what I want.
So, I have a few questions:
How can I make the hibernate session align with my transaction scope?
What is a good pattern for doing the separation transactions in a loop with JPA and declarative transaction management?
I want to retain the declarative style (i.e. no xml), and don't want to do anything Hibernate specific.
Any help appreciated!
Thanks
Marcus
Spring creates a proxy around your service class, which means #Transactional annotations are only applied when annotated methods are called through the proxy (where you have injected this service).
You are calling getEntitiesToProcess() and processEntity() from within control(), which means those calls are not going through proxy but instead have the transactional scope of the control() method (if you aren't also calling control() from another method in the same class).
In order for #Transactional to apply, you need to do something like this
#Autowired
private ApplicationContext applicationContext;
public void control() {
MyService myService = applicationContext.getBean(MyService.class);
List<> entities = myService.getEntitiesToProcess();
for (Entity entity : entities) {
myService.processEntity(entity.getId());
}
}
I want to use Spring + Hibernate in my web application.
My application is written without Spring.
When "open page" action is called I open Hibernate Session, store it in Http Session and share it between my DAOs. When save action is called I start transaction using my old session.
But now I want to migrate my old DAOs to HibernateDaoSupport based DAOs.
How can I share session in this case? If my DAOs reference to the one SessionFactory in beans.xml will they share the same session?
How can I manage session in this case(open new or use old)?
I have write the following code but I get
org.hibernate.LazyInitializationException: could not initialize proxy - no Session
on page System.out.println(obj.getCategory().getName());
public class CategoryObjectDAOSpringImpl extends HibernateDaoSupport implements CategoryObjectDAO {
#Override
public CategoryObject get(int id) throws Exception {
CategoryObject obj = getHibernateTemplate().get(CategoryObject.class, id);
System.out.println(obj.getId());
System.out.println(obj.getCategory().getName());
for (ObjAttrCommon objAttr : obj.getAttributes()) {
//objAttr.setSession(getSession());
System.out.println(objAttr.getId());
}
return obj;
}
It is strange that if I add
getSessionFactory().openSession();
call at the top I have the same exeption.
I think the same awswer is valid for this post:
If your "unit of work" cannot be automatically per request, I think you can create it manually in your service layer using a transaction. Something like this:
public Object serviceMethod(params) {
TransactionTemplate transactionTemplate = getHibernateTemplate().get(CategoryObject.class, id);
transactionTemplate.execute(new TransactionCallbackWithoutResult() {
public void doInTransactionWithoutResult(TransactionStatus status) {
try {
// call your DAO's to update/delete/... and set values to service
} catch (DAOException e) {
LOGGER.error(e);
throw new ServiceException(e);
}
}
});
}
EDITED: So extrange that the exception appears within a transaction. Make sure that the problem is not in your view. If after you retrieve your entites using HibernateTemplate you access it from the view it will explain the LazyException, because when you access your objects from the view, your Hibernate session will be already closed.
In you're using your models in your JSP/JSF your need to extend your unit of work to cover all request context. You have to include a filter that manages your Hibernate session opening, what is called Open Session In view Pattern, take a look at this.
Copy the filter implementation that appears in this post and include it in your application, it should solve the problem.