I have a RESTful web application which stores data against Oracle and the stack is Spring-Data-Jpa and Hibernate.
We have implemented connection pooling using Oracle UCP, but it does not seem to work. There were 1000s of connections in the DB for our NFT tests.
My config is as shown below
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() throws IllegalStateException, SQLException {
LocalContainerEntityManagerFactoryBean entityManagerFactoryBean = new LocalContainerEntityManagerFactoryBean();
entityManagerFactoryBean.setDataSource(dataSource());
entityManagerFactoryBean.setPersistenceProviderClass(HibernatePersistenceProvider.class);
entityManagerFactoryBean.setPackagesToScan(env.getRequiredProperty(ENTITYMANAGER_PACKAGES_TO_SCAN));
entityManagerFactoryBean.setJpaProperties(hibProperties());
return entityManagerFactoryBean;
}
#Bean
public DataSource dataSource() throws IllegalStateException, SQLException {
PoolDataSource dataSource = PoolDataSourceFactory.getPoolDataSource();
dataSource.setConnectionFactoryClassName(env.getRequiredProperty(DB_CONNECTION_FACTORY_CLASS_NAME));
dataSource.setURL(env.getRequiredProperty(DATABASE_URL));
dataSource.setUser(env.getRequiredProperty(DATABASE_USERNAME));
dataSource.setPassword(env.getRequiredProperty(DATABASE_PASSWORD));
dataSource.setMinPoolSize(Integer.parseInt(env.getRequiredProperty(DATABASE_CONNECTION_MIN_POOL_SIZE)));
dataSource.setMaxPoolSize(Integer.parseInt(env.getRequiredProperty(DATABASE_CONNECTION_MAX_POOL_SIZE)));
dataSource.setInitialPoolSize(Integer.parseInt(env.getRequiredProperty(DATABASE_CONNECTION_INITIAL_POOL_SIZE)));
return dataSource;
}
private Properties hibProperties() {
Properties properties = new Properties();
properties.put(HIBERNATE_DIALECT, env.getRequiredProperty(HIBERNATE_DIALECT));
properties.put(HIBERNATE_SHOW_SQL, env.getRequiredProperty(HIBERNATE_SHOW_SQL));
/*properties.put(HIBERNATE_NAMING_STRATEGY, env.getRequiredProperty(HIBERNATE_NAMING_STRATEGY));*/
return properties;
}
Let me know if there is another config I should be using here, such that a new session is not created all the time.
Thanks
Kris
It would seem that a DataSource gets created for each thread and centralising the datasource in the Jetty config sorted the problem.
jetty and Oracle Connection Pooling
Related
Why Spring ChainedTransactionManager is deprecated? Do spring provides any alternative lib to support multiple transaction managers?
My use-case:- We are building a spring boot application that is connected to two data sources let's say (db1 and db2), which performs insert operation on both databases (db1 and db2). And our requirement is something like this :
insert -> DB1 -> SUCCESSFUL
insert -> DB2 -> ERROR
ROLLBACK DB1
Currently, we are using ChaninedTransactionManager and it is working as expected, but I Can see that lib is deprecated? So, just wanted to make sure is it safe to use that, or does Spring provide any alternate lib which we can use as a replacement for this?
ChainedTransactionManager
The configured instances will start transactions in the order given and commit/rollback in reverse order, which means the PlatformTransactionManager most likely to break the transaction should be the last in the list configured.
If you chained transactions in this order : transaction1, transaction2
transaction1 begin
transaction2 begin
transaction2 commit -> error rollbacks, rollbacks transction1 too
transaction1 commit -> error, only rollbacks transaction1
The case insert -> DB1 -> SUCCESSFUL insert -> DB2 -> ERROR ROLLBACK DB1 is working.
But, if you had insert -> DB1 -> FAIL ROLLBAK DB1 -> DB2 -> SUCCESSFUL, the insert is commited for DB2 and not for DB1. More details in this article.
If your are OK with this, you can copy the class in your project et keep using it : https://github.com/spring-projects/spring-data-commons/issues/2232#issuecomment-1018473289
JtaTransactionManager with Atomikos
To have all transactions rollback if one fails, I changed my conf to use JtaTransactionManager (with spring boot and posgtres).
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jta-atomikos</artifactId>
</dependency>
application.properties
my.datasource.one.unique-resource-name=
my.datasource.one.xa-properties.url=jdbc:postgresql://
my.datasource.one.xa-data-source-class-name=org.postgresql.xa.PGXADataSource
my.datasource.one.xa-properties.user=
my.datasource.one.xa-properties.password=
my.datasource.one.max-pool-size=
my.datasource.one.min-pool-size=
my.datasource.two.unique-resource-name=
my.datasource.two.xa-properties.url=jdbc:postgresql://
my.datasource.two.xa-data-source-class-name=org.postgresql.xa.PGXADataSource
my.datasource.two.xa-properties.user=
my.datasource.two.xa-properties.password=
my.datasource.two.max-pool-size=
my.datasource.two.min-pool-size=
#Bean
#Primary
#ConfigurationProperties("my.datasource.one")
public DataSource dataSourceOne() {
return new AtomikosDataSourceBean();
}
#Bean("entityManagerOne")
#DependsOn("transactionManager")
public LocalContainerEntityManagerFactoryBean entityManagerOne(#Autowired JpaVendorAdapter jpaVendorAdapter) {
HashMap<String, Object> properties = new HashMap<>();
properties.put("hibernate.dialect", "org.hibernate.dialect.PostgreSQLDialect");
properties.put("hibernate.default_schema", "public");
properties.put("hibernate.ddl-auto", "none");
LocalContainerEntityManagerFactoryBean entityManager = new LocalContainerEntityManagerFactoryBean();
entityManager.setJtaDataSource(dataSourceOne());
entityManager.setJpaVendorAdapter(jpaVendorAdapter);
entityManager.setPackagesToScan("");
entityManager.setPersistenceUnitName("");
entityManager.setJpaPropertyMap(properties);
return entityManager;
}
#Bean
#Primary
#ConfigurationProperties("my.datasource.two")
public DataSource dataSourceTwo() {
return new AtomikosDataSourceBean();
}
#Bean("entityManagerTwo")
#DependsOn("transactionManager")
public LocalContainerEntityManagerFactoryBean entityManagerTwo(#Autowired JpaVendorAdapter jpaVendorAdapter) {
HashMap<String, Object> properties = new HashMap<>();
properties.put("hibernate.dialect", "org.hibernate.dialect.PostgreSQLDialect");
properties.put("hibernate.default_schema", "public");
properties.put("hibernate.ddl-auto", "none");
LocalContainerEntityManagerFactoryBean entityManager = new LocalContainerEntityManagerFactoryBean();
entityManager.setJtaDataSource(dataSourceTwo());
entityManager.setJpaVendorAdapter(jpaVendorAdapter);
entityManager.setPackagesToScan("");
entityManager.setPersistenceUnitName("");
entityManager.setJpaPropertyMap(properties);
return entityManager;
}
jta.properties
com.atomikos.icatch.enable_logging=false
com.atomikos.icatch.default_jta_timeout=60000000
com.atomikos.icatch.max_timeout=100000000
com.atomikos.icatch.threaded_2pc=true
#Bean
public JpaVendorAdapter jpaVendorAdapter(#Autowired Environment env) {
HibernateJpaVendorAdapter hibernateJpaVendorAdapter = new HibernateJpaVendorAdapter();
hibernateJpaVendorAdapter.setShowSql(false);
return hibernateJpaVendorAdapter;
}
#Bean
public UserTransaction userTransaction() throws SystemException {
var userTransaction = new UserTransactionImp();
userTransaction.setTransactionTimeout(60000);
return userTransaction;
}
#Bean
public TransactionManager atomikosTransactionManager() throws SystemException {
var userTransactionManager = new UserTransactionManager();
userTransactionManager.setForceShutdown(true);
return userTransactionManager;
}
#Bean
public PlatformTransactionManager transactionManager(#Autowired UserTransaction userTransaction, #Autowired TransactionManager atomikosTransactionManager) throws Throwable {
return new JtaTransactionManager(userTransaction, atomikosTransactionManager);
}
Enable prepared transactions for postgres
postgresql.conf
max_prepared_transactions = 100 # zero disables the feature
Sources that helped me to get this :
https://www.baeldung.com/java-atomikos
http://www.thedevpiece.com/configuring-multiple-datasources-using-springboot-and-atomikos/
https://github.com/YihuaWanglv/spring-boot-jta-atomikos-sample
Because there is an API improvement. From the docs
https://docs.spring.io/spring-data/commons/docs/current/api/org/springframework/data/transaction/ChainedTransactionManager.html
Instead of using ChainedTransactionManager for attaching callbacks to transaction commit (pre commit/post commit), either register a TransactionSynchronization to explicitly follow transaction cleanup with simplified semantics in case of exceptions.
You are free to still use it, just keep in mind that at some point in the time that class will be removed in future versions of Spring and upgrading will not be possible without refactoring of that part.
I have configured a datasource configured on Oracle Weblogic 12.1.3 and trying to lookup this datasource in a Spring MVC application.
// Weblogic configuration
Datasource Name in weblogic : Admin Data Source
JNDI Name : bsh/AdminDS
// Spring MVC Configuration
#Bean
public DataSource getDataSourceUsingJndi() throws NamingException {
return (DataSource) new JndiTemplate().lookup("bsh/AdminDS"); // 1st way
}
#Bean
public LocalSessionFactoryBean sessionFactory() throws NamingException {
LocalSessionFactoryBean sessionFactory = new LocalSessionFactoryBean();
sessionFactory.setDataSource(getDataSourceUsingJndi());
sessionFactory.setHibernateProperties(hibernateProperties());
return sessionFactory;
}
private Properties hibernateProperties() {
Properties properties = new Properties();
properties.put(AvailableSettings.DIALECT, hibernateDialect);
properties.put(AvailableSettings.SHOW_SQL, showSql);
properties.put(AvailableSettings.FORMAT_SQL, formatSql);
return properties;
}
I am getting the below stacktrace:
Caused by: javax.naming.NameNotFoundException: Unable to resolve 'bsh.AdminDS'. Resolved 'bsh'; remaining name 'AdminDS'
at weblogic.jndi.internal.BasicNamingNode.newNameNotFoundException(BasicNamingNode.java:1180)
at weblogic.jndi.internal.BasicNamingNode.lookupHere(BasicNamingNode.java:270)
at weblogic.jndi.internal.ServerNamingNode.lookupHere(ServerNamingNode.java:187)
at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:210)
at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:224)
at weblogic.jndi.internal.WLEventContextImpl.lookup(WLEventContextImpl.java:253)
at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:426)
at javax.naming.InitialContext.lookup(InitialContext.java:417)
at org.springframework.jndi.JndiTemplate.lambda$lookup$0(JndiTemplate.java:156)
at org.springframework.jndi.JndiTemplate.execute(JndiTemplate.java:91)
at org.springframework.jndi.JndiTemplate.lookup(JndiTemplate.java:156)
at com.bsh.config.WebConfig.getDataSourceUsingJndi(WebConfig.java:68)
I tried different ways to lookup - 2nd way
public DataSource getDataSourceUsingJndi() {
JndiDataSourceLookup jndiLookup = new JndiDataSourceLookup();
jndiLookup.setResourceRef(true);
return jndiLookup.getDataSource("bsh/AdminDS");
}
Is there anything wrong with my configuration ?
I need to set two Datasources for my SpringBoot Application. Currently, the single Datasource working solution to deal (successfully) with timeouts on MariaDB server sets the following three parameters in application.properties
# Keep the connection alive if idle for a long time (needed in production)
spring.datasource.testWhileIdle = true
spring.datasource.timeBetweenEvictionRunsMillis = 60000
spring.datasource.validationQuery = SELECT 1
Various examples that I have checked using Java-based, Datasource configuration are in general as follows:
#Primary
#Bean
public DataSource userDataSource() {
DriverManagerDataSource dataSource
= new DriverManagerDataSource();
dataSource.setDriverClassName(
env.getProperty("jdbc.driverClassName"));
dataSource.setUrl(env.getProperty("user.jdbc.url"));
dataSource.setUsername(env.getProperty("jdbc.user"));
dataSource.setPassword(env.getProperty("jdbc.pass"));
return dataSource;
}
The problem is that I don't know how to set testWhileIdle and validationQuery using the Datasource class since there are no respective methods and I don't see in MariaDB documentation any related option that can be passed as part of the JDBC URL.
If you are using Spring Boot with Tomcat then it will use org.apache.commons.dbcp.BasicDataSource not DriverManagerDataSource. Change your dataSource method to return a BasicDataSource which has methods to set testWhileIdle and validationQuery.
#Primary
#Bean
public DataSource userDataSource() {
BasicDataSource dataSource
= new BasicDataSource();
dataSource.setDriverClassName(
env.getProperty("jdbc.driverClassName"));
dataSource.setUrl(env.getProperty("user.jdbc.url"));
dataSource.setUsername(env.getProperty("jdbc.user"));
dataSource.setPassword(env.getProperty("jdbc.pass"));
dataSource.setTestWhileIdle(env.getProperty("jdbc.testWhileIdle"));
dataSource.setValidationQuery(env.getProperty("jdbc.validationQuery"));
return dataSource;
}
...
import org.springframework.boot.autoconfigure.jdbc.DataSourceBuilder;
...
#Primary
#Bean
public DataSource dataSource() {
DataSourceBuilder factory = DataSourceBuilder
.create()
.url(...)
.username(...)
.password(...)
.driverClassName(...);
return factory.build();
}
...
I am pulling data from two different databases using MyBatis 3.3.1 and Spring 4.3. The two configuration classes to scan for mappers look at follows:
#Configuration
#MapperScan(value="com.mapper1.map",
SqlSessionFactoryRef="sqlSessionFactory1")
public class AppConfig {
#Bean
public DataSource getDataSource1() {
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/database1");
dataSource.setUsername("user");
dataSource.setPassword("pw");
return dataSource;
}
#Bean
public DataSourceTransactionManager transactionManager1() {
return new DataSourceTransactionManager(getDataSource1());
}
#Bean
public SqlSessionFactory sqlSessionFactory1() throws Exception {
SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
sessionFactory.setDataSource(getDataSource1());
return sessionFactory.getObject();
}
}
#Configuration
#MapperScan(value="com.mapper2.map",
SqlSessionFactoryRef="sqlSessionFactory2")
public class AppConfig {
#Bean
public DataSource getDataSource2() {
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3307/database2");
dataSource.setUsername("user");
dataSource.setPassword("pw");
return dataSource;
}
#Bean
public DataSourceTransactionManager transactionManager2() {
return new DataSourceTransactionManager(getDataSource2());
}
#Bean
public SqlSessionFactory sqlSessionFactory2() throws Exception {
SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
sessionFactory.setDataSource(getDataSource2());
return sessionFactory.getObject();
}
}
The code deploys fine, but only mappers from data source 1 works. When I try to use a mapper from data source 2, I get a "No table found" exception from my database. The problem is that although I am setting the specific SqlSessionFactory that I want to use in the mapperScan, it ends up using the other SqlSessionFactory for all the mappers. If I comment out the SqlSessionFactory in configuration 1, then Configuration 2 will work.
Note that if I don't use MapperScan, but instead use a MapperScannerConfigurer bean, I am able to correctly retrieve data.
Has anyone else had problems using #MapperScan with multiple data sources?
The only issue I see in your code is SqlSessionFactoryRef should be from lowercase: (sqlSessionFactory). Apart from that everything is fine, this approach works for me.
You can also look at ace-mybatis. It allows to work with multiple datasources configuring only one bean.
I am running HSQL DB as a Im memory using run manger Swing.I have to connect the HSQLDB server from Spring JPA repository using annotations.
My repository class.
#RepositoryRestResource
public interface Vehicle extends JpaRepository<Vehicle , BigInteger>{
public List<Vehicle > findAll(Sort sort);
}
service Method:
#Service
public class LocationService {
#Autowired
VehicletRepository vehicleRepository = null;
/**
* This method is to get all the locations from the repository
*/
public List<Vehicle> getVehicless() {
Order order = new Order(Direction.ASC,"vehicleCode");
Sort sort = new Sort(order);
List<Airport> airports = vehicletRepository .findAll(sort);
System.out.println("inside service");
return vehicles;
}
}
Anyone help to achieve Spring JPA conenction with HSQL DB using annotations.
I assume you dont use Spring boot:
you need #Configuration class -(it basically is new way to configure spring applications in java ) with #EnableJpaRepositories which turn it spring data jpa/ spring data rest for you. You will also have to specify your entity manager, transaction manager and data source beans. Example below:
#Configuration
#EnableJpaRepositories("your.package.with.repositories")
public class DBConfig{
#Bean
public JpaTransactionManager transactionManager() {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(entityManagerFactory().getObject());
return transactionManager;
}
#Bean
public DataSource dataSource() {
BasicDataSource dataSource = new BasicDataSource();
DBConfigurationCommon.configureDB(dataSource, your_jdbc_url_here, db_username_here, db_password_here);
return dataSource;
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
LocalContainerEntityManagerFactoryBean entityManagerFactoryBean = new LocalContainerEntityManagerFactoryBean();
entityManagerFactoryBean.setDataSource(dataSource());
entityManagerFactoryBean.setJpaVendorAdapter(new HibernateJpaVendorAdapter());
//you may need to define new Properties(); here with hibernate dialect and add it to entity manager factory
entityManagerFactoryBean.setPackagesToScan("your_package_with_domain_classes_here");
return entityManagerFactoryBean;
}
}