Creating DataSource using DatasSourceBuilder
#Bean
DataSource dataSource() throws SQLException {
DatasSourceBuilder dataSource = DatasSourceBuilder.create();
dataSource.url("");
...
..
return dataSource.build();
}
Creating DataSource using DriverManagerDataSource
#Bean
DriverManagerDataSource dataSource() throws SQLException {
DriverManagerDataSource dataSource = DriverManagerDataSource();
dataSource.setUrl("");
...
..
return dataSource;
}
I am creating jdbc with both of the above method
#Bean
public JdbcTemplate jdbcTemplate()
{
return new jdbcTemplate(dataSource());
}
I am uisng jdbcTemplate like below
void m1()
{
simpleJdbc = new simpleJdbc(jdbcTemplate);
simpleJdbc.execute(procedure)
}
So my question is if i call m1() 50 times repeatedly then how many connections will be created in both cases i.e DriverManagerDataSource and DatasSourceBuilder
The easy answer is
DriverManagerDataSource -> as many connections as you call the method
DataSourceBuilder -> as many connections as specified by the max poolsize property.
However the real answer would be it depends. If you call m1 in a single transaction and call it 50 times then it will open in either way just a single connection. The opened connection is bound to the transaction and reused.
That being said, you shouldn't be using DriverManagerDataSource for a production ready application. It isn't a connection pool and will open connections when needed, opening a connection is slow and it will create an unbounded number of connections (depending on the need) so if you have 100 requests needing a connection it will open 100 connections (probably flooding your DB).
Use DriverManagerDataSource only for tests and demos but not for production applications.
Related
I have a project connecting to multiple databases (Oracle & SQLServer), some of which are not always mandatory nor available when starting the application - especially on a dev environment.
I'm looking for a way to allow the application to start without a database connection right-on. We only require a connection to these DB when doing specific tasks, which use a Staging Database requiring a specific network access not always available on dev.
Here, with our Oracle connection (which is the one being under a specific network) :
We had a configuration that allowed the application to start, but was throwing errors when running the Tests because of the erroneous HikariDataSource cast :
#Configuration
#EnableJpaRepositories(
basePackages = "com.bar.foo.repository.oracle",
entityManagerFactoryRef = "oracleEntityManager",
transactionManagerRef = "oracleTransactionManager"
)
public class OracleConfiguration {
private final Logger log = LoggerFactory.getLogger(this.getClass());
#Inject
private Environment environment;
#Value("${spring.datasource.oracle.ddl-auto:none}")
private String hibernateddlAuto;
#Value("${spring.datasource.oracle.dialect:org.hibernate.dialect.Oracle10gDialect}")
private String hibernateDialect;
#Bean
#Primary
#ConfigurationProperties("spring.datasource.oracle")
public DataSourceProperties oracleDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.hikari")
public DataSource oracleDataSource() {
HikariDataSource ds = (HikariDataSource) oracleDataSourceProperties().initializeDataSourceBuilder().build();
ds.setPoolName(environment.getProperty("spring.datasource.oracle.poolName"));
return ds;
}
#Bean
#Primary
public LocalContainerEntityManagerFactoryBean oracleEntityManager() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setPackagesToScan("com.bar.foo.domain.oracle");
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
HashMap<String, Object> properties = new HashMap<>();
properties.put("hibernate.ddl-auto", hibernateddlAuto);
properties.put("hibernate.dialect", hibernateDialect);
em.setJpaPropertyMap(properties);
em.setDataSource(oracleDataSource());
return em;
}
#Primary
#Bean
public PlatformTransactionManager oracleTransactionManager() {
JpaTransactionManager transactionManager
= new JpaTransactionManager();
transactionManager.setEntityManagerFactory(
oracleEntityManager().getObject());
return transactionManager;
}
}
I moved this configuration to extend HikariConfig as following :
#Configuration
#EnableJpaRepositories(
basePackages = "com.bar.foo.repository.oracle",
entityManagerFactoryRef = "oracleEntityManager",
transactionManagerRef = "oracleTransactionManager"
)
#ConfigurationProperties("spring.datasource.oracle")
public class OracleConfiguration extends HikariConfig{
#Bean
#Primary
public DataSource oracleDataSource() {
return new HikariDataSource(this);
}
#Bean
#Primary
public LocalContainerEntityManagerFactoryBean oracleEntityManager() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setPackagesToScan("com.bar.foo.domain.oracle");
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
HashMap<String, Object> properties = new HashMap<>();
properties.put("hibernate.ddl-auto", hibernateddlAuto);
properties.put("hibernate.dialect", hibernateDialect);
em.setJpaPropertyMap(properties);
em.setDataSource(oracleDataSource());
return em;
}
#Bean
#Primary
public PlatformTransactionManager oracleTransactionManager() {
JpaTransactionManager transactionManager
= new JpaTransactionManager();
transactionManager.setEntityManagerFactory(
oracleEntityManager().getObject());
return transactionManager;
}
Which does not start if the Oracle DB is unavailable.
Using an embedded DB would not suit our use case, as we do not always need this connection, and when we do we need specific data replicated from Production.
Splitting this on multiple microservice/applications is also a no-go, as it would be a huge refacto, and does not really fit our use case (we aggregate data from multiple sources into a final one).
Is there a simple way to allow this ?
HikariCP provides some really nice configuration properties which may suit your needs. Specifically (the first one on that list) initializationFailTimeout:
This property controls whether the pool will "fail fast" if the pool
cannot be seeded with an initial connection successfully...
A value
less than zero will bypass any initial connection attempt, and the
pool will start immediately while trying to obtain connections in the
background. Consequently, later efforts to obtain a connection may
fail.
If you want to solve your problem this way, i.e. by hiding any initialization failures (by setting a negative initializationFailTimeout value), then you'll just have to make sure you have the right logic in case the DB is unaccesible/down when your getting a connection from the pool.
Well, after looking more at the HikariConfig class, I spotted the
initializationFailTimeout which states
* #param initializationFailTimeout the number of milliseconds before the
* pool initialization fails, or 0 to validate connection setup but continue with
* pool start, or less than zero to skip all initialization checks and start the
* pool without delay.
Setting it to zero or below allows the application to start, however it takes way much longer than it used to, and setting it below zero does not prevent it from waiting two connections timeouts.
I ended up also setting the connectionTimeout to the minimum 250ms on our dev configuration file, it seems to be working fine for now.
An alternative solution is to set the following parameter:
spring.jpa.database-platform=org.hibernate.dialect.Oracle10gDialect
Using that setting, I did not experience any slowness in application startup as I did when using the initializationFailTimeout setting.
I need to set two Datasources for my SpringBoot Application. Currently, the single Datasource working solution to deal (successfully) with timeouts on MariaDB server sets the following three parameters in application.properties
# Keep the connection alive if idle for a long time (needed in production)
spring.datasource.testWhileIdle = true
spring.datasource.timeBetweenEvictionRunsMillis = 60000
spring.datasource.validationQuery = SELECT 1
Various examples that I have checked using Java-based, Datasource configuration are in general as follows:
#Primary
#Bean
public DataSource userDataSource() {
DriverManagerDataSource dataSource
= new DriverManagerDataSource();
dataSource.setDriverClassName(
env.getProperty("jdbc.driverClassName"));
dataSource.setUrl(env.getProperty("user.jdbc.url"));
dataSource.setUsername(env.getProperty("jdbc.user"));
dataSource.setPassword(env.getProperty("jdbc.pass"));
return dataSource;
}
The problem is that I don't know how to set testWhileIdle and validationQuery using the Datasource class since there are no respective methods and I don't see in MariaDB documentation any related option that can be passed as part of the JDBC URL.
If you are using Spring Boot with Tomcat then it will use org.apache.commons.dbcp.BasicDataSource not DriverManagerDataSource. Change your dataSource method to return a BasicDataSource which has methods to set testWhileIdle and validationQuery.
#Primary
#Bean
public DataSource userDataSource() {
BasicDataSource dataSource
= new BasicDataSource();
dataSource.setDriverClassName(
env.getProperty("jdbc.driverClassName"));
dataSource.setUrl(env.getProperty("user.jdbc.url"));
dataSource.setUsername(env.getProperty("jdbc.user"));
dataSource.setPassword(env.getProperty("jdbc.pass"));
dataSource.setTestWhileIdle(env.getProperty("jdbc.testWhileIdle"));
dataSource.setValidationQuery(env.getProperty("jdbc.validationQuery"));
return dataSource;
}
...
import org.springframework.boot.autoconfigure.jdbc.DataSourceBuilder;
...
#Primary
#Bean
public DataSource dataSource() {
DataSourceBuilder factory = DataSourceBuilder
.create()
.url(...)
.username(...)
.password(...)
.driverClassName(...);
return factory.build();
}
...
I would like to create data source object on the fly.
I won't be having data base details during compilation time, so I can't put this in either application.properties or hard code it in the code.
To put it simple, assume that I will get the details by calling a web service, by using those details I need to create data source(schema's will be same for all the urls).
#Configuration
public class DataBaseConfig {
#Bean
public DriverManagerDataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
DataBaseDetails details = someWebServiceCall;
dataSource.setUrl(details.getURL());
dataSource.setUsername(details.getUserName());
dataSource.setPassword(details.getPassword());
return dataSource;
}
}
This dataSource() would be called only during application start, is there any way to call to method dataSource for a session/request?
BR,
Kitty
I want to have an optional secondary database for my application such that if the secondary database exists on the deployed environment a repository can retrieve data from it, but if it does not exist, the application will still work fine with just the primary database, and the repository will behave in a way where the application can know this datasource isn't available.
I can definitely think of ways of accomplish this type of thing, but I wonder if there is a more direct configuration type way of achieving this. Or if there is a pattern type way of handling that is preferable.
You can certainly work with multiple databases in a clean way by just implementing 2 database configurations and annotating one of them with #Primary annotation.
#Bean(name = "mainDataSource")
#Primary
public DataSource mainDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(driver);
dataSource.setUrl(url);
dataSource.setUsername(username);
dataSource.setPassword(password);
return dataSource;
}
And for the secondary database, you can create pretty much the same, but don't put the #Primary annotation on it.
#Bean(name = "secondaryDataSource")
public DataSource secondaryDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(driver);
dataSource.setUrl(url);
dataSource.setUsername(username);
dataSource.setPassword(password);
return dataSource;
}
Note that you don't need to put the #Primary annotation just on the DataSource bean, but in any bean you use to connect to your database.
After doing this, all you have to do is annotate your EntityManager or any other connection manager you normally use with the #Qualifier.
#Autowired
#Qualifier("mainEntityManager")
private EntityManager mainDatabase;
#Autowired
#Qualifier("secondEntityManager")
private EntityManager secondDatabase;
In my case, I prefer to make a class to manage this on the backstage.
You can find a complete example of this on my article on linkedin. and check out the example code on GitHub
I have a RESTful web application which stores data against Oracle and the stack is Spring-Data-Jpa and Hibernate.
We have implemented connection pooling using Oracle UCP, but it does not seem to work. There were 1000s of connections in the DB for our NFT tests.
My config is as shown below
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() throws IllegalStateException, SQLException {
LocalContainerEntityManagerFactoryBean entityManagerFactoryBean = new LocalContainerEntityManagerFactoryBean();
entityManagerFactoryBean.setDataSource(dataSource());
entityManagerFactoryBean.setPersistenceProviderClass(HibernatePersistenceProvider.class);
entityManagerFactoryBean.setPackagesToScan(env.getRequiredProperty(ENTITYMANAGER_PACKAGES_TO_SCAN));
entityManagerFactoryBean.setJpaProperties(hibProperties());
return entityManagerFactoryBean;
}
#Bean
public DataSource dataSource() throws IllegalStateException, SQLException {
PoolDataSource dataSource = PoolDataSourceFactory.getPoolDataSource();
dataSource.setConnectionFactoryClassName(env.getRequiredProperty(DB_CONNECTION_FACTORY_CLASS_NAME));
dataSource.setURL(env.getRequiredProperty(DATABASE_URL));
dataSource.setUser(env.getRequiredProperty(DATABASE_USERNAME));
dataSource.setPassword(env.getRequiredProperty(DATABASE_PASSWORD));
dataSource.setMinPoolSize(Integer.parseInt(env.getRequiredProperty(DATABASE_CONNECTION_MIN_POOL_SIZE)));
dataSource.setMaxPoolSize(Integer.parseInt(env.getRequiredProperty(DATABASE_CONNECTION_MAX_POOL_SIZE)));
dataSource.setInitialPoolSize(Integer.parseInt(env.getRequiredProperty(DATABASE_CONNECTION_INITIAL_POOL_SIZE)));
return dataSource;
}
private Properties hibProperties() {
Properties properties = new Properties();
properties.put(HIBERNATE_DIALECT, env.getRequiredProperty(HIBERNATE_DIALECT));
properties.put(HIBERNATE_SHOW_SQL, env.getRequiredProperty(HIBERNATE_SHOW_SQL));
/*properties.put(HIBERNATE_NAMING_STRATEGY, env.getRequiredProperty(HIBERNATE_NAMING_STRATEGY));*/
return properties;
}
Let me know if there is another config I should be using here, such that a new session is not created all the time.
Thanks
Kris
It would seem that a DataSource gets created for each thread and centralising the datasource in the Jetty config sorted the problem.
jetty and Oracle Connection Pooling