Programmatically set SpringBoot Datasource - spring-boot

I need to set two Datasources for my SpringBoot Application. Currently, the single Datasource working solution to deal (successfully) with timeouts on MariaDB server sets the following three parameters in application.properties
# Keep the connection alive if idle for a long time (needed in production)
spring.datasource.testWhileIdle = true
spring.datasource.timeBetweenEvictionRunsMillis = 60000
spring.datasource.validationQuery = SELECT 1
Various examples that I have checked using Java-based, Datasource configuration are in general as follows:
#Primary
#Bean
public DataSource userDataSource() {
DriverManagerDataSource dataSource
= new DriverManagerDataSource();
dataSource.setDriverClassName(
env.getProperty("jdbc.driverClassName"));
dataSource.setUrl(env.getProperty("user.jdbc.url"));
dataSource.setUsername(env.getProperty("jdbc.user"));
dataSource.setPassword(env.getProperty("jdbc.pass"));
return dataSource;
}
The problem is that I don't know how to set testWhileIdle and validationQuery using the Datasource class since there are no respective methods and I don't see in MariaDB documentation any related option that can be passed as part of the JDBC URL.

If you are using Spring Boot with Tomcat then it will use org.apache.commons.dbcp.BasicDataSource not DriverManagerDataSource. Change your dataSource method to return a BasicDataSource which has methods to set testWhileIdle and validationQuery.
#Primary
#Bean
public DataSource userDataSource() {
BasicDataSource dataSource
= new BasicDataSource();
dataSource.setDriverClassName(
env.getProperty("jdbc.driverClassName"));
dataSource.setUrl(env.getProperty("user.jdbc.url"));
dataSource.setUsername(env.getProperty("jdbc.user"));
dataSource.setPassword(env.getProperty("jdbc.pass"));
dataSource.setTestWhileIdle(env.getProperty("jdbc.testWhileIdle"));
dataSource.setValidationQuery(env.getProperty("jdbc.validationQuery"));
return dataSource;
}

...
import org.springframework.boot.autoconfigure.jdbc.DataSourceBuilder;
...
#Primary
#Bean
public DataSource dataSource() {
DataSourceBuilder factory = DataSourceBuilder
.create()
.url(...)
.username(...)
.password(...)
.driverClassName(...);
return factory.build();
}
...

Related

HibernateException: hibernate.dialect not set, but I did set it in application.properties

I am working on a Spring Boot application that uses a number of databases. I define datasources, I define entitymanagers, and I keep getting this error
org.hibernate.HibernateException: Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set
at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.determineDialect(DialectFactoryImpl.java:100)
at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.buildDialect(DialectFactoryImpl.java:54)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:137)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:94)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263)
This is the #Configuration file for one database:
#Configuration
public class CustomerDbConfig {
#Bean
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "customerEntityManager")
#PersistenceContext(unitName = "customerEntityManager")
public LocalContainerEntityManagerFactoryBean customerEntityManagerFactory(
EntityManagerFactoryBuilder builder, DataSource dataSource) {
return builder.dataSource(dataSource).packages("com.example.report_service").build();
}
#Bean
public JpaTransactionManager transactionManager(EntityManagerFactory emf) {
JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(emf);
return jpaTransactionManager;
}
#Bean
public EntityManagerFactoryBuilder entityManagerFactoryBuilder() {
return new EntityManagerFactoryBuilder(new HibernateJpaVendorAdapter(), new HashMap<>(), null);
}
}
and these are the application.properties
spring.datasource.url=jdbc:postgresql://dv1.example.com:5432/customer
spring.datasource.username=${DATABASECUSTOMERUSERNAME:customer}
spring.datasource.password=${DATABASECUSTOMERPASSWORD:customer}
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.maximum-pool-size=${MAX_POOL_SIZE:20}
spring.datasource.tomcat.validation-query=SELECT 1
spring.datasource.tomcat.validation-query-timeout=5000
spring.datasource.tomcat.test-on-borrow=true
spring.datasource.tomcat.test-on-connect=true
spring.transaction.default-timeout=${TRANSACTION_TIMEOUT:3600}
spring.jpa.properties.hibernate.proc.param_null_passing=true
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.PostgreSQLDialect
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation= true
spring.jpa.open-in-view=false
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.format_sql=true
spring.jpa.properties.hibernate.show_sql=false
spring.jpa.properties.hibernate.use_sql_comments=true
spring.jpa.properties.hibernate.type=trace
spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration
When I explicitely set a persistenceUnit name in the entitymanager, the error changes, it complains about not finding a PU with the name I specified.
I limited my application to having only one database, trying to figure out what's happening. When I remove the configuration file there's no issues. What makes this application special is that it uses native queries only, it doesn't have #Entity classes because the database is updated by other applictions. When I remove the .packages(..) part from the code, it complaines like
No persistence units parsed from {classpath*:META-INF/persistence.xml}
I suspect the issue is related to the fact that we don't have classes that need to be scanned.
Edit:
Following M.Deinum's answer, I replaced the config by
#Configuration
#EnableTransactionManagement
public class CustomerDbConfig {
#ConfigurationProperties(prefix = "customer.datasource")
#Bean(name = "customer-datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "customer-jdbcTemplate")
public JdbcTemplate primaryJdbcTemplate(#Qualifier("customer-datasource") DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
}
and I use the four templates in the daos. Works like a charm.
When using JPA you need entities. Using the JPA classes to just execute SQL queries is overkill. Just use a plain Jdbctemplate or NamedParameterJdbcTemplate for that.

Will spring Boot datasource properties work if we configure datasource programmatically in Spring Boot

Will Spring Boot datasource properties work if we configure datasource programmatically?
The following properties worked only when I fetch DB configuration from application.properties. If I configure datasource programmatically the following properties are not working.
spring.datasource.tomcat.initial-size=10
spring.datasource.tomcat.max-active=10
spring.datasource.tomcat.max-idle=5
spring.datasource.tomcat.min-idle=5
I used the following code to configure datasource programmatically
#ConfigurationProperties(prefix = "spring.datasource")
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder.create().username(userName).password(password).url(url).driverClassName(driverName)
.build();
}
To make it work programmatically I used the following code snippet.But I'm not convinced. I feel it is not a cleaner solution. I have to read at least 20 properties from application.properties and add it to PoolProperties.
#ConfigurationProperties(prefix = "spring.datasource")
#Bean
#Primary
public DataSource dataSource() {
PoolProperties poolProperties = new PoolProperties();
poolProperties.setUrl(url);
poolProperties.setDriverClassName(driverName);
poolProperties.setUsername(userName);
poolProperties.setPassword(password);
poolProperties.setTestWhileIdle(false);
poolProperties.setTestOnBorrow(true);
poolProperties.setValidationQuery("SELECT 1 FROM DUAL");
poolProperties.setTestOnReturn(false);
poolProperties.setValidationInterval(30000);
poolProperties.setTimeBetweenEvictionRunsMillis(30000);
poolProperties.setInitialSize(10);
poolProperties.setMaxActive(10);
poolProperties.setMaxIdle(5);
poolProperties.setMinIdle(5);
poolProperties.setMaxWait(10000);
poolProperties.setRemoveAbandonedTimeout(60);
poolProperties.setMinEvictableIdleTimeMillis(30000);
poolProperties.setLogAbandoned(true);
poolProperties.setRemoveAbandoned(true);
DataSource datasource = new DataSource(); // import org.apache.tomcat.jdbc.pool.DataSource;
datasource.setPoolProperties(poolProperties);
return datasource;
}
It there way we can make the following default Spring Boot properties work?
spring.datasource.tomcat.initial-size=10
spring.datasource.tomcat.max-active=10
spring.datasource.tomcat.max-idle=5
spring.datasource.tomcat.min-idle=5
remember to give tomcat from properties for db in property files like this.
# Oracle DB - "foo"
spring.datasource.tomcat.url=jdbc:oracle:thin:#//db-server-foo:1521/FOO
spring.datasource.tomcat.username=fooadmin
spring.datasource.tomcat.password=foo123
spring.datasource.tomcat.initial-size=10
spring.datasource.tomcat.max-active=10
spring.datasource.tomcat.max-idle=5
spring.datasource.tomcat.min-idle=5
then configure datasource like this.
/**
* Auto-configured DataSource
*/
#ConfigurationProperties(prefix = "spring.datasource.tomcat")
#Bean
#Primary
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}

Spring batch boot Multiple datasources Multiple schemas

I have a spring batch job using spring boot which has 2 datasources. Each datasource again has 2 schemas each. I need to specify default schema for both the datasources. I know of property spring.jpa.properties.hibernate.default_schema which i am using to specify default schema for one datasource. Is there a way to specify default schema for another schema?
Currently, to specify default schema for the other datasource , i am using alter session query to switch schema as required. I am trying to get rid of this alter session query from my java code. Any suggestions on it is greatly appreciated.
edit 1: Both are ORACLE databases
If you use multiple datasources, then you probably has a #Configuration class for each datasource. In this case you can set additional properties to the entityManager. This configuration is needed:
props.put("spring.datasource.schema", "test");
Full example
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(entityManagerFactoryRef = "testEntityManagerFactory", transactionManagerRef = "testTransactionManager",
basePackages = {"com.test.repository"})
public class TestDbConfig {
#Bean(name = "testDataSource")
#ConfigurationProperties(prefix = "test.datasource")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "testEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(EntityManagerFactoryBuilder builder, #Qualifier("testDataSource") DataSource dataSource) {
return builder.dataSource(dataSource).packages("com.test.model").persistenceUnit("test").properties(jpaProperties()).build();
}
private Map<String, Object> jpaProperties() {
Map<String, Object> props = new HashMap<>();
props.put("hibernate.physical_naming_strategy", SpringPhysicalNamingStrategy.class.getName());
props.put("hibernate.implicit_naming_strategy", SpringImplicitNamingStrategy.class.getName());
props.put("spring.datasource.schema", "test");
return props;
}
#Bean(name = "testTransactionManager")
public PlatformTransactionManager transactionManager(#Qualifier("testEntityManagerFactory") EntityManagerFactory entityManagerFactory) {
return new JpaTransactionManager(entityManagerFactory);
}
}

MyBatis+Spring MapperScan with Mulitple Data Sources

I am pulling data from two different databases using MyBatis 3.3.1 and Spring 4.3. The two configuration classes to scan for mappers look at follows:
#Configuration
#MapperScan(value="com.mapper1.map",
SqlSessionFactoryRef="sqlSessionFactory1")
public class AppConfig {
#Bean
public DataSource getDataSource1() {
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/database1");
dataSource.setUsername("user");
dataSource.setPassword("pw");
return dataSource;
}
#Bean
public DataSourceTransactionManager transactionManager1() {
return new DataSourceTransactionManager(getDataSource1());
}
#Bean
public SqlSessionFactory sqlSessionFactory1() throws Exception {
SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
sessionFactory.setDataSource(getDataSource1());
return sessionFactory.getObject();
}
}
#Configuration
#MapperScan(value="com.mapper2.map",
SqlSessionFactoryRef="sqlSessionFactory2")
public class AppConfig {
#Bean
public DataSource getDataSource2() {
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3307/database2");
dataSource.setUsername("user");
dataSource.setPassword("pw");
return dataSource;
}
#Bean
public DataSourceTransactionManager transactionManager2() {
return new DataSourceTransactionManager(getDataSource2());
}
#Bean
public SqlSessionFactory sqlSessionFactory2() throws Exception {
SqlSessionFactoryBean sessionFactory = new SqlSessionFactoryBean();
sessionFactory.setDataSource(getDataSource2());
return sessionFactory.getObject();
}
}
The code deploys fine, but only mappers from data source 1 works. When I try to use a mapper from data source 2, I get a "No table found" exception from my database. The problem is that although I am setting the specific SqlSessionFactory that I want to use in the mapperScan, it ends up using the other SqlSessionFactory for all the mappers. If I comment out the SqlSessionFactory in configuration 1, then Configuration 2 will work.
Note that if I don't use MapperScan, but instead use a MapperScannerConfigurer bean, I am able to correctly retrieve data.
Has anyone else had problems using #MapperScan with multiple data sources?
The only issue I see in your code is SqlSessionFactoryRef should be from lowercase: (sqlSessionFactory). Apart from that everything is fine, this approach works for me.
You can also look at ace-mybatis. It allows to work with multiple datasources configuring only one bean.

Spring Configuration Metadata

I am setting up two data sources as shown here at http://docs.spring.io/spring-boot/docs/1.3.0.M2/reference/htmlsingle/#howto-two-datasources using spring boot, but when doing so my application.properties shows warnings that for example x.x.username is an unknown property. This is correct to some extent as javax.sql.DataSource does not contain url, username, password, etc. but the implementation classes do. I have annotation processor set up and it works fine when working with concrete classes.
I notice that org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$NonEmbeddedConfiguration uses both DataSourceProperties and has #ConfigurationProperties annotated on dataSource(). This would probably get rid of my warnings but what is the point of this. Isn't it setting the properties twice this way?
Config:
#Bean
#Primary
#ConfigurationProperties(prefix="datasource.primary")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="datasource.secondary")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
Properties with warnings:
datasource.primary.url=jdbc:...
datasource.primary.username=user
datasource.primary.password=password
datasource.secondary.url=jdbc:...
datasource.secondary.username=user
datasource.secondary.password=password
Since someone bothered to +1 this question I thought I'd post a solution. Note that I think the #ConfigurationProperties on the DataSources themselves are unecessary because they are already set on the DataSourceProperties which is used to build the DataSource, but I left it in there because that's how the Spring team has done it in org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$NonEmbeddedConfiguration. My only guess why would be if your DataSource had additional properties that could be set other than what's exposed in DataSourceProperties, but then you would get warnings in the "Spring Boot application.properties editor" for those properties.
Note that DataSourceBuilder will use Tomcat, HikariCP or Commons DBCP in that order if found on Classpath as DataSource unless you specify something else with dataSourceBuilder.type(Class<? extends DataSource>)
Properties:
datasource.primary.url=jdbc:...
datasource.primary.username=user
datasource.primary.password=password
datasource.secondary.url=jdbc:...
datasource.secondary.username=user
datasource.secondary.password=password
Java Config:
#Bean
#Primary
#ConfigurationProperties(prefix = "datasource.primary")
public DataSourceProperties primaryProps() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties(prefix = "datasource.secondary")
public DataSourceProperties secondaryProps() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties(prefix = "datasource.primary")
public DataSource secondaryDataSource() {
DataSourceProperties props = secondaryProps();
return DataSourceBuilder.create(props.getClassLoader())
.driverClassName(props.getDriverClassName())
.url(props.getUrl())
.username(props.getUsername())
.password(props.getPassword())
.build();
}
#Bean
#ConfigurationProperties(prefix = "datasource.primary")
public DataSource secondaryDataSource() {
DataSourceProperties props = secondaryProps();
return DataSourceBuilder.create(props.getClassLoader())
.driverClassName(props.getDriverClassName())
.url(props.getUrl())
.username(props.getUsername())
.password(props.getPassword())
.build();
}

Resources