Spring Boot 2.x multiple datasource and jdbctemplate - spring-boot

I would like to use Multiple Datasources and JdbcTemplates in my Spring Boot 2.1.3 Application.
I tried all solution advices on Stackoverflow and other web sites. But in my application;
when I try to create a Jdbctemplate instance with specified datasource, it sets null for datasources.
I couldn't understand where the problem is? Hikari Pool starts after application ready event. How to set a datasource of jdbcTemplate after initializations of datasources ?
Here is my code examples:
#Bean(name = "iybsDS")
#ConfigurationProperties(prefix = "datasource.iybs")
public DataSource iybsDSdataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "diyihDS")
#ConfigurationProperties(prefix = "datasource.diyih")
public DataSource diyihDSdataSource() {
return DataSourceBuilder.create().build();
}
JdbcTemplate Config:
#Configuration
public class JdbcTemplateConfig {
#Bean(name = "diyihJdbcTemplate")
public JdbcTemplate CreateDiyihJdbcTemplate(#Qualifier("diyihDS") DataSource diyihDS) {
return new JdbcTemplate(diyihDS);
}
}
Trying for print datasource information on console:
#SpringBootApplication
public class WebservicesApplication implements ApplicationRunner {
#Autowired
#Qualifier("diyihDS")
DataSource diyihDS;
public static void main(String[] args) {
SpringApplication.run(WebservicesApplication.class, args);
}
#Override
public void run(ApplicationArguments args) throws Exception {
System.out.println(diyihDS);
}
}
Output is :
2019-03-07 14:27:20.026 INFO 20100 --- [ main] t.g.a.WebservicesApplication : Started WebservicesApplication in 4.724 seconds (JVM running for 5.594)
HikariDataSource (null)
...
2019-03-07 14:27:20.739 INFO 20100 --- [on(4)-127.0.0.1] com.zaxxer.hikari.HikariDataSource : DiyihPool - Starting...
2019-03-07 14:27:20.817 INFO 20100 --- [on(4)-127.0.0.1] com.zaxxer.hikari.HikariDataSource : DiyihPool - Start completed.
When I tried to send a request to web service in my application, I got error because datasource of JdbcTemplate is wrong or null.
I GOT MY MISTAKE:
I was trying just one JdbcTemplate instance. I created all needed jdbcTemplates and set a Primary one of them. Then each service had own JdbcTemplate and Datasource.

Try adding DataSourceProperties Bean and use it:
#Bean
#Primary
#ConfigurationProperties("datasource.diyih")
public DataSourceProperties diyihDataSourceProperties() {
return new DataSourceProperties();
}
#Bean(name="diyihDS")
#Primary
#ConfigurationProperties("spring.datasource.diyih")
public DataSource diyihDataSource() {
return diyihDataSourceProperties().initializeDataSourceBuilder().build();
}

Related

Non-primary embedded database schema not being initiliazed

I'm currently working on a Spring Boot (1.5.10.RELEASE) application that uses 2 embedded databases. This two databases are defined equally but one's defined on the application itself and one's defined on a custom autoconfiguration class that's on an imported JAR; yet only the one flagged as #Primary (the one on the application) gets the schema initialized.
This is the current definition for both datasources:
Primary, on the application:
#Configuration
public class DataSourceConfiguration {
#Bean
#ConfigurationProperties(prefix = "first.datasource")
#Primary
public DataSourceProperties firstProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
public DataSource firstDataSource() {
return firstProperties().initializeDataSourceBuilder().build();
}
#Bean
#Primary
public JdbcTemplate firstTemplate() {
return new JdbcTemplate(firstDataSource());
}
}
On the autoconfiguration:
#Configuration
#ConditionalOnProperty(name = "second.datasource.url")
public class SecondDataSourceAutoconfiguration {
#Bean
#ConfigurationProperties(prefix = "second.datasource")
public DataSourceProperties secondProperties() {
return new DataSourceProperties();
}
#Bean
public DataSource secondDataSource() {
return secondProperties().initializeDataSourceBuilder().build();
}
#Bean
public JdbcTemplate secondJdbcTemplate(#Qualifier("secondDataSource") DataSource datasource) {
return new JdbcTemplate(datasource);
}
}
And my application.yml fills the properties:
first:
datasource:
url: jdbc:h2:firstdb;DB_CLOSE_ON_EXIT=FALSE
second:
datasource:
url: jdbc:h2:seconddb
platform: h2
My resources folder contains both a schema.sql that's executed on firstdb and a schema-h2.sql which should get executed on seconddb but does not. I tried playing around with the datasource.schema and datasource.initialize properties, switching the script names and the platform property to the first datasource (in that case, the schema-h2.sql gets executed on firstdb, but nothing on the seconddb) and changing the embedded database provider to HSQLDB; but can't get the schema for the non-primary in-memory DB initialized anyhow.

Spring Cloud Task - specify database config

I have Spring Cloud Task that loads data from SQL Server to Cassandra DB which will be run on Spring Cloud Data Flow.
One of the requirement of Spring Task is to provide relational database to persist metadata like task execution state. But I don't want use either of the above databases for that. Instead, I have to specify third database for persistence. But it seems like Spring Cloud Task flow automatically picks up data source properties of SQL Server from application.properties. How can I specify another db for task state persistence?
My Current properties:
spring.datasource.url=jdbc:sqlserver://iphost;databaseName=dbname
spring.datasource.username=user
spring.datasource.password=password
spring.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.jpa.show-sql=false
#spring.jpa.hibernate.dialect=org.hibernate.dialect.SQLServer2012Dialect
spring.jpa.hibernate.naming.physical-strategy=org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
spring.jpa.hibernate.ddl-auto=none
spring.data.cassandra.contact-points=ip
spring.data.cassandra.port=9042
spring.data.cassandra.username=username
spring.data.cassandra.password=password
spring.data.cassandra.keyspace-name=mykeyspace
spring.data.cassandra.schema-action=CREATE_IF_NOT_EXISTS
Update: 1
I added below code to point to 3rd database as suggested by Michael Minella. Now Spring Task is able to connect to this DB and persist state. But now my batch job source queries are also connecting to this database. Only thing I changed was to add datasource for task.
spring.task.datasource.url=jdbc:postgresql://host:5432/testdb?stringtype=unspecified
spring.task.datasource.username=user
spring.task.datasource.password=passwrod
spring.task.datasource.driverClassName=org.postgresql.Driver
#Configuration
public class DataSourceConfigs {
#Bean(name = "taskDataSource")
#ConfigurationProperties(prefix="spring.task.datasource")
public DataSource getDataSource() {
return DataSourceBuilder.create().build();
}
}
#Configuration
public class DDTaskConfigurer extends DefaultTaskConfigurer{
#Autowired
public DDTaskConfigurer(#Qualifier("taskDataSource") DataSource dataSource) {
super(dataSource);
}
}
Update #2:
#Component
#StepScope
public class MyItemReader extends RepositoryItemReader<Scan> implements InitializingBean{
#Autowired
private ScanRepository repository;
private Integer lastScanIdPulled = null;
public MyItemReader(Integer _lastIdPulled) {
super();
if(_lastIdPulled == null || _lastIdPulled <=0 ){
lastScanIdPulled = 0;
} else {
lastScanIdPulled = _lastIdPulled;
}
}
#PostConstruct
protected void setUpRepo() {
final Map<String, Sort.Direction> sorts = new HashMap<>();
sorts.put("id", Direction.ASC);
this.setRepository(this.repository);
this.setSort(sorts);
this.setMethodName("findByScanGreaterThanId");
List<Object> methodArgs = new ArrayList<Object>();
System.out.println("lastScanIdpulled >>> " + lastScanIdPulled);
if(lastScanIdPulled == null || lastScanIdPulled <=0 ){
lastScanIdPulled = 0;
}
methodArgs.add(lastScanIdPulled);
this.setArguments(methodArgs);
}
}
#Repository
public interface ScanRepository extends JpaRepository<Scan, Integer> {
#Query("...")
Page<Scan> findAllScan(final Pageable pageable);
#Query("...")
Page<Scan> findByScanGreaterThanId(int id, final Pageable pageable);
}
Update #3:
If I add config datasource for Repository, I now get below exception. Before you mention that one of the datasource needs to be declared Primary. I already tried that.
Caused by: java.lang.IllegalStateException: Expected one datasource and found 2
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration$TaskBatchExecutionListenerAutoconfiguration.taskBatchExecutionListener(TaskBatchAutoConfiguration.java:65) ~[spring-cloud-task-batch-1.0.3.RELEASE.jar:1.0.3.RELEASE]
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration$TaskBatchExecutionListenerAutoconfiguration$$EnhancerBySpringCGLIB$$baeae6b9.CGLIB$taskBatchExecutionListener$0(<generated>) ~[spring-cloud-task-batch-1.0.3.RELEASE.jar:1.0.3.RELEASE]
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration$TaskBatchExecutionListenerAutoconfiguration$$EnhancerBySpringCGLIB$$baeae6b9$$FastClassBySpringCGLIB$$5a898c9.invoke(<generated>) ~[spring-cloud-task-batch-1.0.3.RELEASE.jar:1.0.3.RELEASE]
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) ~[spring-core-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358) ~[spring-context-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfigu
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "myEntityManagerFactory",
basePackages = { "com.company.dd.collector.tool" },
transactionManagerRef = "TransactionManager"
)
public class ToolDbConfig {
#Bean(name = "myEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean
myEntityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("ToolDataSource") DataSource dataSource
) {
return builder
.dataSource(dataSource)
.packages("com.company.dd.collector.tool")
.persistenceUnit("tooldatasource")
.build();
}
#Bean(name = "myTransactionManager")
public PlatformTransactionManager transactionManager(
#Qualifier("myEntityManagerFactory") EntityManagerFactory
entityManagerFactory
) {
return new JpaTransactionManager(entityManagerFactory);
}
}
#Configuration
public class DataSourceConfigs {
#Bean(name = "taskDataSource")
#ConfigurationProperties(prefix="spring.task.datasource")
public DataSource getDataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean(name = "ToolDataSource")
#ConfigurationProperties(prefix = "tool.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
}
You need to create a TaskConfigurer to specify the DataSource to be used. You can read about this interface in the documentation here: https://docs.spring.io/spring-cloud-task/1.1.1.RELEASE/reference/htmlsingle/#features-task-configurer
The javadoc can be found here: https://docs.spring.io/spring-cloud-task/docs/current/apidocs/org/springframework/cloud/task/configuration/TaskConfigurer.html
UPDATE 1:
When using more than one DataSource, both Spring Batch and Spring Cloud Task follow the same paradigm in that they both have *Configurer interfaces that need to be used to specify what DataSource to use. For Spring Batch, you use the BatchConfigurer (typically by just extending the DefaultBatchConfigurer) and as noted above, the TaskConfigurer is used in Spring Cloud Task. This is because when there is more than one DataSource, the framework has no way of knowing which one to use.

Spring boot multiple datasources

I have a problem with an application using spring boot in 1.5.1 version.
My application need to communicate with 2 databases (Oracle and MySQL)
My application use 2 datasources :
- MySQL datasource
#Configuration
public class OracleDBConfig {
#Bean(name = "oracleDb")
#ConfigurationProperties(prefix = "spring.ds_oracle")
public DataSource oracleDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "oracleJdbcTemplate")
public JdbcTemplate oracleJdbcTemplate(#Qualifier("oracleDb") DataSource dsOracle) {
return new JdbcTemplate(dsOracle);
}
}
Oracle datasource
#Configuration
public class MySQLDBConfig {
#Bean(name = "mysqlDb")
#ConfigurationProperties(prefix = "spring.ds_mysql")
public DataSource mysqlDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "mysqlJdbcTemplate")
public JdbcTemplate mySQLjdbcTemplate(#Qualifier("mysqlDb")DataSource dsMySQL) {
return new JdbcTemplate(dsMySQL);
}
}
I have defined the 2 datasources in my applications.properties by using the prefix.
When I launch the program I have this error :
Parameter 0 of method oracleJdbcTemplate in com.bv.aircraft.config.OracleDBConfig required a single bean, but 2 were found:
- mysqlDb: defined by method 'mysqlDataSource' in class path resource [com/bv/aircraft/config/MySQLDBConfig.class]
- oracleDb: defined by method 'oracleDataSource' in class path resource [com/bv/aircraft/config/OracleDBConfig.class]
I have tried to use #Primary but it is not working when i need to use the other datasource.
Thank you
Add following in your spring boot configuration class
#EnableAutoConfiguration(exclude = {DataSourceTransactionManagerAutoConfiguration.class, DataSourceAutoConfiguration.class})
Sample Usage:
#SpringBootApplication
#EnableAutoConfiguration(exclude = {DataSourceTransactionManagerAutoConfiguration.class, DataSourceAutoConfiguration.class})
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
Either:
separate the template configuration from the datasource configuration and inject in the two datasources with qualifiers into the template configuration, or just call the create datasource method directly, e.g.
public JdbcTemplate oracleJdbcTemplate(oracleDataSource()) DataSource dsOracle) {
return new JdbcTemplate(dsOracle);
}

multiple datasources in Spring Boot application

I'm trying to using two database connections w/in a Spring Boot (v1.2.3) application as described in the docs (http://docs.spring.io/spring-boot/docs/1.2.3.RELEASE/reference/htmlsingle/#howto-two-datasources.
Problem seems to be that the secondary datasource is getting constructed with the properties for the primary datasource.
Can someone point out what I'm missing here?
#SpringBootApplication
class Application {
#Bean
#ConfigurationProperties(prefix="spring.datasource.secondary")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
public JdbcTemplate secondaryJdbcTemplate(DataSource secondaryDataSource) {
return new JdbcTemplate(secondaryDataSource)
}
#Bean
#Primary
#ConfigurationProperties(prefix="spring.datasource.primary")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "primaryJdbcTemplate")
public JdbcTemplate jdbcTemplate(DataSource primaryDataSource) {
return new JdbcTemplate(primaryDataSource)
}
static void main(String[] args) {
SpringApplication.run Application, args
}
}
application.properties:
spring.datasource.primary.url=jdbc:oracle:thin:#example.com:1521:DB1
spring.datasource.primary.username=user1
spring.datasource.primary.password=
spring.datasource.primary.driverClassName=oracle.jdbc.OracleDriver
spring.datasource.secondary.url=jdbc:oracle:thin:#example.com:1521:DB2
spring.datasource.secondary.username=user2
spring.datasource.secondary.password=
spring.datasource.secondary.driverClassName=oracle.jdbc.OracleDriver
Both JdbcTemplate beans will be getting created with the primary DataSource. You can use #Qualifier to have the secondary DataSource injected into the secondary JdbcTemplate. Alternatively, you could call the DataSource methods directly when creating the JdbcTemplate beans.

Spring Boot does not seem to pick up Atomikos when used for tests

I am working on a prototype for using Spring Boot in our project. We have a JBoss server in production and I was thinking of running integration tests against Undertow embedded server using an embedded transaction manager like Atomikos, because a persistence.xml exists that I have to reuse. My test app context file has the following lines:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = Application.class)
#WebAppConfiguration
#EnableAutoConfiguration
#IntegrationTest("server.port:0")
#ActiveProfiles("test")
public abstract class TestApplicationContext {
...
}
I have also added a custom test configuration as:
#Configuration
public class TestConfiguration {
#Value("${spring.jpa.hibernate.dialect}")
private String dialectClassName;
#Value("${spring.jpa.hibernate.transaction.manager_lookup_class}")
private String transactionManagerClass;
#Bean
public EmbeddedServletContainerFactory servletContainer() {
return new UndertowEmbeddedServletContainerFactory(9000); // Don't know if this can be avoided using some properties
}
#Bean
#ConfigurationProperties(prefix = DataSourceProperties.PREFIX)
public DataSource dataSource() throws Exception {
return DataSourceBuilder.create().build();
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory(EntityManagerFactoryBuilder builder,
DataSource dataSource) {
LocalContainerEntityManagerFactoryBean entityManagerFactoryBean =
builder.dataSource(dataSource).persistenceUnit("main").build();
Properties additionalProperties = new Properties();
additionalProperties.put("hibernate.dialect", dialectClassName);
additionalProperties.put("hibernate.transaction.manager_lookup_class", transactionManagerClass);
entityManagerFactoryBean.setJpaProperties(additionalProperties);
return entityManagerFactoryBean;
}
#Bean
public PlatformTransactionManager transactionManager() {
// this should not be needed if I have included Atomikos but it seems to pick
// JPA Transaction manager still and fails with the famous NullPointerException at
// CMTTransaction class - because it cannot find a JTA environment
// return new JtaTransactionManager(userTransaction, transactionManager);
}
}
My gradle include for Atomikos is:
testCompile('org.springframework.boot:spring-boot-starter-jta-atomikos')
I am using Spring Boot 1.2.0-RC2.
CAn someone point out what I am doing wrong or how to solve this?
Thanks,
Paddy

Resources