Non-primary embedded database schema not being initiliazed - spring

I'm currently working on a Spring Boot (1.5.10.RELEASE) application that uses 2 embedded databases. This two databases are defined equally but one's defined on the application itself and one's defined on a custom autoconfiguration class that's on an imported JAR; yet only the one flagged as #Primary (the one on the application) gets the schema initialized.
This is the current definition for both datasources:
Primary, on the application:
#Configuration
public class DataSourceConfiguration {
#Bean
#ConfigurationProperties(prefix = "first.datasource")
#Primary
public DataSourceProperties firstProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
public DataSource firstDataSource() {
return firstProperties().initializeDataSourceBuilder().build();
}
#Bean
#Primary
public JdbcTemplate firstTemplate() {
return new JdbcTemplate(firstDataSource());
}
}
On the autoconfiguration:
#Configuration
#ConditionalOnProperty(name = "second.datasource.url")
public class SecondDataSourceAutoconfiguration {
#Bean
#ConfigurationProperties(prefix = "second.datasource")
public DataSourceProperties secondProperties() {
return new DataSourceProperties();
}
#Bean
public DataSource secondDataSource() {
return secondProperties().initializeDataSourceBuilder().build();
}
#Bean
public JdbcTemplate secondJdbcTemplate(#Qualifier("secondDataSource") DataSource datasource) {
return new JdbcTemplate(datasource);
}
}
And my application.yml fills the properties:
first:
datasource:
url: jdbc:h2:firstdb;DB_CLOSE_ON_EXIT=FALSE
second:
datasource:
url: jdbc:h2:seconddb
platform: h2
My resources folder contains both a schema.sql that's executed on firstdb and a schema-h2.sql which should get executed on seconddb but does not. I tried playing around with the datasource.schema and datasource.initialize properties, switching the script names and the platform property to the first datasource (in that case, the schema-h2.sql gets executed on firstdb, but nothing on the seconddb) and changing the embedded database provider to HSQLDB; but can't get the schema for the non-primary in-memory DB initialized anyhow.

Related

Spring Cloud Task - specify database config

I have Spring Cloud Task that loads data from SQL Server to Cassandra DB which will be run on Spring Cloud Data Flow.
One of the requirement of Spring Task is to provide relational database to persist metadata like task execution state. But I don't want use either of the above databases for that. Instead, I have to specify third database for persistence. But it seems like Spring Cloud Task flow automatically picks up data source properties of SQL Server from application.properties. How can I specify another db for task state persistence?
My Current properties:
spring.datasource.url=jdbc:sqlserver://iphost;databaseName=dbname
spring.datasource.username=user
spring.datasource.password=password
spring.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.jpa.show-sql=false
#spring.jpa.hibernate.dialect=org.hibernate.dialect.SQLServer2012Dialect
spring.jpa.hibernate.naming.physical-strategy=org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
spring.jpa.hibernate.ddl-auto=none
spring.data.cassandra.contact-points=ip
spring.data.cassandra.port=9042
spring.data.cassandra.username=username
spring.data.cassandra.password=password
spring.data.cassandra.keyspace-name=mykeyspace
spring.data.cassandra.schema-action=CREATE_IF_NOT_EXISTS
Update: 1
I added below code to point to 3rd database as suggested by Michael Minella. Now Spring Task is able to connect to this DB and persist state. But now my batch job source queries are also connecting to this database. Only thing I changed was to add datasource for task.
spring.task.datasource.url=jdbc:postgresql://host:5432/testdb?stringtype=unspecified
spring.task.datasource.username=user
spring.task.datasource.password=passwrod
spring.task.datasource.driverClassName=org.postgresql.Driver
#Configuration
public class DataSourceConfigs {
#Bean(name = "taskDataSource")
#ConfigurationProperties(prefix="spring.task.datasource")
public DataSource getDataSource() {
return DataSourceBuilder.create().build();
}
}
#Configuration
public class DDTaskConfigurer extends DefaultTaskConfigurer{
#Autowired
public DDTaskConfigurer(#Qualifier("taskDataSource") DataSource dataSource) {
super(dataSource);
}
}
Update #2:
#Component
#StepScope
public class MyItemReader extends RepositoryItemReader<Scan> implements InitializingBean{
#Autowired
private ScanRepository repository;
private Integer lastScanIdPulled = null;
public MyItemReader(Integer _lastIdPulled) {
super();
if(_lastIdPulled == null || _lastIdPulled <=0 ){
lastScanIdPulled = 0;
} else {
lastScanIdPulled = _lastIdPulled;
}
}
#PostConstruct
protected void setUpRepo() {
final Map<String, Sort.Direction> sorts = new HashMap<>();
sorts.put("id", Direction.ASC);
this.setRepository(this.repository);
this.setSort(sorts);
this.setMethodName("findByScanGreaterThanId");
List<Object> methodArgs = new ArrayList<Object>();
System.out.println("lastScanIdpulled >>> " + lastScanIdPulled);
if(lastScanIdPulled == null || lastScanIdPulled <=0 ){
lastScanIdPulled = 0;
}
methodArgs.add(lastScanIdPulled);
this.setArguments(methodArgs);
}
}
#Repository
public interface ScanRepository extends JpaRepository<Scan, Integer> {
#Query("...")
Page<Scan> findAllScan(final Pageable pageable);
#Query("...")
Page<Scan> findByScanGreaterThanId(int id, final Pageable pageable);
}
Update #3:
If I add config datasource for Repository, I now get below exception. Before you mention that one of the datasource needs to be declared Primary. I already tried that.
Caused by: java.lang.IllegalStateException: Expected one datasource and found 2
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration$TaskBatchExecutionListenerAutoconfiguration.taskBatchExecutionListener(TaskBatchAutoConfiguration.java:65) ~[spring-cloud-task-batch-1.0.3.RELEASE.jar:1.0.3.RELEASE]
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration$TaskBatchExecutionListenerAutoconfiguration$$EnhancerBySpringCGLIB$$baeae6b9.CGLIB$taskBatchExecutionListener$0(<generated>) ~[spring-cloud-task-batch-1.0.3.RELEASE.jar:1.0.3.RELEASE]
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration$TaskBatchExecutionListenerAutoconfiguration$$EnhancerBySpringCGLIB$$baeae6b9$$FastClassBySpringCGLIB$$5a898c9.invoke(<generated>) ~[spring-cloud-task-batch-1.0.3.RELEASE.jar:1.0.3.RELEASE]
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) ~[spring-core-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358) ~[spring-context-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfigu
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "myEntityManagerFactory",
basePackages = { "com.company.dd.collector.tool" },
transactionManagerRef = "TransactionManager"
)
public class ToolDbConfig {
#Bean(name = "myEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean
myEntityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("ToolDataSource") DataSource dataSource
) {
return builder
.dataSource(dataSource)
.packages("com.company.dd.collector.tool")
.persistenceUnit("tooldatasource")
.build();
}
#Bean(name = "myTransactionManager")
public PlatformTransactionManager transactionManager(
#Qualifier("myEntityManagerFactory") EntityManagerFactory
entityManagerFactory
) {
return new JpaTransactionManager(entityManagerFactory);
}
}
#Configuration
public class DataSourceConfigs {
#Bean(name = "taskDataSource")
#ConfigurationProperties(prefix="spring.task.datasource")
public DataSource getDataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean(name = "ToolDataSource")
#ConfigurationProperties(prefix = "tool.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
}
You need to create a TaskConfigurer to specify the DataSource to be used. You can read about this interface in the documentation here: https://docs.spring.io/spring-cloud-task/1.1.1.RELEASE/reference/htmlsingle/#features-task-configurer
The javadoc can be found here: https://docs.spring.io/spring-cloud-task/docs/current/apidocs/org/springframework/cloud/task/configuration/TaskConfigurer.html
UPDATE 1:
When using more than one DataSource, both Spring Batch and Spring Cloud Task follow the same paradigm in that they both have *Configurer interfaces that need to be used to specify what DataSource to use. For Spring Batch, you use the BatchConfigurer (typically by just extending the DefaultBatchConfigurer) and as noted above, the TaskConfigurer is used in Spring Cloud Task. This is because when there is more than one DataSource, the framework has no way of knowing which one to use.

SpringBoot web services multiple data sources only one working

I have developed two webservices using Spring Boot framework and I have them in the same project. Each webservice use a different DB, say ws1 uses Oracle1 and ws2 uses Oracle2. I have defined a DataBaseConfig with the beans definition but when I run the app, always works one webservice ( and it's always the same ).
DataBaseConfig
#Configuration
public class DataBaseConfig {
#Bean(name = "ora1")
#ConfigurationProperties(prefix="spring.datasource")
public DataSource mysqlDataSource() {
return DataSourceBuilder.create().build();}
#Bean(name = "ora2")
#ConfigurationProperties(prefix="spring.secondDatasource")
public DataSource sqliteDataSource() {
return DataSourceBuilder.create().build();}
#Bean(name = "clients")
#Autowired
#ConfigurationProperties(prefix = "spring.datasource")
#Qualifier("datasource")
public JdbcTemplate slaveJdbcTemplate(DataSource datasource) {
return new JdbcTemplate(datasource); }
#Bean(name = "places")
#Autowired
#Primary
#ConfigurationProperties(prefix = "spring.secondDatasource")
#Qualifier("secondDatasource")
public JdbcTemplate masterJdbcTemplate(DataSource secondDatasource) {
return new JdbcTemplate(secondDatasource);}
}
I have the services definition with the sql statements and the definition
#Service
public class ClientsService {
#Autowired
#Qualifier("clients")
private JdbcTemplate jdbcTemplate;
and the other service
#Service
public class PlacesService {
#Autowired
#Qualifier("places")
private JdbcTemplate jdbcTemplate;
Then in each controller I have de mapping #RequestMapping. When I run the app I have no connection-related errors and if I separate the webservices in 2 projects, each works fine.
You have a few things going wrong here, including some unnecessary annotations. See below, note the location of #Qualifier and the qualifier name:
#Bean(name = "clients")
public JdbcTemplate slaveJdbcTemplate(#Qualifier("ora1") DataSource datasource) {
return new JdbcTemplate(datasource);
}
#Bean(name = "places")
#Primary
public JdbcTemplate masterJdbcTemplate(#Qualifier("ora2") DataSource secondDatasource) {
return new JdbcTemplate(secondDatasource);
}
Instead of resolving by bean name, which is a bad idea IMO because it's not typesafe, why don't you use constructor injection and create the services in the configuration class (ditch #Service annotation). Create the DataSource and JdbcTemplate beans as usual, don't give them any names (the default is the method name), and also create new PlacesService(placesJdbcTemplate()). The result is a lot simpler code.
This is assuming you want both databases active at runtime. If not, use #Profile.

Spring boot multiple datasources

I have a problem with an application using spring boot in 1.5.1 version.
My application need to communicate with 2 databases (Oracle and MySQL)
My application use 2 datasources :
- MySQL datasource
#Configuration
public class OracleDBConfig {
#Bean(name = "oracleDb")
#ConfigurationProperties(prefix = "spring.ds_oracle")
public DataSource oracleDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "oracleJdbcTemplate")
public JdbcTemplate oracleJdbcTemplate(#Qualifier("oracleDb") DataSource dsOracle) {
return new JdbcTemplate(dsOracle);
}
}
Oracle datasource
#Configuration
public class MySQLDBConfig {
#Bean(name = "mysqlDb")
#ConfigurationProperties(prefix = "spring.ds_mysql")
public DataSource mysqlDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "mysqlJdbcTemplate")
public JdbcTemplate mySQLjdbcTemplate(#Qualifier("mysqlDb")DataSource dsMySQL) {
return new JdbcTemplate(dsMySQL);
}
}
I have defined the 2 datasources in my applications.properties by using the prefix.
When I launch the program I have this error :
Parameter 0 of method oracleJdbcTemplate in com.bv.aircraft.config.OracleDBConfig required a single bean, but 2 were found:
- mysqlDb: defined by method 'mysqlDataSource' in class path resource [com/bv/aircraft/config/MySQLDBConfig.class]
- oracleDb: defined by method 'oracleDataSource' in class path resource [com/bv/aircraft/config/OracleDBConfig.class]
I have tried to use #Primary but it is not working when i need to use the other datasource.
Thank you
Add following in your spring boot configuration class
#EnableAutoConfiguration(exclude = {DataSourceTransactionManagerAutoConfiguration.class, DataSourceAutoConfiguration.class})
Sample Usage:
#SpringBootApplication
#EnableAutoConfiguration(exclude = {DataSourceTransactionManagerAutoConfiguration.class, DataSourceAutoConfiguration.class})
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
Either:
separate the template configuration from the datasource configuration and inject in the two datasources with qualifiers into the template configuration, or just call the create datasource method directly, e.g.
public JdbcTemplate oracleJdbcTemplate(oracleDataSource()) DataSource dsOracle) {
return new JdbcTemplate(dsOracle);
}

Spring Configuration Metadata

I am setting up two data sources as shown here at http://docs.spring.io/spring-boot/docs/1.3.0.M2/reference/htmlsingle/#howto-two-datasources using spring boot, but when doing so my application.properties shows warnings that for example x.x.username is an unknown property. This is correct to some extent as javax.sql.DataSource does not contain url, username, password, etc. but the implementation classes do. I have annotation processor set up and it works fine when working with concrete classes.
I notice that org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$NonEmbeddedConfiguration uses both DataSourceProperties and has #ConfigurationProperties annotated on dataSource(). This would probably get rid of my warnings but what is the point of this. Isn't it setting the properties twice this way?
Config:
#Bean
#Primary
#ConfigurationProperties(prefix="datasource.primary")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="datasource.secondary")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
Properties with warnings:
datasource.primary.url=jdbc:...
datasource.primary.username=user
datasource.primary.password=password
datasource.secondary.url=jdbc:...
datasource.secondary.username=user
datasource.secondary.password=password
Since someone bothered to +1 this question I thought I'd post a solution. Note that I think the #ConfigurationProperties on the DataSources themselves are unecessary because they are already set on the DataSourceProperties which is used to build the DataSource, but I left it in there because that's how the Spring team has done it in org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$NonEmbeddedConfiguration. My only guess why would be if your DataSource had additional properties that could be set other than what's exposed in DataSourceProperties, but then you would get warnings in the "Spring Boot application.properties editor" for those properties.
Note that DataSourceBuilder will use Tomcat, HikariCP or Commons DBCP in that order if found on Classpath as DataSource unless you specify something else with dataSourceBuilder.type(Class<? extends DataSource>)
Properties:
datasource.primary.url=jdbc:...
datasource.primary.username=user
datasource.primary.password=password
datasource.secondary.url=jdbc:...
datasource.secondary.username=user
datasource.secondary.password=password
Java Config:
#Bean
#Primary
#ConfigurationProperties(prefix = "datasource.primary")
public DataSourceProperties primaryProps() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties(prefix = "datasource.secondary")
public DataSourceProperties secondaryProps() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties(prefix = "datasource.primary")
public DataSource secondaryDataSource() {
DataSourceProperties props = secondaryProps();
return DataSourceBuilder.create(props.getClassLoader())
.driverClassName(props.getDriverClassName())
.url(props.getUrl())
.username(props.getUsername())
.password(props.getPassword())
.build();
}
#Bean
#ConfigurationProperties(prefix = "datasource.primary")
public DataSource secondaryDataSource() {
DataSourceProperties props = secondaryProps();
return DataSourceBuilder.create(props.getClassLoader())
.driverClassName(props.getDriverClassName())
.url(props.getUrl())
.username(props.getUsername())
.password(props.getPassword())
.build();
}

multiple datasources in Spring Boot application

I'm trying to using two database connections w/in a Spring Boot (v1.2.3) application as described in the docs (http://docs.spring.io/spring-boot/docs/1.2.3.RELEASE/reference/htmlsingle/#howto-two-datasources.
Problem seems to be that the secondary datasource is getting constructed with the properties for the primary datasource.
Can someone point out what I'm missing here?
#SpringBootApplication
class Application {
#Bean
#ConfigurationProperties(prefix="spring.datasource.secondary")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
public JdbcTemplate secondaryJdbcTemplate(DataSource secondaryDataSource) {
return new JdbcTemplate(secondaryDataSource)
}
#Bean
#Primary
#ConfigurationProperties(prefix="spring.datasource.primary")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "primaryJdbcTemplate")
public JdbcTemplate jdbcTemplate(DataSource primaryDataSource) {
return new JdbcTemplate(primaryDataSource)
}
static void main(String[] args) {
SpringApplication.run Application, args
}
}
application.properties:
spring.datasource.primary.url=jdbc:oracle:thin:#example.com:1521:DB1
spring.datasource.primary.username=user1
spring.datasource.primary.password=
spring.datasource.primary.driverClassName=oracle.jdbc.OracleDriver
spring.datasource.secondary.url=jdbc:oracle:thin:#example.com:1521:DB2
spring.datasource.secondary.username=user2
spring.datasource.secondary.password=
spring.datasource.secondary.driverClassName=oracle.jdbc.OracleDriver
Both JdbcTemplate beans will be getting created with the primary DataSource. You can use #Qualifier to have the secondary DataSource injected into the secondary JdbcTemplate. Alternatively, you could call the DataSource methods directly when creating the JdbcTemplate beans.

Resources