I have a problem, I need to create two DataSource but I cannot use #Primary because in other module I also have two DataSource and then in third module I include both modules, so there are two primary modules.
I wanted to use #Qualifier but it does not work.
#Bean(name = "secondDataSourceProperties")
#ConfigurationProperties("second")
public DataSourceProperties secondDataSourceProperties() {
return new DataSourceProperties();
}
#Bean(name = "secondDataSource")
#ConfigurationProperties("second.configuration")
public DataSource secondDataSource(#Qualifier("secondDataSourceProperties") DataSourceProperties dataSourceProperties) {
HikariDataSource ds = dataSourceProperties.initializeDataSourceBuilder().type(HikariDataSource.class).build();
ds.setConnectionTestQuery("SELECT 1");
ds.setConnectionInitSql("SELECT 1");
ds.setPoolName("jdbc/second");
return ds;
}
#Bean(name = "secondTransactionManager")
public DataSourceTransactionManager secondDataSourceTransactionManager(#Qualifier("secondDataSource") DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
#Bean
#ConfigurationProperties("first")
public DataSourceProperties firstDataSourceProperties() {
return new DataSourceProperties();
}
#Bean(name = "firstDataSource")
#ConfigurationProperties("first.configuration")
public DataSource firstDataSource(#Qualifier("firstDataSourceProperties") DataSourceProperties dataSourceProperties) {
HikariDataSource ds = dataSourceProperties.initializeDataSourceBuilder().type(HikariDataSource.class).build();
ds.setConnectionTestQuery("SELECT 1");
ds.setConnectionInitSql("SELECT 1");
ds.setPoolName("jdbc/first");
return ds;
}
#Bean(name = "firstTransactionManager")
public DataSourceTransactionManager firstDataSourceTransactionManager(#Qualifier("firstDataSource") DataSource dataSource) {
return new DataSourceTransactionManager(dataSource);
}
Error I'm getting is:
No qualifying bean of type
'org.springframework.boot.autoconfigure.jdbc.DataSourceProperties'
available: expected single matching bean but found 5:
secondDataSourceProperties,firstDataSourceProperties,3DataSourceProperties,4DataSourceProperties,spring.datasource-org.springframework.boot.autoconfigure.jdbc.DataSourceProperties
I use DataSource like that in code:
#Autowired
#Qualifier("first")
private DataSource dataSource;
and
#Transactional(value = "firstTransactionManager")
How about just doing the following? Without #Qualifier mess? The name of the method for #Bean is by default used as the "qualifier" of the bean.
#Bean
#ConfigurationProperties("first")
public DataSourceProperties firstDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("second")
public DataSourceProperties secondDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
public DataSource firstDataSource(DataSourceProperties firstDataSourceProperties) {
HikariDataSource ds = firstDataSourceProperties.initializeDataSourceBuilder().type(HikariDataSource.class).build();
ds.setConnectionTestQuery("SELECT 1");
ds.setConnectionInitSql("SELECT 1");
ds.setPoolName("jdbc/first");
return ds;
}
#Bean
public DataSource secondDataSource(DataSourceProperties secondDataSourceProperties) {
HikariDataSource ds = secondDataSourceProperties.initializeDataSourceBuilder().type(HikariDataSource.class).build();
ds.setConnectionTestQuery("SELECT 1");
ds.setConnectionInitSql("SELECT 1");
ds.setPoolName("jdbc/second");
return ds;
}
#Bean
public DataSourceTransactionManager firstTransactionManager(DataSource firstDataSource) {
return new DataSourceTransactionManager(firstDataSource);
}
#Bean
public DataSourceTransactionManager secondTransactionManager(DataSource secondDataSource) {
return new DataSourceTransactionManager(secondDataSource);
}
You can just autowire with this name only;
#Autowire
private DataSource firstDateSource;
and
#Transactional("secondTransactionManager")
Helped adding
#EnableAutoConfiguration(exclude={DataSourceAutoConfiguration.class})
Or if you use #SpringBootApplication
#SpringBootApplication(scanBasePackages={"a.b.c"}, exclude = {DataSourceAutoConfiguration.class})
Related
I have implemented the application using Spring RoutingDataSource.
Spring -> DS1,
DS2
Based on the logged in URL I am changing the Data Source. it is working fine.
Coming to the quartz, I am unable to change the data source dynamically. Always jobs are getting scheduled on default data source.
#Configuration
public class SchedulerConfig {
#Autowired
private DataSource dataSource;
#Autowired
private QuartzProperties quartzProperties;
#Bean
public JobFactory jobFactory(ApplicationContext applicationContext) {
AutowiringSpringBeanJobFactory jobFactory = new AutowiringSpringBeanJobFactory();
jobFactory.setApplicationContext(applicationContext);
return jobFactory;
}
#Bean
public SchedulerFactoryBean schedulerFactoryBean(JobFactory jobFactory) {
Properties properties = new Properties();
properties.putAll(quartzProperties.getProperties());
SchedulerFactoryBean factory = new SchedulerFactoryBean();
factory.setJobFactory(jobFactory);
factory.setDataSource(dataSource);
factory.setGlobalJobListeners(jobListener());
factory.setQuartzProperties(properties);
return factory;
}
#Bean
public JobListenerSupport jobListener() {
return new JobListener();
}
}
Data Source Routing Configuration::
#Component
public class DataSourceRouting extends AbstractRoutingDataSource {
private DataSourceOneConfig dataSourceOneConfig;
private DataSourceTwoConfig dataSourceTwoConfig;
private DataSourceContextHolder dataSourceContextHolder;
public DataSourceRouting(DataSourceContextHolder dataSourceContextHolder, DataSourceOneConfig dataSourceOneConfig,
DataSourceTwoConfig dataSourceTwoConfig) {
this.dataSourceOneConfig = dataSourceOneConfig;
this.dataSourceTwoConfig = dataSourceTwoConfig;
this.dataSourceContextHolder = dataSourceContextHolder;
Map<Object, Object> dataSourceMap = new HashMap<>();
dataSourceMap.put(DataSourceEnum.tenant1, dataSourceOneDataSource());
dataSourceMap.put(DataSourceEnum.tenant2, dataSourceTwoDataSource());
this.setTargetDataSources(dataSourceMap);
this.setDefaultTargetDataSource(dataSourceTwoDataSource());
}
#Override
protected Object determineCurrentLookupKey() {
return dataSourceContextHolder.getBranchContext();
}
public DataSource dataSourceOneDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setUrl(dataSourceOneConfig.getUrl());
dataSource.setUsername(dataSourceOneConfig.getUsername());
dataSource.setPassword(dataSourceOneConfig.getPassword());
return dataSource;
}
public DataSource dataSourceTwoDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setUrl(dataSourceTwoConfig.getUrl());
dataSource.setUsername(dataSourceTwoConfig.getUsername());
dataSource.setPassword(dataSourceTwoConfig.getPassword());
return dataSource;
}
}
Data Soruce Config ::
#RequiredArgsConstructor
#DependsOn("dataSourceRouting")
public class DataSourceConfig {
private final DataSourceRouting dataSourceRouting;
#Bean
#Primary
public DataSource dataSource() {
return dataSourceRouting;
}
#Bean(name = "entityManager")
public LocalContainerEntityManagerFactoryBean entityManagerFactoryBean(EntityManagerFactoryBuilder builder) {
return builder.dataSource(dataSource()).packages("com.model.entity").build();
}
#Bean(name = "transcationManager")
public JpaTransactionManager transactionManager(
#Autowired #Qualifier("entityManager") LocalContainerEntityManagerFactoryBean entityManagerFactoryBean) {
return new JpaTransactionManager(entityManagerFactoryBean.getObject());
}
}
I try to set multiple datasource, but just always only can get primary datasource.
source Primary:
#Configuration
#EnableJdbcRepositories("com.xxx.repository.abc")
public class AbcDataSourceConfig {
#Bean
#Primary
#ConfigurationProperties(prefix="spring.abc")
public DataSourceProperties abcDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
public DataSource abcDataSource(#Qualifier("abcDataSourceProperties") DataSourcePropertiesproperties) {
log.debug("abc properties : {}", properties.getDriverClassName());
return properties.initializeDataSourceBuilder().build();
}
#Bean
#Primary
public NamedParameterJdbcOperations abcJdbcOperations(#Qualifier("abcDataSource") DataSource abcDs) {
return new NamedParameterJdbcTemplate(abcDs);
}
}
source 2:
#Configuration
#EnableJdbcRepositories(jdbcOperationsRef = "cbaJdbcOperations", basePackages = "com.xxx.repository.cba")
public class CbaDataSourceConfig {
#Bean
#ConfigurationProperties(prefix="spring.cba")
public DataSourceProperties cbaDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
public DataSource cbaDataSource(#Qualifier("cbaDataSourceProperties") DataSourceProperties properties) {
return properties.initializeDataSourceBuilder().build();
}
#Bean
public NamedParameterJdbcOperations cbaJdbcOperations(#Qualifier("cbaDataSource") DataSource cbaDs) {
return new NamedParameterJdbcTemplate(cbaDs);
}
}
when I try to select cba's data, it show is error: "Caused by: java.sql.SQLSyntaxErrorException: Table 'abc.cba_data' doesn't exist".
but cba_data is in the cba datasource.
how to setting is correct?
My application.properties
spring.datasource.url=jdbc:h2:mem:testdb
spring.datasource.username=${env.H2_USER}
spring.datasource.password=${env.H2_PASS}
spring.token-datasource.url=jdbc:mysql://${env.MYSQL_HOST}/${env.MYSQL_DB}
spring.token-datasource.username=${env.MYSQL_USER}
spring.token-datasource.password=${env.MYSQL_PASS}
My Configuration.java code snippet
#Configuration
public class DataSourceConfiguration {
#Bean
#Primary
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource regularDataSource() {
DataSource dataSource = DataSourceBuilder.create().build();
return dataSource;
}
#Bean
#ConfigurationProperties(prefix = "spring.token-datasource")
public DataSource tokenDataSource() {
DataSource dataSource = DataSourceBuilder.create().build();
return dataSource;
}
}
However, if I put a break point at each of return dataSource;, I see the DataSource object not populated with the correponding property values, e.g. jdbcUrl would be null.
Any help?
Thanks!
Followed https://docs.spring.io/spring-boot/docs/2.4.2/reference/html/howto.html#howto-two-datasources and crafted following code which works:
#Configuration
public class DataSourceConfiguration {
#Bean
#Primary
#ConfigurationProperties("spring.datasource")
public DataSourceProperties regularDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.configuration")
public HikariDataSource regularDataSource() {
HikariDataSource dataSource = regularDataSourceProperties().initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
return dataSource;
}
#Bean
#ConfigurationProperties("spring.token-datasource")
public DataSourceProperties tokenDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("spring.token-datasource.configuration")
public DataSource tokenDataSource() {
DataSource dataSource = tokenDataSourceProperties().initializeDataSourceBuilder().build();
return dataSource;
}
}
It's interesting that the original code snippet does NOT work, while it's still the approach recommened by many Java tutorials and stackoverflow posts. For example,
https://www.baeldung.com/spring-data-jpa-multiple-databases
Spring Boot Configure and Use Two DataSources
Im new to spring framework and I Want to migrate data from One DB to another using Spring boot n Batch
Im trying to read from an Mysql db with an item reader and write it to an Oracle db using an item writer of the same job.
Im able to read the data from the Mysql db but unable to write it to Oracle Db as the writer is trying it write it in Mysql Db itself.
Im not sure why the connection is not switching to Oracle db.
Please help me over here what im doing wrong.
Application properties
server.port=8082
spring.batch.job.enabled= false
spring.datasource.url = jdbc:mysql://localhost:3306/projectdb1
spring.datasource.username = root
spring.datasource.password = Mech_2015
spring.datasource.driverClassName = com.mysql.jdbc.Driver
#second db2 ...
db2.datasource.url = jdbc:oracle:thin:#localhost:1521:xe
db2.datasource.username = system
db2.datasource.password = Mech_2015
db2.datasource.driverClassName = oracle.jdbc.driver.OracleDriver
Entity Managers
package com.techprimers.springbatchexample1.entity.manger;
#Configuration
#EnableJpaRepositories(entityManagerFactoryRef = "radiusEntityManager", transactionManagerRef = "radiusTransactionManager", basePackages = "com.techprimers.springbatchexample1.repository.userrepo")
public class RadiusConfig {
private final PersistenceUnitManager persistenceUnitManager;
public RadiusConfig(ObjectProvider<PersistenceUnitManager> persistenceUnitManager) {
this.persistenceUnitManager = persistenceUnitManager.getIfAvailable();
}
#Bean
#ConfigurationProperties("spring.jpa")
public JpaProperties radiusJpaProperties() {
return new JpaProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource")
public DataSourceProperties radiusDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties(prefix = "spring.datasource.properties")
public DataSource radiusDataSource() {
return (DataSource) radiusDataSourceProperties().initializeDataSourceBuilder().type(DataSource.class).build();
}
#Bean(name = "radiusEntityManager")
public LocalContainerEntityManagerFactoryBean radiusEntityManager(JpaProperties radiusJpaProperties) {
EntityManagerFactoryBuilder builder = createEntityManagerFactoryBuilder(radiusJpaProperties);
return builder.dataSource(radiusDataSource()).packages(User.class).persistenceUnit("userDs").build();
}
#Bean
public JpaTransactionManager radiusTransactionManager(EntityManagerFactory radiusEntityManager) {
return new JpaTransactionManager(radiusEntityManager);
}
private EntityManagerFactoryBuilder createEntityManagerFactoryBuilder(JpaProperties radiusJpaProperties) {
JpaVendorAdapter jpaVendorAdapter = createJpaVendorAdapter(radiusJpaProperties);
return new EntityManagerFactoryBuilder(jpaVendorAdapter, radiusJpaProperties.getProperties(),
this.persistenceUnitManager);
}
private JpaVendorAdapter createJpaVendorAdapter(JpaProperties jpaProperties) {
AbstractJpaVendorAdapter adapter = new HibernateJpaVendorAdapter();
adapter.setShowSql(jpaProperties.isShowSql());
adapter.setDatabase(jpaProperties.getDatabase());
adapter.setDatabasePlatform(jpaProperties.getDatabasePlatform());
adapter.setGenerateDdl(jpaProperties.isGenerateDdl());
return adapter;
}
}
package com.techprimers.springbatchexample1.entity.manger;
#Configuration
#EnableJpaRepositories(entityManagerFactoryRef = "esbEntityManager", transactionManagerRef = "esbDetailsTransactionManager", basePackages = "com.techprimers.springbatchexample1.repository.userdetails")
public class EsbConfig {
private final PersistenceUnitManager persistenceUnitManager;
public EsbConfig(ObjectProvider<PersistenceUnitManager> persistenceUnitManager) {
this.persistenceUnitManager = persistenceUnitManager.getIfAvailable();
}
#Bean
#ConfigurationProperties("db2.jpa")
public JpaProperties esbJpaProperties() {
return new JpaProperties();
}
#Bean
#ConfigurationProperties("db2.datasource")
public DataSourceProperties esbDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties(prefix = "db2.datasource.properties")
public DataSource esbDataSource() {
return (DataSource) esbDataSourceProperties().initializeDataSourceBuilder().type(DataSource.class).build();
}
#Bean(name = "esbEntityManager")
public LocalContainerEntityManagerFactoryBean esbEntityManager(JpaProperties esbJpaProperties) {
EntityManagerFactoryBuilder builder = createEntityManagerFactoryBuilder(esbJpaProperties);
return builder.dataSource(esbDataSource()).packages(UserDetails.class).persistenceUnit("userDetailDs").build();
}
#Bean
public JpaTransactionManager esbTransactionManager(EntityManagerFactory esbEntityManager) {
return new JpaTransactionManager(esbEntityManager);
}
private EntityManagerFactoryBuilder createEntityManagerFactoryBuilder(JpaProperties esbJpaProperties) {
JpaVendorAdapter jpaVendorAdapter = createJpaVendorAdapter(esbJpaProperties);
return new EntityManagerFactoryBuilder(jpaVendorAdapter, esbJpaProperties.getProperties(),
this.persistenceUnitManager);
}
private JpaVendorAdapter createJpaVendorAdapter(JpaProperties jpaProperties) {
AbstractJpaVendorAdapter adapter = new HibernateJpaVendorAdapter();
adapter.setShowSql(jpaProperties.isShowSql());
adapter.setDatabase(jpaProperties.getDatabase());
adapter.setDatabasePlatform(jpaProperties.getDatabasePlatform());
adapter.setGenerateDdl(jpaProperties.isGenerateDdl());
return adapter;
}
}
Batch Config
package com.techprimers.springbatchexample1.config;
#Configuration
#EnableBatchProcessing
public class SpringBatchConfig {
private static final Logger LOGGER = LoggerFactory.getLogger(SpringBatchConfig.class);
#Bean
#Primary
#Qualifier("radiusDatasource")
#ConfigurationProperties(prefix = "spring.datasource")
DataSource mysqlDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix = "db2.datasource")
#Qualifier("esbDatasource")
DataSource oracleDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
public InspectionProcessor processor() {
return new InspectionProcessor();
}
#Bean
public JdbcCursorItemReader<User> reader() {
JdbcCursorItemReader<User> cursorItemReader = new JdbcCursorItemReader<>();
cursorItemReader.setDataSource(mysqlDataSource());
cursorItemReader.setSql("SELECT ID,CNAME,SID,CREATEDDATE,COMPLETEDDATE FROM INSPECTION");
cursorItemReader.setRowMapper(new InspectionDetailRowmapper());
return cursorItemReader;
}
private static final String QUERY_INSERT_STUDENT = "INSERT "
+ "INTO inspect(id,cname,completeddate,createddate,lastupdateddate,sid) " + "VALUES (?,?,?,?,?,?)";
#Bean
ItemWriter<UserDetails> databaseItemWriter(DataSource dataSource, NamedParameterJdbcTemplate jdbcTemplate) {
LOGGER.info("Starting writer");
JdbcBatchItemWriter<UserDetails> databaseItemWriter = new JdbcBatchItemWriter<>();
databaseItemWriter.setDataSource(oracleDataSource());
databaseItemWriter.setJdbcTemplate(jdbcTemplate);
LOGGER.info(" writer");
databaseItemWriter.setSql(QUERY_INSERT_STUDENT)
ItemPreparedStatementSetter<UserDetails> valueSetter = new UserDetailsPreparedStatementSetter();
databaseItemWriter.setItemPreparedStatementSetter(valueSetter);
return databaseItemWriter;
}
#Bean
Step dataMigrationStep(ItemReader<User> reader, ItemProcessor<User, UserDetails> processor,
ItemWriter<UserDetails> databsaeItemWriter, StepBuilderFactory stepBuilderFactory) {
return stepBuilderFactory.get("dataMigrationStep").<User, UserDetails>chunk(5).reader(reader)
.processor(processor).writer(databsaeItemWriter).build();
}
#Bean
Job dataMigrationJob(JobBuilderFactory jobBuilderFactory, #Qualifier("dataMigrationStep") Step dataMigrationStep) {
return jobBuilderFactory.get("csvFileToDatabaseJob").incrementer(new RunIdIncrementer()).flow(dataMigrationStep)
.end().build();
}
}
Your writer does not inject properly the database where to write your data
this
#Bean
ItemWriter<UserDetails> databaseItemWriter(DataSource dataSource, NamedParameterJdbcTemplate jdbcTemplate)
instead of
#Bean
ItemWriter<UserDetails> databaseItemWriter(#Qualifier("esbDatasource") DataSource dataSource, NamedParameterJdbcTemplate jdbcTemplate)
I get ORA-00942: table or view does not exist at runtime when the code reaches this line : session.merge(role);
So how to show the query text executed during this merge ?
update :
here is how I configured the app :
#Configuration
#ComponentScan("com.ambre.pta")
#EnableTransactionManagement
#PropertySources({
#PropertySource("classpath:jdbc.properties"),
#PropertySource("classpath:fr/global.properties"),
#PropertySource("classpath:fr/main.properties"),
#PropertySource("classpath:fr/admin.properties"),
#PropertySource("classpath:fr/referentiel.properties"),
#PropertySource("classpath:fr/departement.properties"),
#PropertySource("classpath:fr/exercice.properties"),
#PropertySource("classpath:fr/defi.properties")
})
public class ApplicationContextConfig {
#Autowired
private Environment env;
#Bean
public static PropertySourcesPlaceholderConfigurer properties() {
return new PropertySourcesPlaceholderConfigurer();
}
#Bean(name = "viewResolver")
public InternalResourceViewResolver getViewResolver() {
InternalResourceViewResolver viewResolver = new InternalResourceViewResolver();
viewResolver.setPrefix("/WEB-INF/views/");
viewResolver.setSuffix(".jsp");
return viewResolver;
}
#Bean(name = "dataSource")
public DataSource getDataSource() {
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName("oracle.jdbc.driver.OracleDriver");
dataSource.setUrl("jdbc:oracle:thin:#"+env.getProperty("jdbc.server")+":"+env.getProperty("jdbc.port")+":"+env.getProperty("jdbc.instance"));
dataSource.setUsername(env.getProperty("jdbc.login.default"));
dataSource.setPassword(env.getProperty("jdbc.pwd.default"));
return dataSource;
}
#Autowired
#Bean(name = "sessionFactory")
public SessionFactory getSessionFactory(DataSource dataSource) {
LocalSessionFactoryBuilder sessionBuilder = new LocalSessionFactoryBuilder(dataSource);
sessionBuilder.scanPackages("com.ambre.pta.model");
return sessionBuilder.buildSessionFactory();
}
#Autowired
#Bean(name = "transactionManager")
public HibernateTransactionManager getTransactionManager(SessionFactory sessionFactory) {
HibernateTransactionManager transactionManager = new HibernateTransactionManager(sessionFactory);
return transactionManager;
}
#Autowired
#Bean(name = "utilisateurDao")
public UtilisateurDAO getUtilisateurDao(SessionFactory sessionFactory) {
return new UtilisateurDAOImpl(sessionFactory);
}
#Autowired
#Bean(name = "menuDao")
public MenuDAO getMenuDao(SessionFactory sessionFactory) {
return new MenuDAOImpl(sessionFactory);
}
...
}
Please add the below tag to your hibernate config
<property name="show_sql">true</property>
look into
hibernate config file
application.yml file, there might be hibernate configuration and show-sql: true option.