This is my config file
#Configuration
#ComponentScan
public class Config {
#Bean
public DataSource datasource() {
return new EmbeddedDatabaseBuilder().setName("MyDB").setType(EmbeddedDatabaseType.H2).addScript("schema.sql").build();
}
#Bean
public JdbcOperations jdbcTemplate(DataSource ds) {
return new JdbcTemplate(ds);
}
}
After I run the program, I can not find the "MyDB" database.
I know it is an in-memory database. How to make it embedded so that when I close the program the data on the database persist and I can find "MyDB" on the project folder.
#Bean
public DataSource h2DataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName(driver);
dataSource.setUrl(url);
dataSource.setUsername(username);
dataSource.setPassword(password);
return dataSource;
}
driver: org.h2.Driver
url: jdbc:h2:file:${java.io.tmpdir}/database/db_name;AUTO_SERVER=TRUE
username: user
password: pass
Related
I have 2 JDBC Datasources defined in a Spring Boot application utilizing used in a Spring Batch job. However, after autowiring the datasources, only one gets used. The one used is the one annotated #Primary. If I place the annotation on the other JDBC datasource that gets used instead. In a nutshell only one of the JDBC datasources ever gets used. I use Lombok in some places but I'm unsure if that is playing a part.
Here are the datasources:
application.yml
symphony:
datasource:
driver-class-name: oracle.jdbc.OracleDriver
url: ...
type: com.zaxxer.hikari.HikariDataSource
username: <USR>
password: <PWD>
jndi-name: false
repo:
datasource:
driver-class-name: org.h2.Driver
url: jdbc:h2:mem:db;DB_CLOSE_DELAY=-1
username: sa
password: sa
jndi-name: false
Here is the first datasource:
#Configuration
public class RepoDbConfig {
#Bean
#ConfigurationProperties("repo.datasource")
public DataSourceProperties repoDataProperties() {
return new DataSourceProperties();
}
#Bean(name = "repoDataSource")
public DataSource dataSourcerepo() {
DataSource dataSource = repoDataProperties().initializeDataSourceBuilder().type(BasicDataSource.class)
.build();
return dataSource;
}
#Bean(name = "repoJdbcTemplate")
public JdbcTemplate repoJdbcTemplate(DataSource repoDataSource) {
return new JdbcTemplate(repoDataSource);
}
}
Here is the second datasource:
#Configuration
public class SymphonyDbConfig {
#Primary
#Bean
#ConfigurationProperties("symphony.datasource")
public DataSourceProperties symphonyDataSourceProperties() {
return new DataSourceProperties();
}
#Primary
#Bean(name = "symphonyDataSource")
public DataSource dataSourcesymphony() {
HikariDataSource dataSource = symphonyDataSourceProperties().initializeDataSourceBuilder().type(HikariDataSource.class)
.build();
return dataSource;
}
#Primary
#Bean(name = "symphonyJdbcTemplate")
public JdbcTemplate symphonyJdbcTemplate(DataSource symphonyDataSource) {
return new JdbcTemplate(symphonyDataSource);
}
}
The JobRepository beans are configured like this:
#Configuration
#RequiredArgsConstructor
public class JobRepositoryConfig {
final #Qualifier("repoDataSource")
DataSource repoDataSource;
#Bean("repoTransactionManager")
AbstractPlatformTransactionManager repoTransactionManager() {
return new ResourcelessTransactionManager();
}
#Bean("repoJobRepository")
public JobRepository repoJobRepository(DataSource repoDataSource) throws Exception {
JobRepositoryFactoryBean jobRepositoryFactoryBean = new JobRepositoryFactoryBean();
jobRepositoryFactoryBean.setDataSource(repoDataSource);
jobRepositoryFactoryBean.setTransactionManager(repoTransactionManager());
jobRepositoryFactoryBean.setDatabaseType(DatabaseType.H2.getProductName());
return jobRepositoryFactoryBean.getObject();
}
#Bean("repoAppJobLauncher")
public JobLauncher careLocationAppJobLauncher(JobRepository repoJobRepository) {
SimpleJobLauncher simpleJobLauncher = new SimpleJobLauncher();
simpleJobLauncher.setJobRepository(repoJobRepository);
return simpleJobLauncher;
}
}
Finally the Batch Job beans used for the Job are configured here: The only part not shown is the launching of the job. All the required beans used are shown here:
#Configuration
#EnableBatchProcessing
#EnableScheduling
#RequiredArgsConstructor
#Slf4j
public class CellBatchConfig {
private final JobBuilderFactory jobBuilderFactory;
#Qualifier("repoAppJobLauncher")
private final JobLauncher repoAppJobLauncher;
private final StepBuilderFactory stepBuilderFactory;
#Value("${chunk-size}")
private int chunkSize;
#Qualifier("symphonyDataSource")
final DataSource symphonyDataSource;
#Qualifier("repoDataSource")
final DataSource symphonyDataSource;
#Bean
public JdbcPagingItemReader<CenterDto> cellItemReader(PagingQueryProvider pagingQueryProvider) {
return new JdbcPagingItemReaderBuilder<CenterDto>()
.name("cellItemReader")
.dataSource(symphonyDataSource)
.queryProvider(pagingQueryProvider)
.pageSize(chunkSize)
.rowMapper(new CellRowMapper())
.build();
}
#Bean
public PagingQueryProvider pagingQueryProvider() {
OraclePagingQueryProvider pagingQueryProvider = new OraclePagingQueryProvider();
final Map<String, Order> sortKeys = new HashMap<>();
sortKeys.put("ID", Order.ASCENDING);
pagingQueryProvider.setSortKeys(sortKeys);
pagingQueryProvider.setSelectClause(" ID, CELL_NO, MAT_VO ");
pagingQueryProvider.setFromClause(" from pvc.cells");
return pagingQueryProvider;
}
.......
}
The error results from only one of the datasources being used. That results that being used to query the Spring Batch job repository resulting in it failing: Here is the key portion of the stacktrace. It is trying to use the oracle datasource to query for the JobRespository resources and fails as a result:
Caused by: org.springframework.jdbc.BadSqlGrammarException:
PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID, JOB_NAME from
BATCH_JOB_INSTANCE
where JOB_NAME = ? and JOB_KEY = ?]; nested exception is
java.sql.SQLSyntaxErrorException: ORA-00942: table or view does not exist
In the class JobRepositoryConfig:
In the bean:
#Bean("symphonyJobRepository")
public JobRepository symphonyJobRepository(DataSource repoDataSource) throws Exception {
JobRepositoryFactoryBean jobRepositoryFactoryBean = new JobRepositoryFactoryBean();
jobRepositoryFactoryBean.setDataSource(repoDataSource);
jobRepositoryFactoryBean.setTransactionManager(repoTransactionManager());
jobRepositoryFactoryBean.setDatabaseType(DatabaseType.H2.getProductName());
return jobRepositoryFactoryBean.getObject();
}
You didn't use the variable:
final #Qualifier("repoDataSource") DataSource repoDataSource;
So Spring uses a DataSource object which is annotated with #Primary annotation
I fixed it by making one bean the primary and also adding the qualifiers on the specific beans which had been missing, an omission on my part. For example, here I added the #Qualifier:
#Primary
#Bean(name = "symphonyDataSource")
#Qualifier("symphonyDataSource") // This was missing
public DataSource dataSourcesymphony() {
HikariDataSource dataSource = symphonyDataSourceProperties().initializeDataSourceBuilder().type(HikariDataSource.class)
.build();
return dataSource;
}
My application.properties
spring.datasource.url=jdbc:h2:mem:testdb
spring.datasource.username=${env.H2_USER}
spring.datasource.password=${env.H2_PASS}
spring.token-datasource.url=jdbc:mysql://${env.MYSQL_HOST}/${env.MYSQL_DB}
spring.token-datasource.username=${env.MYSQL_USER}
spring.token-datasource.password=${env.MYSQL_PASS}
My Configuration.java code snippet
#Configuration
public class DataSourceConfiguration {
#Bean
#Primary
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource regularDataSource() {
DataSource dataSource = DataSourceBuilder.create().build();
return dataSource;
}
#Bean
#ConfigurationProperties(prefix = "spring.token-datasource")
public DataSource tokenDataSource() {
DataSource dataSource = DataSourceBuilder.create().build();
return dataSource;
}
}
However, if I put a break point at each of return dataSource;, I see the DataSource object not populated with the correponding property values, e.g. jdbcUrl would be null.
Any help?
Thanks!
Followed https://docs.spring.io/spring-boot/docs/2.4.2/reference/html/howto.html#howto-two-datasources and crafted following code which works:
#Configuration
public class DataSourceConfiguration {
#Bean
#Primary
#ConfigurationProperties("spring.datasource")
public DataSourceProperties regularDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource.configuration")
public HikariDataSource regularDataSource() {
HikariDataSource dataSource = regularDataSourceProperties().initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
return dataSource;
}
#Bean
#ConfigurationProperties("spring.token-datasource")
public DataSourceProperties tokenDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("spring.token-datasource.configuration")
public DataSource tokenDataSource() {
DataSource dataSource = tokenDataSourceProperties().initializeDataSourceBuilder().build();
return dataSource;
}
}
It's interesting that the original code snippet does NOT work, while it's still the approach recommened by many Java tutorials and stackoverflow posts. For example,
https://www.baeldung.com/spring-data-jpa-multiple-databases
Spring Boot Configure and Use Two DataSources
I'm working with multiple data sources (Oracle and SQL-Server) in spring boot rest application. In this application, I have more than 25+ end-points exist to process client requests. But when one of the databases is down like Oracle or SQL-server is not available for some reason, my application is unable to start the server.
Looked couple examples on google and stack overflow but they're different what I'm looking for...
package com.foobar;
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories
(
entityManagerFactoryRef = "entityManagerFactory",
basePackages = { "com.foobar.foo.repo" }
)
public class FooDbConfig
{
#Primary
#Bean(name = "dataSource")
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean(name = "entityManagerFactory")
public LocalContainerEntityManagerFactoryBean
entityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("dataSource") DataSource dataSource
) {
return builder
.dataSource(dataSource)
.packages("com.foobar.foo.domain")
.persistenceUnit("foo")
.build();
}
#Primary
#Bean(name = "transactionManager")
public PlatformTransactionManager transactionManager(
#Qualifier("entityManagerFactory") EntityManagerFactory
entityManagerFactory)
{
return new JpaTransactionManager(entityManagerFactory);
}
}
same configuration for 2nd data-source but with different properties.
I'm using below example as a base code reference to implement my requirements
Example link
I'm looking for a solution if one DB server is available out of N application should start and process client requests and whenever 2nd DB server is available then it connects automatically and processes other endpoints requests
I've created a solution recently for multitenancy with datasources and using liquibase, but if not use the liquibase, just remove that works too!
Example of application.yml
spring:
dataSources:
- tenantId: db1
url: jdbc:postgresql://localhost:5432/db1
username: postgres
password: 123456
driver-class-name: org.postgresql.Driver
liquibase:
enabled: true
default-schema: public
change-log: classpath:db/master/changelog/db.changelog-master.yaml
- tenantId: db2
url: jdbc:postgresql://localhost:5432/db2
username: postgres
password: 123456
driver-class-name: org.postgresql.Driver
- tenantId: db3
url: jdbc:postgresql://localhost:5432/db3
username: postgres
password: 123456
driver-class-name: org.postgresql.Driver
DataSourceConfiguration
#Configuration
#EnableTransactionManagement
#EntityScan(basePackages = { "br.com.dijalmasilva.springbootmultitenancyliquibase" })
#EnableJpaRepositories(basePackages = { "br.com.dijalmasilva.springbootmultitenancyliquibase" })
public class DataSourceConfiguration {
#Bean(name = "dataSources")
#Primary
public Map<Object, Object> getDataSources(DataSourceProperties dataSourceProperties) {
return dataSourceProperties.getDataSources().stream().map(dataSourceProperty -> {
DataSource dataSource = DataSourceBuilder.create()
.url(dataSourceProperty.getUrl())
.username(dataSourceProperty.getUsername())
.password(dataSourceProperty.getPassword())
.driverClassName(dataSourceProperty.getDriverClassName())
.build();
return new TenantIdDataSource(dataSourceProperty.getTenantId(), dataSource);
}).collect(Collectors.toMap(TenantIdDataSource::getTenantId, TenantIdDataSource::getDataSource));
}
#Bean(name = "tenantRoutingDataSource")
#DependsOn("dataSources")
public DataSource dataSource(Map<Object, Object> dataSources) {
AbstractRoutingDataSource tenantRoutingDataSource = new TenantRoutingDataSource();
tenantRoutingDataSource.setTargetDataSources(dataSources);
tenantRoutingDataSource.setDefaultTargetDataSource(dataSources.get("db1"));
tenantRoutingDataSource.afterPropertiesSet();
return tenantRoutingDataSource;
}
#Data
#AllArgsConstructor
private class TenantIdDataSource {
private Object tenantId;
private Object dataSource;
}
}
TenantRoutingDataSource
public class TenantRoutingDataSource extends AbstractRoutingDataSource {
#Override
protected Object determineCurrentLookupKey() {
return TenantContext.getCurrentTenant();
}
}
DataSourceProperties
#Data
#Component
#ConfigurationProperties(prefix = "spring")
public class DataSourceProperties {
private List<DataSourceProperty> dataSources = new ArrayList<>();
}
DataSourceProperty
#Data
public class DataSourceProperty {
private String tenantId;
private String url;
private String username;
private String password;
private String driverClassName;
private LiquibaseProperties liquibase;
}
See the complete code, maybe help you!
Link of project: https://github.com/dijalmasilva/spring-boot-multitenancy-datasource-liquibase
In my application i need to integrate two datasource but when i integrate the second database using JdbcTemplate then the previous one not working, instead all table created in the second datasource
#1 DataSource
#Configuration
#Profile("mariadb4j")
#EnableAutoConfiguration(exclude={DataSourceAutoConfiguration.class})
public class EmbeddedMariaDBConfig {
private static final Logger L =
LoggerFactory.getLogger(EmbeddedMariaDBConfig.class);
private static final String DB_SERVICE = "dbServiceBean";
#Bean(name = {DB_SERVICE})
MariaDB4jSpringService mariaDB4jSpringService() {
L.info("Initializing MariaDB4j service");
return new MariaDB4jSpringService();
}
#Bean(name = "adminDataSource")
#Primary
#DependsOn(DB_SERVICE)
DataSource dataSource(MariaDB4jSpringService mdb, DataSourceProperties dataSourceProperties) throws ManagedProcessException {
String dbName = dataSourceProperties.getName();
L.debug("Embedded MariaDB datasource properties from spring: [{}]", dataSourceProperties);
mdb.getDB().createDB(dbName);
if(L.isDebugEnabled()) {
DBConfigurationBuilder ecfg = mdb.getConfiguration();
L.debug("JDBC URL for embedded MariaDB as reported by driver: [{}]", ecfg.getURL(dbName));
L.debug("JDBC URL from spring config: [{}]", dataSourceProperties.getUrl());
L.debug("JDBC Username: [{}]", dataSourceProperties.getUsername());
L.debug("JDBC Password: [{}]", dataSourceProperties.getPassword());
}
return DataSourceBuilder
.create()
.username(dataSourceProperties.getUsername())
.password(dataSourceProperties.getPassword())
.url(dataSourceProperties.getUrl())
.driverClassName(dataSourceProperties.getDriverClassName())
.build();
}
}
Yaml Configuration
spring:
profiles: mariadb4j
datasource:
username: root
password: password
driver-class-name: com.mysql.jdbc.Driver
url: jdbc:mysql://localhost:3306/t1
mariaDB4j:
dataDir: /tmp/mariadb
port: 3900
2 DataSource
#Configuration
public class ExoDBConfig {
#Bean(name="user")
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/t2");
dataSource.setUsername("root");
dataSource.setPassword("password");
return dataSource;
}
#Bean
public JdbcTemplate jdbcTemplate(#Qualifier("user")DataSource dataSource)
{
return new JdbcTemplate(dataSource);
}
}
When i use only one DataSource the 1st one then it's working fine and tables are creating in t1 database but when i integrate the 2nd DataSource then it points to second DataSource means all the tables is creating in the t2 database.
Just Create DataSource object in the JdbcTemplate itself, working for me, and you can create separate properties file for t2 database
#Bean
public JdbcTemplate jdbcTemplate() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("jdbc:mysql://localhost:3306/t2");
dataSource.setUsername("user");
dataSource.setPassword("password");
return new JdbcTemplate(dataSource);
}
The scripts in schema.sql gets executes but scripts from data.sql are not executing,
not sure what I am missing?
I am using Spring Boot with two data source my data base configuration is as follows
#PropertySource({ "classpath:application.properties" })
#Configuration
#EnableJpaRepositories(
basePackages = "com.projectx.mysql",
entityManagerFactoryRef = "userEntityManager",
transactionManagerRef = "userTransactionManager"
)
public class DataBaseConfig {
#Autowired
Environment env;
#Bean
#Primary
public LocalContainerEntityManagerFactoryBean userEntityManager() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(primaryDataSource());
em.setPackagesToScan(new String[] { "com.projectx.mysql" });
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
HashMap<String, Object> properties = new HashMap<String, Object>();
properties.put("hibernate.hbm2ddl.auto", env.getProperty("spring.jpa.hibernate.ddl-auto_mysql"));
properties.put("hibernate.dialect", env.getProperty("spring.jpa.properties.hibernate.dialect_mysql"));
properties.put("hibernate.show_sql", env.getProperty("spring.jpa.show-sql"));
em.setJpaPropertyMap(properties);
return em;
}
#Bean
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean
public PlatformTransactionManager userTransactionManager() {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(userEntityManager().getObject());
return transactionManager;
}
}
and .properties file configuration as follows
spring.datasource.initialize=true
spring.datasource.url=jdbc:mysql://localhost/test
spring.datasource.username=root
spring.datasource.password=
spring.datasource.driverClassName=com.mysql.jdbc.Driver
spring.jpa.show-sql: true
spring.jpa.hibernate.ddl-auto_mysql=update
spring.jpa.properties.hibernate.dialect_mysql=org.hibernate.dialect.MySQL5Dialect
For those of you who stumble upon this question in a SpringBoot 2.1+ world.
First, what I think is the full class name of the important class (the object of the "publishEvent"... "org.springframework.boot.autoconfigure.jdbc.DataSourceInitializedEvent"
For future readers, this class seems to have disappeared between these two versions :
Below does exist:
https://docs.spring.io/spring-boot/docs/2.0.0.M3/api/org/springframework/boot/autoconfigure/jdbc/DataSourceInitializedEvent.html
Below no longer exists:
https://docs.spring.io/spring-boot/docs/2.1.0.M1/api/org/springframework/boot/autoconfigure/jdbc/DataSourceInitializedEvent.html
Here is how I "coded up" the seed data ("data.sql") .. when I had a Java-Config heavy class.
import org.springframework.jdbc.datasource.init.DataSourceInitializer;
import org.springframework.jdbc.datasource.init.ResourceDatabasePopulator;
#Bean
public DataSource getDataSource() {
//not shown
}
#Bean
public DataSourceInitializer dataSourceInitializer(DataSource ds) {
ResourceDatabasePopulator resourceDatabasePopulator = new ResourceDatabasePopulator();
resourceDatabasePopulator.addScript(new ClassPathResource("/data.sql"));
DataSourceInitializer dataSourceInitializer = new DataSourceInitializer();
dataSourceInitializer.setDataSource(ds);
dataSourceInitializer.setDatabasePopulator(resourceDatabasePopulator);
return dataSourceInitializer;
}
Debug tip. You may want to use a file name that is NOT a magic file name ("data.sql" is a magic name) to purposely avoid spring boot magic. Especially since spring boot 2.5.
The issue was with datasource initialization when we see content of org.springframework.boot.autoconfigure.jdbc.DataSourceInitializer class that takes care of Database initialization through *.sql files.It has post construct method as follows
#PostConstruct
public void init() {
if (!this.properties.isInitialize()) {
logger.debug("Initialization disabled (not running DDL scripts)");
return;
}
if (this.applicationContext.getBeanNamesForType(DataSource.class, false,
false).length > 0) {
this.dataSource = this.applicationContext.getBean(DataSource.class);
}
if (this.dataSource == null) {
logger.debug("No DataSource found so not initializing");
return;
}
runSchemaScripts();
}
The runSchemaScripts() method will initialize the data before hibernate schema creation and update operation is perfomed so if database schema is not generated then these method will create schema if you provide that in SQL script, but I want to perform operation after the schema is created/updated, for that class contains
#Override
public void onApplicationEvent(DataSourceInitializedEvent event) {
if (!this.properties.isInitialize()) {
logger.debug("Initialization disabled (not running data scripts)");
return;
}
// NOTE the event can happen more than once and
// the event datasource is not used here
if (!this.initialized) {
runDataScripts();
this.initialized = true;
}
}
this is called if after the hibernate schema creation/updation operation when we have spring boots default Datasource creation mechanism.
But as I was creating Datasource by myself,so it was not creating DataSourceInitializedEvent,so the data initilization scripts data.sql was not executed.
So I have changed my Data source creation logic to create DataSourceInitializedEvent as follows and that solved my issue.
#Autowired
private ConfigurableApplicationContext applicationContext;
#Bean
#Primary
public LocalContainerEntityManagerFactoryBean userEntityManager() {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(primaryDataSource());
em.setPackagesToScan(new String[] { "com.projectx.mysql" });
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
em.setJpaVendorAdapter(vendorAdapter);
HashMap<String, Object> properties = new HashMap<String, Object>();
properties.put("hibernate.hbm2ddl.auto", env.getProperty("spring.jpa.hibernate.ddl-auto_mysql"));
properties.put("hibernate.dialect", env.getProperty("spring.jpa.properties.hibernate.dialect_mysql"));
properties.put("hibernate.show_sql", env.getProperty("spring.jpa.show-sql"));
em.setJpaPropertyMap(properties);
this.applicationContext.publishEvent(new DataSourceInitializedEvent(primaryDataSource()));
return em;
}
#Bean
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
Added this.applicationContext.publishEvent(new DataSourceInitializedEvent(primaryDataSource())); to create the DataSourceInitializedEvent event
I managed to get 2 data-sources instantiated and initiate schema and data in one of them with this test project. Hope that helps, maybe i missed some requirement of yours that makes my suggestions invalid :(
For ref (guess you already saw this): https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto-two-datasources