how to use #DataJpaTest with multiple datasourse - spring-boot

I tried to write integration test using annotation #DataJpaTest .
I have two datasource: Primary and secondary (class config)
in result i have an error:
expected single matching bean but found 2: primaryDataSource,secondary
then i tried to add a annotation
#AutoConfigureTestDatabase(replace= AutoConfigureTestDatabase.Replace.AUTO_CONFIGURED)
and With AUTO_CONFIGURED only DataSources configured by properties will be replaced but instead embedded h2 i saw Dialect : HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
how using #DataJpaTest with multiple datasources ?
public class DataSourcesConfig {
#Bean
#Primary
#ConfigurationProperties(prefix="spring.datasource")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "secondary")
#ConfigurationProperties(prefix="datasource.secondary")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
}

I have a #Primary configuration class
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
basePackages = {"com.something"}
)
public class APrimaryDBDBConfiguration {
#Primary
#Bean
#ConfigurationProperties("spring.datasource")
public DataSourceProperties dataSourceProperties() {
return new DataSourceProperties();
}
#Primary
#Bean(name = "dataSource")
#ConfigurationProperties("spring.datasource.hikari")
public HikariDataSource dataSource() {
return dataSourceProperties()
.initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
}
#Primary
#Bean(name = "entityManagerFactory")
#Profile("!test")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("dataSource") DataSource dataSource) {
return builder
.dataSource(dataSource)
.packages("com")
.persistenceUnit("some_persistence_unit")
.build();
}
#Primary
#Bean(name = "transactionManager")
public PlatformTransactionManager transactionManager(
#Qualifier("entityManagerFactory") EntityManagerFactory entityManagerFactory) {
return new JpaTransactionManager(entityManagerFactory);
}
}
Later, in my repository test classes:
#RunWith(SpringRunner.class)
#DataJpaTest
#ActiveProfiles("test")
#Import(APrimaryDBDBConfiguration.class)
Finally, my test properties have:
spring.jpa.hibernate.ddl-auto=create-drop

Check if you have h2 database added as dependency in test scope. If not, add and try:
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>test</scope>
</dependency>

Found a possible solution here.
Basically you manually configure the H2 database appropriately instead of letting Spring do it automatically.
Create an application.properties file in “src/test/resources” with the following content
# Let Spring autodetect the different SQL Dialects of each datasource
spring.jpa.database=default
# Generate the DB schema in the In-Memory H2 databases based on the JPA Entities
spring.jpa.generate-ddl=true
# H2 In-Memory Database "foo" (used in tests instead of a real PostgreSQL DB)
spring.datasource.url=jdbc:h2:mem:foo;DB_CLOSE_ON_EXIT=FALSE
spring.datasource.username=sa
spring.datasource.password=
spring.datasource.driver-class-name=org.h2.Driver
# H2 In-Memory Database "bar" (used in tests instead of a real PostgreSQL DB)
bar.datasource.url=jdbc:h2:mem:bar;DB_CLOSE_ON_EXIT=FALSE
bar.datasource.username=sa
bar.datasource.password=
bar.datasource.driver-class-name=org.h2.Driver

Related

Spring boot Hibernate error "Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set" when working with multiple data sources

I am building a Springboot application that needs to talk to 2 different databases (DB2 and Oracle).
Spring-boot-starter-parent version 2.6.6
hibernate version 5.6.7.Final
I first added DB2 support and that was working fine until I added oracle related settings in application.properties file below.
Adding oracle related settings causes following error:
org.hibernate.HibernateException: Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set
Below are the databases and hibernate related settings from my application.properties:
# ==============================
# = DB2
db2.datasource.jdbc-url=jdbc:db2://SERVER1:PORT/DATABASE-1:currentSchema=SCHEMA;
db2.datasource.username=USER1
db2.datasource.password=PASSWORD1
db2.datasource.driver-class-name=com.ibm.db2.jcc.DB2Driver
db2.jpa.properties.hibernate.dialect=org.hibernate.dialect.DB2390Dialect
# ==============================
# = ORACLE
oracle.datasource.jdbc-url=jdbc:oracle:thin:#SERVER2:PORT/DATABASE-2
oracle.datasource.username=USER2
oracle.datasource.password=PASSWORD2
oracle.jpa.properties.hibernate.dialect=org.hibernate.dialect.Oracle10gDialect
# ==============================
# = JPA / HIBERNATE
spring.jpa.hibernate.ddl-auto=update
spring.jpa.show-sql=true
My pom.xml contains dependencies for oracle and DB2 amongs other required dependencies:
...
<dependency>
<groupId>com.ibm.db2</groupId>
<artifactId>jcc</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.oracle.database.jdbc</groupId>
<artifactId>ojdbc8</artifactId>
<scope>runtime</scope>
</dependency>
...
I have placed my entities, repositories, and data source configurations into different packages as I read is required in this article https://www.javadevjournal.com/spring-boot/multiple-data-sources-with-spring-boot/. My package structure looks like:
project
- dataconfig
- db2
- config
- entity
- repository
- oracle
- config
- entity
- repository
I also added some entities and repositories and my configuration classes.
Here is my DB2Configuration class:
package project.dataconfig.db2.config;
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "db2EntityManagerFactory",
transactionManagerRef = "db2TransactionManager",
basePackages = {
"project.dataconfig.db2.repository"
}
)
public class Db2Configuration {
#Primary
#Bean(name = "db2DataSource")
#ConfigurationProperties(prefix = "db2.datasource")
public DataSource db2DataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean(name = "db2EntityManagerFactory")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
EntityManagerFactoryBuilder builder
, #Qualifier("db2DataSource") DataSource dataSource) {
return builder
.dataSource(dataSource)
.packages("project.dataconfig.db2.entity")
.persistenceUnit("db2persistanceunit")
.build();
}
#Primary
#Bean(name = "db2TransactionManager")
public PlatformTransactionManager db2TransactionManager(
#Qualifier("db2EntityManagerFactory") EntityManagerFactory db2EntityManagerFactory) {
return new JpaTransactionManager(db2EntityManagerFactory);
}
}
Here is my OracleConfiguration class:
package project.dataconfig.oracle.config;
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "oracleEntityManagerFactory",
transactionManagerRef = "oracleTransactionManager",
basePackages = {
"project.dataconfig.oracle.repository"
}
)
public class OracleConfiguration {
#Bean(name = "oracleDataSource")
#ConfigurationProperties(prefix = "oracle.datasource")
public DataSource oracleDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "oracleEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
EntityManagerFactoryBuilder builder
, #Qualifier("oracleDataSource") DataSource dataSource) {
return builder
.dataSource(dataSource)
.packages("project.dataconfig.oracle.entity")
.persistenceUnit("oraclepersistanceunit")
.build();
}
#Bean(name = "oracleTransactionManager")
public PlatformTransactionManager oracleTransactionManager(
#Qualifier("oracleEntityManagerFactory") EntityManagerFactory oracleEntityManagerFactory) {
return new JpaTransactionManager(oracleEntityManagerFactory);
}
}
Before I added oracle related settings in application.properties file, my application worked just find with only DB2 settings as described above.
Once I added oracle related settings and configuration, I started getting this error:
org.hibernate.HibernateException: Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set
Based on the error, I think the problem is caused by my settings for Hibernate dialect in application.properties file. I think it is caused by one of the two settings
...
db2.jpa.properties.hibernate.dialect=org.hibernate.dialect.DB2390Dialect
...
oracle.jpa.properties.hibernate.dialect=org.hibernate.dialect.Oracle10gDialect
How do I resolve this issue?
I figure it out.
Modify method entityManagerFactory for both Db2Configuration and OracleConfiguration to supply them with the information about hibernate dialect:
for DB2
#Primary
#Bean(name = "db2EntityManagerFactory")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
EntityManagerFactoryBuilder builder
, #Qualifier("db2DataSource") DataSource dataSource) {
final HashMap<String, Object> hibernateProperties = new HashMap<String, Object>();
hibernateProperties.put("hibernate.dialect", "org.hibernate.dialect.DB2390Dialect");
return builder
.dataSource(dataSource)
.packages("project.dataconfig.db2.entity")
.properties(hibernateProperties)
.persistenceUnit("db2persistanceunit")
.build();
}
for Oracle
#Bean(name = "oracleEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
EntityManagerFactoryBuilder builder
, #Qualifier("oracleDataSource") DataSource dataSource) {
final HashMap<String, Object> hibernateProperties = new HashMap<String, Object>();
hibernateProperties.put("hibernate.dialect", "org.hibernate.dialect.Oracle10gDialect");
return builder
.dataSource(dataSource)
.packages("project.dataconfig.oracle.entity")
.properties(hibernateProperties)
.persistenceUnit("oraclepersistanceunit")
.build();
}
After this, my console shows when running app, indicating all is good:
HHH000400: Using dialect: org.hibernate.dialect.DB2390Dialect
Initialized JPA EntityManagerFactory for persistence unit 'db2persistanceunit'
HHH000204: Processing PersistenceUnitInfo [name: oraclepersistanceunit]
HHH000400: Using dialect: org.hibernate.dialect.Oracle10gDialect
Initialized JPA EntityManagerFactory for persistence unit 'oraclepersistanceunit'
My actuator /health endpoint also sees both databases as up and running.

HibernateException: hibernate.dialect not set, but I did set it in application.properties

I am working on a Spring Boot application that uses a number of databases. I define datasources, I define entitymanagers, and I keep getting this error
org.hibernate.HibernateException: Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set
at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.determineDialect(DialectFactoryImpl.java:100)
at org.hibernate.engine.jdbc.dialect.internal.DialectFactoryImpl.buildDialect(DialectFactoryImpl.java:54)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:137)
at org.hibernate.engine.jdbc.env.internal.JdbcEnvironmentInitiator.initiateService(JdbcEnvironmentInitiator.java:35)
at org.hibernate.boot.registry.internal.StandardServiceRegistryImpl.initiateService(StandardServiceRegistryImpl.java:94)
at org.hibernate.service.internal.AbstractServiceRegistryImpl.createService(AbstractServiceRegistryImpl.java:263)
This is the #Configuration file for one database:
#Configuration
public class CustomerDbConfig {
#Bean
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "customerEntityManager")
#PersistenceContext(unitName = "customerEntityManager")
public LocalContainerEntityManagerFactoryBean customerEntityManagerFactory(
EntityManagerFactoryBuilder builder, DataSource dataSource) {
return builder.dataSource(dataSource).packages("com.example.report_service").build();
}
#Bean
public JpaTransactionManager transactionManager(EntityManagerFactory emf) {
JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(emf);
return jpaTransactionManager;
}
#Bean
public EntityManagerFactoryBuilder entityManagerFactoryBuilder() {
return new EntityManagerFactoryBuilder(new HibernateJpaVendorAdapter(), new HashMap<>(), null);
}
}
and these are the application.properties
spring.datasource.url=jdbc:postgresql://dv1.example.com:5432/customer
spring.datasource.username=${DATABASECUSTOMERUSERNAME:customer}
spring.datasource.password=${DATABASECUSTOMERPASSWORD:customer}
spring.datasource.driver-class-name=org.postgresql.Driver
spring.datasource.hikari.minimum-idle=2
spring.datasource.hikari.maximum-pool-size=${MAX_POOL_SIZE:20}
spring.datasource.tomcat.validation-query=SELECT 1
spring.datasource.tomcat.validation-query-timeout=5000
spring.datasource.tomcat.test-on-borrow=true
spring.datasource.tomcat.test-on-connect=true
spring.transaction.default-timeout=${TRANSACTION_TIMEOUT:3600}
spring.jpa.properties.hibernate.proc.param_null_passing=true
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.PostgreSQLDialect
spring.jpa.properties.hibernate.jdbc.lob.non_contextual_creation= true
spring.jpa.open-in-view=false
spring.jpa.show-sql=true
spring.jpa.properties.hibernate.format_sql=true
spring.jpa.properties.hibernate.show_sql=false
spring.jpa.properties.hibernate.use_sql_comments=true
spring.jpa.properties.hibernate.type=trace
spring.autoconfigure.exclude=org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration
When I explicitely set a persistenceUnit name in the entitymanager, the error changes, it complains about not finding a PU with the name I specified.
I limited my application to having only one database, trying to figure out what's happening. When I remove the configuration file there's no issues. What makes this application special is that it uses native queries only, it doesn't have #Entity classes because the database is updated by other applictions. When I remove the .packages(..) part from the code, it complaines like
No persistence units parsed from {classpath*:META-INF/persistence.xml}
I suspect the issue is related to the fact that we don't have classes that need to be scanned.
Edit:
Following M.Deinum's answer, I replaced the config by
#Configuration
#EnableTransactionManagement
public class CustomerDbConfig {
#ConfigurationProperties(prefix = "customer.datasource")
#Bean(name = "customer-datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "customer-jdbcTemplate")
public JdbcTemplate primaryJdbcTemplate(#Qualifier("customer-datasource") DataSource dataSource) {
return new JdbcTemplate(dataSource);
}
}
and I use the four templates in the daos. Works like a charm.
When using JPA you need entities. Using the JPA classes to just execute SQL queries is overkill. Just use a plain Jdbctemplate or NamedParameterJdbcTemplate for that.

Hikari - Spring Boot is ignoring hikari properties

I have a Spring Boot 2 application which has two datasources. Both datasources work, however I am unable to change properties such as maximum-pool-size, my changes are not taking effect.
I am configuration my two datasources in my application.properties file;
spring.datasource.url=jdbc:sqlserver://server;databaseName=ProductionMetrics
spring.datasource.username=ProductionMeusernametricsUser
spring.datasource.password=password
spring.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.hikari.maximum-pool-size=20
# Products
trwbi.datasource.url=jdbc:sqlserver://server;databaseName=TRWBI
trwbi.datasource.username=username
trwbi.datasource.password=password
trwbi.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
trwbi.datasource.hikari.maximum-pool-size=20
Then, I'm setting up the two datasources like this;
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "defaultEntityManagerFactory",
transactionManagerRef = "defaultTransactionManager",
basePackages = {
"com.domain.visualisation.shared.repository"
}
)
public class DefaultDbConfig {
#Primary
#Bean(name = "defaultDataSource")
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource defaultDataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean(name = "defaultEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean
entityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("defaultDataSource") DataSource dataSource
) {
return builder
.dataSource(dataSource)
.packages("com.domain.visualisation.shared.entities")
.persistenceUnit("default")
.build();
}
#Primary
#Bean(name = "defaultTransactionManager")
public PlatformTransactionManager defaultTransactionManager(
#Qualifier("defaultEntityManagerFactory") EntityManagerFactory defaultEntityManagerFactory
) {
return new JpaTransactionManager(defaultEntityManagerFactory);
}
}
When I turn on debugging for Hikari, I can see that the maximumPoolSize value is still 10, rather than the value of 20 that I have defined. I've tried to apply other properties such as leak-detection-threshhold, pool-name and idle-timeout, but neither of those are being applied either.
Why are they not being applied?
You should use property name maximumPoolSize
spring.datasource.hikari.maximumPoolSize=20
maximumPoolSize
This property controls the maximum size that the pool is allowed to reach, including both idle and in-use connections.
In case of multiple datasources and because you are using DataSourceBuilder the following works.
spring.datasource.maximum-pool-size=20
trwbi.datasource.maximum-pool-size=20
Also in your case I would recommend to turn off autoconfiguration:
#SpringBootApplication(scanBasePackages = {...},
exclude = {DataSourceAutoConfiguration.class} )
#EnableSwagger2
public class Application {
....
If you don't use builder, but use autoconfiguration, then use
spring.datasource.hikari.minimumIdle=1
spring.datasource.hikari.maximum-pool-size=3

Spring Configuration Metadata

I am setting up two data sources as shown here at http://docs.spring.io/spring-boot/docs/1.3.0.M2/reference/htmlsingle/#howto-two-datasources using spring boot, but when doing so my application.properties shows warnings that for example x.x.username is an unknown property. This is correct to some extent as javax.sql.DataSource does not contain url, username, password, etc. but the implementation classes do. I have annotation processor set up and it works fine when working with concrete classes.
I notice that org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$NonEmbeddedConfiguration uses both DataSourceProperties and has #ConfigurationProperties annotated on dataSource(). This would probably get rid of my warnings but what is the point of this. Isn't it setting the properties twice this way?
Config:
#Bean
#Primary
#ConfigurationProperties(prefix="datasource.primary")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="datasource.secondary")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
Properties with warnings:
datasource.primary.url=jdbc:...
datasource.primary.username=user
datasource.primary.password=password
datasource.secondary.url=jdbc:...
datasource.secondary.username=user
datasource.secondary.password=password
Since someone bothered to +1 this question I thought I'd post a solution. Note that I think the #ConfigurationProperties on the DataSources themselves are unecessary because they are already set on the DataSourceProperties which is used to build the DataSource, but I left it in there because that's how the Spring team has done it in org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$NonEmbeddedConfiguration. My only guess why would be if your DataSource had additional properties that could be set other than what's exposed in DataSourceProperties, but then you would get warnings in the "Spring Boot application.properties editor" for those properties.
Note that DataSourceBuilder will use Tomcat, HikariCP or Commons DBCP in that order if found on Classpath as DataSource unless you specify something else with dataSourceBuilder.type(Class<? extends DataSource>)
Properties:
datasource.primary.url=jdbc:...
datasource.primary.username=user
datasource.primary.password=password
datasource.secondary.url=jdbc:...
datasource.secondary.username=user
datasource.secondary.password=password
Java Config:
#Bean
#Primary
#ConfigurationProperties(prefix = "datasource.primary")
public DataSourceProperties primaryProps() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties(prefix = "datasource.secondary")
public DataSourceProperties secondaryProps() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties(prefix = "datasource.primary")
public DataSource secondaryDataSource() {
DataSourceProperties props = secondaryProps();
return DataSourceBuilder.create(props.getClassLoader())
.driverClassName(props.getDriverClassName())
.url(props.getUrl())
.username(props.getUsername())
.password(props.getPassword())
.build();
}
#Bean
#ConfigurationProperties(prefix = "datasource.primary")
public DataSource secondaryDataSource() {
DataSourceProperties props = secondaryProps();
return DataSourceBuilder.create(props.getClassLoader())
.driverClassName(props.getDriverClassName())
.url(props.getUrl())
.username(props.getUsername())
.password(props.getPassword())
.build();
}

Spring Boot does not seem to pick up Atomikos when used for tests

I am working on a prototype for using Spring Boot in our project. We have a JBoss server in production and I was thinking of running integration tests against Undertow embedded server using an embedded transaction manager like Atomikos, because a persistence.xml exists that I have to reuse. My test app context file has the following lines:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = Application.class)
#WebAppConfiguration
#EnableAutoConfiguration
#IntegrationTest("server.port:0")
#ActiveProfiles("test")
public abstract class TestApplicationContext {
...
}
I have also added a custom test configuration as:
#Configuration
public class TestConfiguration {
#Value("${spring.jpa.hibernate.dialect}")
private String dialectClassName;
#Value("${spring.jpa.hibernate.transaction.manager_lookup_class}")
private String transactionManagerClass;
#Bean
public EmbeddedServletContainerFactory servletContainer() {
return new UndertowEmbeddedServletContainerFactory(9000); // Don't know if this can be avoided using some properties
}
#Bean
#ConfigurationProperties(prefix = DataSourceProperties.PREFIX)
public DataSource dataSource() throws Exception {
return DataSourceBuilder.create().build();
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory(EntityManagerFactoryBuilder builder,
DataSource dataSource) {
LocalContainerEntityManagerFactoryBean entityManagerFactoryBean =
builder.dataSource(dataSource).persistenceUnit("main").build();
Properties additionalProperties = new Properties();
additionalProperties.put("hibernate.dialect", dialectClassName);
additionalProperties.put("hibernate.transaction.manager_lookup_class", transactionManagerClass);
entityManagerFactoryBean.setJpaProperties(additionalProperties);
return entityManagerFactoryBean;
}
#Bean
public PlatformTransactionManager transactionManager() {
// this should not be needed if I have included Atomikos but it seems to pick
// JPA Transaction manager still and fails with the famous NullPointerException at
// CMTTransaction class - because it cannot find a JTA environment
// return new JtaTransactionManager(userTransaction, transactionManager);
}
}
My gradle include for Atomikos is:
testCompile('org.springframework.boot:spring-boot-starter-jta-atomikos')
I am using Spring Boot 1.2.0-RC2.
CAn someone point out what I am doing wrong or how to solve this?
Thanks,
Paddy

Resources