Hikari - Spring Boot is ignoring hikari properties - spring-boot

I have a Spring Boot 2 application which has two datasources. Both datasources work, however I am unable to change properties such as maximum-pool-size, my changes are not taking effect.
I am configuration my two datasources in my application.properties file;
spring.datasource.url=jdbc:sqlserver://server;databaseName=ProductionMetrics
spring.datasource.username=ProductionMeusernametricsUser
spring.datasource.password=password
spring.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.hikari.maximum-pool-size=20
# Products
trwbi.datasource.url=jdbc:sqlserver://server;databaseName=TRWBI
trwbi.datasource.username=username
trwbi.datasource.password=password
trwbi.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
trwbi.datasource.hikari.maximum-pool-size=20
Then, I'm setting up the two datasources like this;
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "defaultEntityManagerFactory",
transactionManagerRef = "defaultTransactionManager",
basePackages = {
"com.domain.visualisation.shared.repository"
}
)
public class DefaultDbConfig {
#Primary
#Bean(name = "defaultDataSource")
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource defaultDataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean(name = "defaultEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean
entityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("defaultDataSource") DataSource dataSource
) {
return builder
.dataSource(dataSource)
.packages("com.domain.visualisation.shared.entities")
.persistenceUnit("default")
.build();
}
#Primary
#Bean(name = "defaultTransactionManager")
public PlatformTransactionManager defaultTransactionManager(
#Qualifier("defaultEntityManagerFactory") EntityManagerFactory defaultEntityManagerFactory
) {
return new JpaTransactionManager(defaultEntityManagerFactory);
}
}
When I turn on debugging for Hikari, I can see that the maximumPoolSize value is still 10, rather than the value of 20 that I have defined. I've tried to apply other properties such as leak-detection-threshhold, pool-name and idle-timeout, but neither of those are being applied either.
Why are they not being applied?

You should use property name maximumPoolSize
spring.datasource.hikari.maximumPoolSize=20
maximumPoolSize
This property controls the maximum size that the pool is allowed to reach, including both idle and in-use connections.

In case of multiple datasources and because you are using DataSourceBuilder the following works.
spring.datasource.maximum-pool-size=20
trwbi.datasource.maximum-pool-size=20
Also in your case I would recommend to turn off autoconfiguration:
#SpringBootApplication(scanBasePackages = {...},
exclude = {DataSourceAutoConfiguration.class} )
#EnableSwagger2
public class Application {
....
If you don't use builder, but use autoconfiguration, then use
spring.datasource.hikari.minimumIdle=1
spring.datasource.hikari.maximum-pool-size=3

Related

spring boot with multiple databases

I'm trying to write an application that accesses data from two sources. I'm using Spring Boot 2.3.2. I've looked at several sources for info about how to configure the app: the Spring documentation talks about setting up multiple datasources, but does not explain how to link up JPA repositories. This Baeldung article goes a lot further, but I'm looking to take advantage of autoconfiguration in Spring.
So far, I've created a separate package, added a config class (along with model and repositories), and included this package in scanBasePackages so that it's picked up. Since I'll have more than one datasource, I've added this to my #SpringBootApplication:
#Bean
#Primary
public DataSourceProperties dataSourceProperties() {
return new DataSourceProperties();
}
This successfully loads up my Spring app using the standard spring config values. The two databases are on different servers, but should share characteristics (other than url and credentials).
So, my auxiliary configuration file looks like this
#Configuration
#EnableAutoConfiguration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "orgEntityManagerFactory",
transactionManagerRef = "orgTransactionManager",
basePackages = {
"pacage2.repositories"
}
)
public class DataSourceConfiguration {
// added because of this answer: https://stackoverflow.com/a/51305724/167889
#Bean
public EntityManagerFactoryBuilder entityManagerFactoryBuilder() {
return new EntityManagerFactoryBuilder(new HibernateJpaVendorAdapter(), new HashMap<>(), null);
}
#Bean
#ConfigurationProperties(prefix = "external.datasource")
public DataSourceProperties orgDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
public HikariDataSource orgDataSource(#Qualifier("orgDataSourceProperties") DataSourceProperties properties) {
return properties.initializeDataSourceBuilder().type(HikariDataSource.class)
.build();
}
#Bean
public LocalContainerEntityManagerFactoryBean orgEntityManagerFactory(EntityManagerFactoryBuilder builder,
#Qualifier("orgDataSource") DataSource dataSource) {
return builder
.dataSource(dataSource)
.packages("package2.model")
.build();
}
#Bean
public PlatformTransactionManager orgTransactionManager(
#Qualifier("orgEntityManagerFactory") EntityManagerFactory entityManagerFactory
) {
return new JpaTransactionManager(entityManagerFactory);
}
}
Now, the error I'm getting right now is Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set. However, I have that value in my config and it's applied by the Spring auto config. I believe it needs to be set in the EntityManagerFactoryBuilder and by creating my own, the autoconfig isn't getting applied.
How can I have my cake and eat it too? I'd like to leverage as much of the robust autoconfiguration that Spring provides to setup datasources and wire them to the appropriate repositories. Effectively, all that I want to change is the url and credentials, and I can separate the entities and repositories into a completely separate package for easy scanning.

How to dynamicly create bean at application start with properties in Spring Boot

I use multiple databases to store different entities. My entities and repos are splited in different packeges. And for each database I need to create #Configuration to persist data properly and create tables properly.
Here is #Configuration file for one of me databases
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "entityManagerFactory",
basePackages = { "com.domain.shop.users.repositories" },
transactionManagerRef = "transactionManager"
)
public class UsersDatabaseConfig {
#Autowired
private DatasourceConnectionManager dscm;
#Primary
#Bean(name = "dataSource1")
public DataSource dataSource() {
return dscm.getDataSource("users");
}
#Primary
#Bean(name = "entityManagerFactory")
public LocalContainerEntityManagerFactoryBean
entityManagerFactory(EntityManagerFactoryBuilder builder,
#Qualifier("dataSource1") DataSource dataSource1) {
HashMap<String, Object> properties = new HashMap<>();
properties.put("hibernate.hbm2ddl.auto", "update");
return builder
.dataSource(dataSource1)
.packages("com.domain.shop.users.models")
.properties(properties)
.build();
}
#Primary
#Bean(name = "transactionManager")
public PlatformTransactionManager
transactionManager(#Qualifier("entityManagerFactory") EntityManagerFactory
entityManagerFactory) {
return new JpaTransactionManager(entityManagerFactory);
}
}
It works pretty fine! But I need to create separate class for each database
I'd like to create such beans at application start reading properties .yml file.
And look at annotations on top - how to pass some parameters to annotations?
Other words, I have .yml file with database connections properties. I want to add some property to each database (like, rootdirectory = com.domain.shop.products). After that I want to create dynamic bean with following code:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "productsEntityManagerFactory",
basePackages = { "com.domain.shop.products.repositories" },
transactionManagerRef = "productsTransactionManager"
)
and next
#Bean(name = "productsDataSource")
public DataSource dataSource() {
return dscm.getDataSource("products");
}
You can use #Profile annotation. And then create properties file for every profile. E.g.
#Profile("test")
#Profile("dev")
application-test.yml
application-dev.yml
Use #ConditionalOnExpression will load the `#Configuration class if expression validates to true
#ConditionalExpression("${my.rest.controller.enabled}")
or use #ConditionalOnProperty
#ConditionalOnProperty(prefix = "spring", name = "example.values")

Spring batch boot Multiple datasources Multiple schemas

I have a spring batch job using spring boot which has 2 datasources. Each datasource again has 2 schemas each. I need to specify default schema for both the datasources. I know of property spring.jpa.properties.hibernate.default_schema which i am using to specify default schema for one datasource. Is there a way to specify default schema for another schema?
Currently, to specify default schema for the other datasource , i am using alter session query to switch schema as required. I am trying to get rid of this alter session query from my java code. Any suggestions on it is greatly appreciated.
edit 1: Both are ORACLE databases
If you use multiple datasources, then you probably has a #Configuration class for each datasource. In this case you can set additional properties to the entityManager. This configuration is needed:
props.put("spring.datasource.schema", "test");
Full example
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(entityManagerFactoryRef = "testEntityManagerFactory", transactionManagerRef = "testTransactionManager",
basePackages = {"com.test.repository"})
public class TestDbConfig {
#Bean(name = "testDataSource")
#ConfigurationProperties(prefix = "test.datasource")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "testEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(EntityManagerFactoryBuilder builder, #Qualifier("testDataSource") DataSource dataSource) {
return builder.dataSource(dataSource).packages("com.test.model").persistenceUnit("test").properties(jpaProperties()).build();
}
private Map<String, Object> jpaProperties() {
Map<String, Object> props = new HashMap<>();
props.put("hibernate.physical_naming_strategy", SpringPhysicalNamingStrategy.class.getName());
props.put("hibernate.implicit_naming_strategy", SpringImplicitNamingStrategy.class.getName());
props.put("spring.datasource.schema", "test");
return props;
}
#Bean(name = "testTransactionManager")
public PlatformTransactionManager transactionManager(#Qualifier("testEntityManagerFactory") EntityManagerFactory entityManagerFactory) {
return new JpaTransactionManager(entityManagerFactory);
}
}

Spring Data JPA - Multiple EnableJpaRepositories

My application has multiple data sources , so i have created two data source configuration classes based on this URL .
But while running the spring boot application am getting error
Description:
Field userDataRepo in com.cavion.services.UserDataService required a bean named 'entityManagerFactory' that could not be found.
Action:
Consider defining a bean named 'entityManagerFactory' in your configuration.
From this Question on StackOverflow helped me to figure out the issue.i need to specify the entityManagerFactoryRef on my JPA repositories .
But i have many repository classes some of them uses Entitymanager 'A' and some of them uses 'B' . my current spring boot application class is like this
#SpringBootApplication(exclude = { DataSourceAutoConfiguration.class, HibernateJpaAutoConfiguration.class,
DataSourceTransactionManagerAutoConfiguration.class })
#EnableTransactionManagement
#EntityScan("com.info.entity")
#ComponentScan({"com.info.services","com.info.restcontroller"})
#EnableJpaRepositories("com.info.repositories")
public class CavionApplication {
public static void main(String[] args) {
SpringApplication.run(CavionApplication.class, args);
}
#Bean
public CommandLineRunner commandLineRunner(ApplicationContext ctx) {
return args -> {
System.out.println("Let's inspect the beans provided by Spring Boot:");
String[] beanNames = ctx.getBeanDefinitionNames();
Arrays.sort(beanNames);
for (String beanName : beanNames) {
System.out.println(beanName);
}
};
}}
I have given the EnableJpaRepositories on the spring boot class , so how can i configure multiple EnableJpaRepositories so that i can configure multiple entityManagerFactory ?
Please suggest the best way to setup the multiple data sources .
In order to let spring knows what DataSource is related to what Repository you should define it at the #EnableJpaRepositories annotation. Let's assume that we have two entities, the Servers entity and the Domains entity and each one has its own Repo then each Repository has its own JpaDataSource configuration.
1. Group all the repositories based on the Data Source that they are related to. For example
Repository for Domains entities (package: org.springdemo.multiple.datasources.repository.domains):
package org.springdemo.multiple.datasources.repository.domains;
import org.springdemo.multiple.datasources.domain.domains.Domains;
import org.springframework.data.jpa.repository.JpaRepository;
public interface DomainsRepository extends JpaRepository<Domains,Long> {
}
Repository for Servers entities (package: org.springdemo.multiple.datasources.repository.servers)
package org.springdemo.multiple.datasources.repository.servers;
import org.springdemo.multiple.datasources.domain.servers.Servers;
import org.springframework.data.jpa.repository.JpaRepository;
public interface ServersRepository extends JpaRepository<Servers,Long> {
}
2. For each JPA Data Soruce you need to define a configuration, in this example I show how to configure two different DataSources
Domains Jpa Configuration: the relationship between the Data Source and the repository is defined in the basePackages value, that is the reason why is necessary to group the repositories in different packages depending on the entity manager that each repo will use.
#Configuration
#EnableJpaRepositories(
entityManagerFactoryRef = "domainsEntityManager",
transactionManagerRef = "domainsTransactionManager",
basePackages = {"org.springdemo.multiple.datasources.repository.domains"}
)
public class DomainsConfig {
Servers Data Source Configuration: as you can see the basePackages value has the package name of the Servers Repository , and also the values of entityManagerFactoryRef and transactionManagerRef are different in order to let spring separate each entityManager.
#Configuration
#EnableJpaRepositories(
entityManagerFactoryRef = "serversEntityManager",
transactionManagerRef = "serversTransactionManager",
basePackages = {"org.springdemo.multiple.datasources.repository.servers"}
)
public class ServersConfig {
3. Set one Datasource as primary
In order to avoid the error message: Parameter 0 of constructor in org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration required a single bean, but 2 were found: just set one of the datasource as #Primary, in this example I select the Servers Datasource as primary:
#Bean("serversDataSourceProperties")
#Primary
#ConfigurationProperties("app.datasource.servers")
public DataSourceProperties serversDataSourceProperties(){
return new DataSourceProperties();
}
#Bean("serversDataSource")
#Primary
#ConfigurationProperties("app.datasource.servers")
public DataSource serversDataSource(#Qualifier("serversDataSourceProperties") DataSourceProperties serversDataSourceProperties) {
return serversDataSourceProperties().initializeDataSourceBuilder().build();
}
If you need more information please see the full example for each configuration:
Servers JPA Configuration
#Configuration
#EnableJpaRepositories(
entityManagerFactoryRef = "serversEntityManager",
transactionManagerRef = "serversTransactionManager",
basePackages = {"org.springdemo.multiple.datasources.repository.servers"}
)
public class ServersConfig {
#Bean(name = "serversEntityManager")
public LocalContainerEntityManagerFactoryBean getServersEntityManager(EntityManagerFactoryBuilder builder,
#Qualifier("serversDataSource") DataSource serversDataSource){
return builder
.dataSource(serversDataSource)
.packages("org.springdemo.multiple.datasources.domain.servers")
.persistenceUnit("servers")
.properties(additionalJpaProperties())
.build();
}
Map<String,?> additionalJpaProperties(){
Map<String,String> map = new HashMap<>();
map.put("hibernate.hbm2ddl.auto", "create");
map.put("hibernate.dialect", "org.hibernate.dialect.MySQLDialect");
map.put("hibernate.show_sql", "true");
return map;
}
#Bean("serversDataSourceProperties")
#Primary
#ConfigurationProperties("app.datasource.servers")
public DataSourceProperties serversDataSourceProperties(){
return new DataSourceProperties();
}
#Bean("serversDataSource")
#Primary
#ConfigurationProperties("app.datasource.servers")
public DataSource serversDataSource(#Qualifier("serversDataSourceProperties") DataSourceProperties serversDataSourceProperties) {
return serversDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean(name = "serversTransactionManager")
public JpaTransactionManager transactionManager(#Qualifier("serversEntityManager") EntityManagerFactory serversEntityManager){
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(serversEntityManager);
return transactionManager;
}
}
Domains JPA Configuration
#Configuration
#EnableJpaRepositories(
entityManagerFactoryRef = "domainsEntityManager",
transactionManagerRef = "domainsTransactionManager",
basePackages = {"org.springdemo.multiple.datasources.repository.domains"}
)
public class DomainsConfig {
#Bean(name = "domainsEntityManager")
public LocalContainerEntityManagerFactoryBean getdomainsEntityManager(EntityManagerFactoryBuilder builder
,#Qualifier("domainsDataSource") DataSource domainsDataSource){
return builder
.dataSource(domainsDataSource)
.packages("org.springdemo.multiple.datasources.domain.domains")
.persistenceUnit("domains")
.properties(additionalJpaProperties())
.build();
}
Map<String,?> additionalJpaProperties(){
Map<String,String> map = new HashMap<>();
map.put("hibernate.hbm2ddl.auto", "create");
map.put("hibernate.dialect", "org.hibernate.dialect.H2Dialect");
map.put("hibernate.show_sql", "true");
return map;
}
#Bean("domainsDataSourceProperties")
#ConfigurationProperties("app.datasource.domains")
public DataSourceProperties domainsDataSourceProperties(){
return new DataSourceProperties();
}
#Bean("domainsDataSource")
public DataSource domainsDataSource(#Qualifier("domainsDataSourceProperties") DataSourceProperties domainsDataSourceProperties) {
return domainsDataSourceProperties.initializeDataSourceBuilder().build();
}
#Bean(name = "domainsTransactionManager")
public JpaTransactionManager transactionManager(#Qualifier("domainsEntityManager") EntityManagerFactory domainsEntityManager){
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(domainsEntityManager);
return transactionManager;
}
}
In order to separate each datasource I put the configuration in the application.properties file, like this:
app.datasource.domains.url=jdbc:h2:mem:~/test
app.datasource.domains.driver-class-name=org.h2.Driver
app.datasource.servers.driver-class-name=com.mysql.jdbc.Driver
app.datasource.servers.url=jdbc:mysql://localhost:3306/v?autoReconnect=true&useSSL=false
app.datasource.servers.username=myuser
app.datasource.servers.password=mypass
If you need more information please see the following documentation:
Spring Documentation: howto-two-datasources
A similar example of how configure two different databases: github example
The answered provided by #Daniel C. is correct. Small correction/observation from my side.
#Primary is not required if you don't want to mark any datasource as
default one, otherwise necessary.
If you are defining any of the EntityManagerFactoryBean with #Bean name as entityManagerFactory then it's better to mark it #Primary to avoid conflict.
#ConfigurationProperties("app.datasource.servers")
can be marked at class level instead of defining at method level.
Better to return HikariDataSource as datasource if you using Spring
Boot 2.x or higher version as it has been changed.
Make sure you define exact property for jdbc-url which is being used by
HikariDataSource to refer JDBC Connection URL.
I just added a module aware multi database aware library for mysql in github.Some application properties need to be added and you are done .
Documentation and other details could be found at :-
https://github.com/yatharthamishra0419/spring-boot-data-multimodule-mysql

Spring Configuration Metadata

I am setting up two data sources as shown here at http://docs.spring.io/spring-boot/docs/1.3.0.M2/reference/htmlsingle/#howto-two-datasources using spring boot, but when doing so my application.properties shows warnings that for example x.x.username is an unknown property. This is correct to some extent as javax.sql.DataSource does not contain url, username, password, etc. but the implementation classes do. I have annotation processor set up and it works fine when working with concrete classes.
I notice that org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$NonEmbeddedConfiguration uses both DataSourceProperties and has #ConfigurationProperties annotated on dataSource(). This would probably get rid of my warnings but what is the point of this. Isn't it setting the properties twice this way?
Config:
#Bean
#Primary
#ConfigurationProperties(prefix="datasource.primary")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="datasource.secondary")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
Properties with warnings:
datasource.primary.url=jdbc:...
datasource.primary.username=user
datasource.primary.password=password
datasource.secondary.url=jdbc:...
datasource.secondary.username=user
datasource.secondary.password=password
Since someone bothered to +1 this question I thought I'd post a solution. Note that I think the #ConfigurationProperties on the DataSources themselves are unecessary because they are already set on the DataSourceProperties which is used to build the DataSource, but I left it in there because that's how the Spring team has done it in org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$NonEmbeddedConfiguration. My only guess why would be if your DataSource had additional properties that could be set other than what's exposed in DataSourceProperties, but then you would get warnings in the "Spring Boot application.properties editor" for those properties.
Note that DataSourceBuilder will use Tomcat, HikariCP or Commons DBCP in that order if found on Classpath as DataSource unless you specify something else with dataSourceBuilder.type(Class<? extends DataSource>)
Properties:
datasource.primary.url=jdbc:...
datasource.primary.username=user
datasource.primary.password=password
datasource.secondary.url=jdbc:...
datasource.secondary.username=user
datasource.secondary.password=password
Java Config:
#Bean
#Primary
#ConfigurationProperties(prefix = "datasource.primary")
public DataSourceProperties primaryProps() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties(prefix = "datasource.secondary")
public DataSourceProperties secondaryProps() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties(prefix = "datasource.primary")
public DataSource secondaryDataSource() {
DataSourceProperties props = secondaryProps();
return DataSourceBuilder.create(props.getClassLoader())
.driverClassName(props.getDriverClassName())
.url(props.getUrl())
.username(props.getUsername())
.password(props.getPassword())
.build();
}
#Bean
#ConfigurationProperties(prefix = "datasource.primary")
public DataSource secondaryDataSource() {
DataSourceProperties props = secondaryProps();
return DataSourceBuilder.create(props.getClassLoader())
.driverClassName(props.getDriverClassName())
.url(props.getUrl())
.username(props.getUsername())
.password(props.getPassword())
.build();
}

Resources