Unable to set HikariCP max-pool-size - spring-boot

I'm using Spring Boot 2.0.3 release. I want to increase the maximum pool size of HikariCP which is 10 by default.
I tried changing it in applicaiton.properties with
spring.datasource.hikari.maximum-pool-size=200
but it is not working because in the logs it still show that max pool size is 10.
The reason I want to change is because I am somehow reaching that limit in staging and I have no idea what's causing it.

I faced similar issue recently (Spring Boot v2.1.3). Posting it here in case people bump into the same scenario.
Long story short: if you're initializing DataSource using #ConfigurationProperties, those properties don't seem to require hikari prefix for maximum-pool-size, unless I'm missing something. So spring.datasource.maximum-pool-size should work if you use #ConfigurationProperties(prefix = "spring.datasource").
Details: In my case I'm initializing DataSource myself using org.springframework.boot.jdbc.DataSourceBuilder, so that I could later use it in org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean:
#Bean
#Primary
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource primaryDataSource()
{
return DataSourceBuilder.create().build();
}
In this case property spring.datasource.hikari.maximum-pool-size taken from Common App Properties section in Spring Boot doc did not help. Neither did suggested above maximumPoolSize.
So I went and debugged Spring Boot code which lead me to org.springframework.boot.context.properties.bind.JavaBeanBinder and it's method bind. Value for property name for HikariDataSource setter setMaximumPoolSize was "maximum-pool-size", so just for sake of testing I renamed my property to be spring.datasource.maximum-pool-size and it finally worked.
Hope it helps. Please let me know in the comments if I missed something or took wrong path. Thanks!

As per HikariCP Github README it's maximumPoolSize so try using:
spring.datasource.hikari.maximumPoolSize = 200
But this will work only if you allow Spring Boot to create the DataSource. If you create the DataSource yourself Spring Boot properties have no effect.
Do note that 200 is a very high value that may negatively impact your database as each physical connection requires server resources. In most cases a lower value will yield better performance, see HikariCP wiki: About Pool Sizing.

Related

Configuration Bean in Quarkus

This is regarding CDI spec of quarkus. Would want to understand is there a configuration bean for quarkus? How does one do any sort of configuration in quarkus?
If I get it right the original question is about #Configuration classes that can contain #Bean definitions. If so then CDI producer methods and fields annotated with #javax.enterprise.inject.Produces are the corresponding alternative.
Application configuration is a completely different question though and Jay is right that the Quarkus configuration reference is the ultimate source of information ;-).
First of all reading how cdi spec of quarkus differs from spring is important.
Please refer this guide:
https://quarkus.io/guides/cdi-reference
The learnings from this guide is there is #Produces which is an alternative to #Configuration bean in Quarkus.
Let us take an example for libs that might require a configuration through code. Example: Microsoft Azure IOT Service Client.
public class IotHubConfiguration {
#ConfigProperty(name="iothub.device.connection.string")
String connectionString;
private static final Logger LOG = Logger.getLogger(IotHubConfiguration.class);
#Produces
public ServiceClient getIot() throws URISyntaxException, IOException {
LOG.info("Inside Service Client bean");
if(connectionString==null) {
LOG.info("Connection String is null");
throw new RuntimeException("IOT CONNECTION STRING IS NULL");
}
ServiceClient serviceClient = new ServiceClient(connectionString, IotHubServiceClientProtocol.AMQPS);
serviceClient.open();
LOG.info("opened Service Client Successfully");
return serviceClient;
}
For all libs vertically intergrated with quarkus application.properties can be used and then you will get a driver obj for that broker/dbs available directly through #Inject in your #applicationScoped/#Singleton bean So, Why is that?
To Simplify and Unify Configuration
To Make Sure no code is required for configuring anything i.e. database config, broker config , quarkus config etc.
This drastically reduces the amount of code written for configuring and also Junits needed to cover that code.
Let us take an example where kafka producer configuration needs to be added: in application.properties
kafka.bootstrap.servers=${KAFKA_BROKER_URL:localhost:9092}
mp.messaging.outgoing.incoming_kafka_topic_test.topic=${KAFKA_INPUT_TOPIC_FOR_IOT_HUB:input_topic1}
mp.messaging.outgoing.incoming_kafka_topic_test.connector=smallrye-kafka
mp.messaging.outgoing.incoming_kafka_topic_test.value.deserializer=org.apache.kafka.common.serialization.StringDeserializer
mp.messaging.outgoing.incoming_kafka_topic_test.key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
mp.messaging.outgoing.incoming_kafka_topic_test.health-readiness-enabled=true
For full blown project reference: https://github.com/JayGhiya/QuarkusExperiments/tree/initial_version_v1/KafkaProducerQuarkus
Quarkus References for Config:
https://quarkus.io/guides/config-reference
Example for reactive sql config: https://quarkus.io/guides/reactive-sql-clients
Now let us talk about a bonus feature that quarkus provides which improves developer experience by atleast an order of magnitude that is profile driven development and testing.
Quarkus provides three profiles:
dev - Activated when in development mode (i.e. quarkus:dev)
test - Activated when running tests
prod - The default profile when not running in development or test
mode
Let us just say that in the given example you wanted to have different topics for development and different topics for production. Let us achieve that!
%dev.mp.messaging.outgoing.incoming_kafka_topic_test.topic=${KAFKA_INPUT_TOPIC_FOR_IOT_HUB:input_topic1}
%prod.mp.messaging.outgoing.incoming_kafka_topic_test.topic=${KAFKA_INPUT_TOPIC_FOR_IOT_HUB:prod_topic}
This is how simple it is. This is extremely useful in cases where your deployments run with ssl enabled brokers/dbs etc and for dev purposes you have unsecure local brokers/dbs. This is a game changer.

Spring Boot - Hikari Database Configuration

Recently worked on migrating code to Spring Boot 2.x and am running into an issue configuring the datasource bean. We have two datasources and so we are building the DataSource beans by hand using a DataSourceBuilder.
Per the documentation , I am setting these properties:
spring.datasource.driver-class-name=org.h2.Driver
spring.datasource.url=jdbc:h2:mem:testdb
spring.datasource.username=sa
spring.datasource.password=
However, when I do that I get the following error:
java.lang.IllegalArgumentException: jdbcUrl is required with driverClassName.
at com.zaxxer.hikari.HikariConfig.validate(HikariConfig.java:1059) ~[HikariCP-2.7.9.jar:na]
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:109) ~[HikariCP-2.7.9.jar:na]
If I change my settings to do this:
spring.datasource.jdbcUrl=jdbc:mysql://localhost/test
then it works.
It also works for a single datasource if I comment out the bean that creates the datasource explicitly which leads me to believe there's a problem in the datasourcebuilder code.
An example repository with just a single data source showing the issue is available here: https://github.com/azizabah/hikari-issue
So what changed between SB 1.5.X and SB 2.X from a configuration stand point when having multiple data sources?
You are missing the configuration of DataSourceProperties and directly trying to wire it.
You need to follow the following documentation to achieve what you want to do in first place :
https://docs.spring.io/spring-boot/docs/current/reference/htmlsingle/#howto-two-datasources
Co-incidentally somebody did same mistake and raised as bug in git, if you want you can see the same documentation getting referred as solution:
https://github.com/spring-projects/spring-boot/issues/12758

Refresh Spring Boot Auto-Configured Properties

Can I dynamically refresh properties that are used by Spring Boot's auto configuration setup?
For example, I have the following properties set (via cloud config) to auto configure a dataSource:
spring.datasource.username=user1
spring.datasource.password=test
Now if I change the password prop on the config server, and hit the /refresh endpoint, I can see that the updated prop is retrieved but the DataSource is not refreshed.
I know I can manually configure the DataSource beans and make sure they fall under a RefreshScope, but I was hoping to find a way to mark the auto configured properties as "refreshable". I have some use cases where I'd want to refresh props used by Spring Boot for other beans besides DataSources, and setting up some of those beans manually could be a pain.
I think I spoke too soon, at least as far as my DataSource example goes. A new db connection was being created with the updated props.
Which makes sense especially when looking at the docs here
This didn't re-connect some of my spring.cloud.stream.bindings properties I had, but in that case I can probably solve the issue with #RefreshScope.
There's a configuration property to set in case of the Autoconfigured bean is immutable (don't change the properties after initialized)
You can put a list (set) of classes that you need to be refreshed and you don't have control over the source code, you can put them under the property: spring.cloud.refresh.extra-refreshable
e.g.:
spring
cloud
refresh
extra-refreshable:
- org.springframework.mail.javamail.JavaMailSenderImpl
see: https://cloud.spring.io/spring-cloud-static/Greenwich.SR1/single/spring-cloud.html#refresh-scope

GemFire - Spring Boot Configuration

I am working on a project that has a requirement of Pivotal GemFire.
I am unable to find a proper tutorial about how to configure gemFire with Spring Boot.
I have created a partitioned Region and I want to configure Locators as well, but I need only server-side configuration as client is handled by someone else.
I am totally new to Pivotal GemFire and really confused. I have tried creating a cache.xml but then somehow a cache.out.xml gets created and there are many issues.
#Priyanka-
Best place to start is with the Guides on spring.io. Specifically, have a look at...
"Accessing Data with GemFire"
There is also...
"Cache Data with GemFire", and...
"Accessing GemFire Data with REST"
However, these guides focus mostly on "client-side" application concerns, "data access" (over REST), "caching", etc.
Still, you can use Spring Data GemFire (in a Spring Boot application even) to configure a GemFire Server. I have many examples of this. One in particular...
"Spring Boot GemFire Server Example"
This example demonstrates how to bootstrap a Spring Boot application as a GemFire Server (technically, a peer node in the cluster). Additionally, the GemFire properties are specified Spring config and can use Spring's normal conventions (property placeholders, SpEL expression) to configure these properties, like so...
https://github.com/jxblum/spring-boot-gemfire-server-example/blob/master/src/main/java/org/example/SpringBootGemFireServer.java#L59-L84
This particular configuration makes the GemFire Server a "GemFire Manager", possibly with an embedded "Locator" (indicated by the start-locator GemFie property, not to be confused with the "locators" GemFire property which allows our node to join and "existing" cluster) as well as a GemFire CacheServer to serve GemFire cache clients (with a ClientCache).
This example creates a "Factorials" Region, with a CacheLoader (definition here) to populate the "Factorials" Region on cache misses.
Since this example starts an embedded GemFire Manager in the Spring Boot GemFire Server application process, you can even connect to it using Gfsh, like so...
gfsh> connect --jmx-manager=localhost[1099]
Then you can run "gets" on the "Factorial" Region to see it compute factorials of the numeric keys you give it.
To see more advanced configuration, have a look at my other repos, in particular the Contacts Application RI (here).
Hope this helps!
-John
Well, I had the same problem, let me share with you what worked for me, in this case I'm using Spring Boot and Pivotal GemFire as cache client.
Install and run GemFire
Read the 15 minutes quick start guide
Create a locator(let's call it locator1) and a server(server1) and a region(region1)
Go to the folder where you started the 'Gee Fish'(gfsh) and then go to the locator's folder and open the log file, in that file you can get the port your locator is using.
Now let's see the Spring boot side:
In you Application with the main method add the #EnablegemFireCaching annotation
In the method(wherever it is) you want to cache, add the #Cacheable("region1") annotation.
Now let's create a configuration file for the caching:
//this is my working class
#Configuration
public class CacheConfiguration {
#Bean
ClientCacheFactoryBean gemfireCacheClient() {
return new ClientCacheFactoryBean();
}
#Bean(name = GemfireConstants.DEFAULT_GEMFIRE_POOL_NAME)
PoolFactoryBean gemfirePool() {
PoolFactoryBean gemfirePool = new PoolFactoryBean();
gemfirePool.addLocators(Collections.singletonList(new ConnectionEndpoint("localhost", HERE_GOES_THE_PORT_NUMBER_FROM_STEP_4)));
gemfirePool.setName(GemfireConstants.DEFAULT_GEMFIRE_POOL_NAME);
gemfirePool.setKeepAlive(false);
gemfirePool.setPingInterval(TimeUnit.SECONDS.toMillis(5));
gemfirePool.setRetryAttempts(1);
gemfirePool.setSubscriptionEnabled(true);
gemfirePool.setThreadLocalConnections(false);
return gemfirePool;
}
#Bean
ClientRegionFactoryBean<Long, Long> getRegion(ClientCache gemfireCache, Pool gemfirePool) {
ClientRegionFactoryBean<Long, Long> region = new ClientRegionFactoryBean<>();
region.setName("region1");
region.setLookupEnabled(true);
region.setCache(gemfireCache);
region.setPool(gemfirePool);
region.setShortcut(ClientRegionShortcut.PROXY);
return region;
}
That's all!, also do not forget to serialize(implements Serializable) the class is being cached(The class your cached method is returning)

Reload property value when external property file changes ,spring boot

I am using spring boot, and I have two external properties files, so that I can easily change its value.
But I hope spring app will reload the changed value when it is updated, just like reading from files. Since property file is easy enough to meet my need, I hope I don' nessarily need a db or file.
I use two different ways to load property value, code sample will like:
#RestController
public class Prop1Controller{
#Value("${prop1}")
private String prop1;
#RequestMapping(value="/prop1",method = RequestMethod.GET)
public String getProp() {
return prop1;
}
}
#RestController
public class Prop2Controller{
#Autowired
private Environment env;
#RequestMapping(value="/prop2/{sysId}",method = RequestMethod.GET)
public String prop2(#PathVariable String sysId) {
return env.getProperty("prop2."+sysId);
}
}
I will boot my application with
-Dspring.config.location=conf/my.properties
I'm afraid you will need to restart Spring context.
I think the only way to achieve your need is to enable spring-cloud. There is a refresh endpoint /refresh which refreshes the context and beans.
I'm not quite sure if you need a spring-cloud-config-server (its a microservice and very easy to build) where your config is stored(Git or svn). Or if its also useable just by the application.properties file in the application.
Here you can find the doc to the refresh scope and spring cloud.
You should be able to use Spring Cloud for that
Add this as a dependency
compile group: 'org.springframework.cloud', name: 'spring-cloud-starter', version: '1.1.2.RELEASE'
And then use #RefreshScope annotation
A Spring #Bean that is marked as #RefreshScope will get special treatment when there is a configuration change. This addresses the problem of stateful beans that only get their configuration injected when they are initialized. For instance if a DataSource has open connections when the database URL is changed via the Environment, we probably want the holders of those connections to be able to complete what they are doing. Then the next time someone borrows a connection from the pool he gets one with the new URL.
Also relevant if you have Spring Actuator
For a Spring Boot Actuator application there are some additional management endpoints:
POST to
/env to update the Environment and rebind #ConfigurationProperties and log levels
/refresh for re-loading the boot strap context and refreshing the #RefreshScope beans
Spring Cloud Doc
(1) Spring Cloud's RestartEndPoint
You may use the RestartEndPoint: Programatically restart Spring Boot application / Refresh Spring Context
RestartEndPoint is an Actuator EndPoint, bundled with spring-cloud-context.
However, RestartEndPoint will not monitor for file changes, you'll have to handle that yourself.
(2) devtools
I don't know if this is for a production application or not. You may hack devtools a little to do what you want.
Take a look at this other answer I wrote for another question: Force enable spring-boot DevTools when running Jar
Devtools monitors for file changes:
Applications that use spring-boot-devtools will automatically restart
whenever files on the classpath change.
Technically, devtools is built to only work within an IDE. With the hack, it also works when launched from a jar. However, I may not do that for a real production application, you decide if it fits your needs.
I know this is a old thread, but it will help someone in future.
You can use a scheduler to periodically refresh properties.
//MyApplication.java
#EnableScheduling
//application.properties
management.endpoint.refresh.enabled = true
//ContextRefreshConfig.java
#Autowired
private RefreshEndpoint refreshEndpoint;
#Scheduled(fixedDelay = 60000, initialDelay = 10000)
public Collection<String> refreshContext() {
final Collection<String> properties = refreshEndpoint.refresh();
LOGGER.log(Level.INFO, "Refreshed Properties {0}", properties);
return properties;
}
//add spring-cloud-starter to the pom file.
Attribues annotated with #Value is refreshed if the bean is annotated with #RefreshScope.
Configurations annotated with #ConfigurationProperties is refreshed without #RefreshScope.
Hope this will help.
You can follow the ContextRefresher.refresh() code implements.
public synchronized Set<String> refresh() {
Map<String, Object> before = extract(
this.context.getEnvironment().getPropertySources());
addConfigFilesToEnvironment();
Set<String> keys = changes(before,
extract(this.context.getEnvironment().getPropertySources())).keySet();
this.context.publishEvent(new EnvironmentChangeEvent(context, keys));
this.scope.refreshAll();
return keys;
}

Resources