I have a spring boot(1.4.3.RELEASE) application with MySQL as a backend. Using Spring Data with Oracle UCP and MySQL Java connection for connection pooling. Things are working fine.
For, UCP connection pooling, I create a data source with the below code.
#Bean(name = "appDataSource")
#Primary
public PoolDataSource appDataSource() {
this.pooledDataSource = PoolDataSourceFactory.getPoolDataSource();
final String databaseDriver = environment.getRequiredProperty("application.datasource.driverClassName");
final String databaseUrl = environment.getRequiredProperty("application.datasource.url");
final String databaseUsername = environment.getRequiredProperty("application.datasource.username");
final String databasePassword = environment.getRequiredProperty("application.datasource.password");
final String initialSize = environment.getRequiredProperty("application.datasource.initialSize");
final String maxPoolSize = environment.getRequiredProperty("application.datasource.maxPoolSize");
final String minPoolSize = environment.getRequiredProperty("application.datasource.minPoolSize");
// final String poolName =
// environment.getRequiredProperty("application.datasource.poolName");
try {
pooledDataSource.setConnectionFactoryClassName(databaseDriver);
pooledDataSource.setURL(databaseUrl);
pooledDataSource.setUser(databaseUsername);
pooledDataSource.setPassword(databasePassword);
pooledDataSource.setInitialPoolSize(Integer.parseInt(initialSize));
pooledDataSource.setMaxPoolSize(Integer.parseInt(maxPoolSize));
pooledDataSource.setMinPoolSize(Integer.parseInt(minPoolSize));
pooledDataSource.setSQLForValidateConnection(TEST_SQL);
pooledDataSource.setValidateConnectionOnBorrow(Boolean.TRUE);
pooledDataSource.setConnectionWaitTimeout(CONNECTION_WAIT_TIMEOUT_SECS);
// pooledDataSource.setConnectionPoolName(poolName);
}
catch (final NumberFormatException e) {
LOGGER.error("Unable to parse passed numeric value", e);
}
catch (final SQLException e) {
LOGGER.error("exception creating data pool", e);
}
LOGGER.info("Setting up datasource for user:{} and databaseUrl:{}", databaseUsername, databaseUrl);
return this.pooledDataSource;
}
Now, I am trying to move to C3P0 based connection pooling and hence added the below property in my application.properties
Configure the C3P0 database connection pooling module
spring.jpa.properties.hibernate.c3p0.max_size = 15
spring.jpa.properties.hibernate.c3p0.min_size = 6
spring.jpa.properties.hibernate.c3p0.timeout = 2500
spring.jpa.properties.hibernate.c3p0.max_statements_per_connection = 10
spring.jpa.properties.hibernate.c3p0.idle_test_period = 3000
spring.jpa.properties.hibernate.c3p0.acquire_increment = 3
spring.jpa.properties.hibernate.c3p0.validate = false
spring.jpa.properties.hibernate.c3p0.numHelperThreads = 15
and injecting Datasource a below.
#Bean
public ComboPooledDataSource dataSource() {
return new ComboPooledDataSource();
}
But, it is not working. Is there anything I am missing here.
Hope, C3P0 is better than UCP.
Thanks,
Related
I have some code that works properly on spring boot prior to version 2 and I find it hard to convert it to work with spring boot 2.
Can somebody assist?
yml file:
spring:
datasource:
type: org.apache.tomcat.jdbc.pool.DataSource
tomcat:
initial-size: 10
max-active: 20
max-idle: 10
max-wait: 10000
Old code:
DataSource dataSource = builder.build();
Map<String, Object> properties = null;
try {
RelaxedPropertyResolver relaxedProperty = new RelaxedPropertyResolver(environment);
properties = relaxedProperty
.getSubProperties("spring.datasource.tomcat.");
} catch (Exception e) {
// No need to log an error
}
MutablePropertyValues mutableProperties = new MutablePropertyValues(properties);
if(properties == null || properties.isEmpty()) {
return dataSource;
}
new RelaxedDataBinder(dataSource).bind(mutableProperties);
return dataSource;
New Code with version 2:
DataSourceBuilder<?> dataSourceBuilder =
DataSourceBuilder.create(this.getClass().getClassLoader());
javax.sql.DataSource dataSource = dataSourceBuilder.build();
Map<String, Object> properties = null;
try {
properties = Binder.get(environment)
.bind("spring.datasource.tomcat", Bindable.mapOf(String.class, Object.class)).orElseGet(Collections::emptyMap);
} catch (Exception e) {
// No need to log an error
}
if(properties == null || properties.isEmpty()) {
return dataSource;
}
return Binder.get(environment)
.bind("spring.datasource.tomcat", DataSource.class).get();
I am getting below error while binding data to DataSource.class. When I replace datasource to poolProperties class, it will able to bind but I am not able to bind this datasource.
Binder.get(environment)
.bind("spring.datasource.tomcat", PoolProperties.class).get();:
Error:
java.util.NoSuchElementException: No value bound
at org.springframework.boot.context.properties.bind.BindResult.get(BindResult.java:56)
I know how to connect to a database in usual way. What I need is to choose what database to connect at runtime.
I have a default database (used by the system) and many other options to user choose to aquire data. It implies in have no Entities or JPA mappings any way.
In older times I was using this:
try (Connection connection = DriverManager.getConnection(connectionString, user, password);
PreparedStatement preparedStatement = connection.prepareStatement(nativeQuery)) {
preparedStatement.setString( 1, coordinate );
try (ResultSet resultSet = preparedStatement.executeQuery()) {
while (resultSet.next())
result = resultSet.getString(RESULT_PARAM);
}
} catch (SQLException ex) {
CodeUtils.log(QUERY_ERROR_MSG, this);
CodeUtils.log(ex.getMessage(), this);
}
But I don't know how to port this to Spring Boot.
You can define database configuration as below-
#Configuration
public class MyDBConfig {
#Bean("db1Ds")
#Primary
#ConfigurationProperties("app.ds.db1")
public DataSource db1DataSource() {
return DataSourceBuilder.create().build();
}
#Bean("db2Ds")
#ConfigurationProperties("app.ds.db2")
public DataSource db2DataSource() {
return DataSourceBuilder.create().build();
}
#Bean("db1JdbcTemplate")
#Autowired
public JdbcTemplate db1JdbcTemplate(#Qualifier("db1Ds") DataSource ds) {
return new JdbcTemplate(ds);
}
#Bean("db2JdbcTemplate")
#Autowired
public JdbcTemplate db2JdbcTemplate(#Qualifier("db2Ds") DataSource ds) {
return new JdbcTemplate(ds);
}
}
Switch the Jdbc template based on what user selects.
Official Doc:
https://docs.spring.io/spring-boot/docs/current/reference/html/howto-data-access.html#howto-two-datasources
I have a method which has a UPDATE query and a select query. I annotated the method with #Transactional for the below use case.
For concurrent executions - if two users are updating the same table ,I need the update and select query to be run as a unit
If not using #Transactional , I am using jdbc template and i am trying to get the current connection set auto commit to false and commit to true at the end of the method
Issue 1:
Update is getting commited immediately after the statement is executed
Issue 2:
With jdbc template , unable to get the current connection used for transaction .
I have tried the below two ways to get the current connection , but it seems to be a new connection from the pool
1.Connection conn = DataSourceUtils.getConnection(template.getDataSource());
2.Connection con=template.getDataSource().getConnection();
Application deployed in WebLogic Server using using java configuration , created bean for jdbc template , datasource and transaction management and used autowiring
#Transactional
public Integer getNextId(String tablename) {
Integer nextId = null;
int swId = template.update("update " + tablename + " set swId = swId + 1");
//int swId1 = template.update("update " + tablename + " set swId = swId + 1");
if (swId == 1) {
nextId = template.queryForObject("select swId from " + tablename,
int.class);
}
return nextId;
}
}
#Scope("prototype")
#Bean
public DataSource dataSource() throws NamingException {
javax.naming.InitialContext ctx = new javax.naming.InitialContext();
DataSource dataSource = (javax.sql.DataSource) ctx.lookup(dsnName);
return dataSource;
}
#Scope("prototype")
#Bean
public DataSourceTransactionManager dataSourceTransactionManager(DataSource dataSource) {
DataSourceTransactionManager dataSourceTransactionManager = new DataSourceTransactionManager();
dataSourceTransactionManager.setDataSource(dataSource);
return dataSourceTransactionManager;
}
#Scope("prototype")
#Bean
public JdbcTemplate template(DataSource dataSource) {
JdbcTemplate template = new JdbcTemplate(dataSource);
return template;
}
Expected results.
Commit should happen after all the statements in the method is executed
jdbc template need to get the active connection sed for current transaction
I am using multi tenancy by schema in MySQL as,
class SchemaPerTenantConnectionProvider : MultiTenantConnectionProvider {
#Autowired
private lateinit var dataSource: DataSource
#Throws(SQLException::class)
override fun getAnyConnection() = this.dataSource.connection
#Throws(SQLException::class)
override fun releaseAnyConnection(connection: Connection) {
connection.close()
}
#Throws(SQLException::class)
override fun getConnection(tenantIdentifier: String): Connection {
val connection = this.anyConnection
try {
connection.createStatement().execute("USE $tenantIdentifier ")
} catch (e: SQLException) {
throw SQLException("Could not alter JDBC connection to schema [$tenantIdentifier]")
}
return connection
}
...
}
My connection pool size is 10, now if any invalid tenantIdentifier is passed 10 times, 10 connections are exhausted, after that application is unable to acquire any connection.
Tried throwing Exception, HibernateException and it didn't help. Using connection with default schema will fetch wrong results. Is there a way to handle this scenario in getConnection(), to not to exhaust connection limits?
Below configuration should work, overriding public void releaseConnection(String tenantIdentifier, Connection connection) will ensure connection get released back to the connection pool.
public class MultiTenantConnectionProviderImpl
implements MultiTenantConnectionProvider, Stoppable {
private final ConnectionProvider connectionProvider = ConnectionProviderUtils.buildConnectionProvider( "master" );
#Override
public Connection getAnyConnection() throws SQLException {
return connectionProvider.getConnection();
}
#Override
public void releaseAnyConnection(Connection connection) throws SQLException {
connectionProvider.closeConnection( connection );
}
#Override
public Connection getConnection(String tenantIdentifier) throws SQLException {
final Connection connection = getAnyConnection();
try {
connection.createStatement().execute( "USE " + tenanantIdentifier );
}
catch ( SQLException e ) {
throw new HibernateException(
"Could not alter JDBC connection to specified schema [" +
tenantIdentifier + "]",
e
);
}
return connection;
}
#Override
public void releaseConnection(String tenantIdentifier, Connection connection) throws SQLException {
try {
connection.createStatement().execute( "USE master" );
}
catch ( SQLException e ) {
// on error, throw an exception to make sure the connection is not returned to the pool.
// your requirements may differ
throw new HibernateException(
"Could not alter JDBC connection to specified schema [" +
tenantIdentifier + "]",
e
);
}
connectionProvider.closeConnection( connection );
}
...
}
Next, fine tuning the datasource configuration in spring boot:
# Number of ms to wait before throwing an exception if no connection is available.
spring.datasource.tomcat.max-wait=10000
# Maximum number of active connections that can be allocated from this pool at the same time.
spring.datasource.tomcat.max-active=50
Reference : Working with datasources
If the issue still persist, go ahead with datasource connection pooling mechanism support such as Hikari etc.
closing connection, in case of error solved the problem.
#Throws(SQLException::class)
override fun getConnection(tenantIdentifier: String): Connection {
val connection = this.anyConnection
try {
connection.createStatement().execute("USE $tenantIdentifier ")
} catch (e: SQLException) {
connection.close()
throw SQLException("Could not alter JDBC connection to schema [$tenantIdentifier]")
}
return connection
}
I am using a properties File to store some configuration properties, that are accessed this way:
#Value("#{configuration.path_file}")
private String pathFile;
Is it possible (with Spring 3) to use the same #Value annotation, but loading the properties from a database instead of a file ?
Assuming you have a table in your database stored key/value pairs:
Define a new bean "applicationProperties" - psuedo-code follows...
public class ApplicationProperties {
#AutoWired
private DataSource datasource;
public getPropertyValue(String key) {
// transact on your datasource here to fetch value for key
// SNIPPED
}
}
Inject this bean where required in your application. If you already have a dao/service layer then you would just make use of that.
Yes, you can keep your #Value annotation, and use the database source with the help of EnvironmentPostProcessor.
As of Spring Boot 1.3, we're able to use the EnvironmentPostProcessor to customize the application's Environment before application context is refreshed.
For example, create a class which implements EnvironmentPostProcessor:
public class ReadDbPropertiesPostProcessor implements EnvironmentPostProcessor {
private static final String PROPERTY_SOURCE_NAME = "databaseProperties";
private String[] CONFIGS = {
"app.version"
// list your properties here
};
#Override
public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
Map<String, Object> propertySource = new HashMap<>();
try {
// the following db connections properties must be defined in application.properties
DataSource ds = DataSourceBuilder
.create()
.username(environment.getProperty("spring.datasource.username"))
.password(environment.getProperty("spring.datasource.password"))
.url(environment.getProperty("spring.datasource.url"))
.driverClassName("com.mysql.jdbc.Driver")
.build();
try (Connection connection = ds.getConnection();
// suppose you have a config table storing the properties name/value pair
PreparedStatement preparedStatement = connection.prepareStatement("SELECT value FROM config WHERE name = ?")) {
for (int i = 0; i < CONFIGS.length; i++) {
String configName = CONFIGS[i];
preparedStatement.setString(1, configName);
ResultSet rs = preparedStatement.executeQuery();
while (rs.next()) {
propertySource.put(configName, rs.getString("value"));
}
// rs.close();
preparedStatement.clearParameters();
}
}
environment.getPropertySources().addFirst(new MapPropertySource(PROPERTY_SOURCE_NAME, propertySource));
} catch (Throwable e) {
throw new RuntimeException(e);
}
}
}
Finally, don't forget to put your spring.factories in META-INF. An example:
org.springframework.boot.autoconfigure.EnableAutoConfiguration=
com.baeldung.environmentpostprocessor.autoconfig.PriceCalculationAutoConfig
Although not having used spring 3, I'd assume you can, if you make a bean that reads the properties from the database and exposes them with getters.