How to set autocommit to false in spring jdbc template - spring

Currently I'm setting autocommit to false in spring through adding a property to a datasource bean id like below :
<property name="defaultAutoCommit" value="false" />
But i need to add it specifically in a single java method before executing my procedure.
I used the below code snippet.
getJdbcTemplate().getDataSource().getConnection().setAutoCommit(false);
But the above line was not setting autocommit to false?
Am i missing anything ?
or any alternative to set autocommit in a specific java method by spring
Thanks

The problem is that you are setting autocommit on a Connection, but JdbcTemplate doesn't remember that Connection; instead, it gets a new Connection for each operation, and that might or might not be the same Connection instance, depending on your DataSource implementation. Since defaultAutoCommit is not a property on DataSource, you have two options:
Assuming your concrete datasource has a setter for defaultAutoCommit (for example, org.apache.commons.dbcp.BasicDataSource), cast the DataSource to your concrete implementation. Of course this means that you can no longer change your DataSource in your Spring configuration, which defeats the purpose of dependency injection.
((BasicDataSource)getJdbcTemplate().getDataSource()).setDefaultAutoCommit(false);
Set the DataSource to a wrapper implementation that sets AutoCommit to false each time you fetch a connection.
final DataSource ds = getJdbcTemplate().getDataSource();
getJdbcTemplate().setDataSource(new DataSource(){
// You'll need to implement all the methods, simply delegating to ds
#Override
public Connection getConnection() throws SQLException {
Connection c = ds.getConnection();
c.setAutoCommit(false);
return c;
}
});

You need to get the current connection. e.g.
Connection conn = DataSourceUtils.getConnection(jdbcTemplate.getDataSource());
try {
conn.setAutoCommit(false);
/**
* Your Code
*/
conn.commit();
} catch (SQLException e) {
conn.rollback();
e.printStackTrace();
}

I'm posting this because I was looking for it everywhere: I used configuration property in Spring boot to achieve setting the default autocommit mode with:
spring.datasource.hikari.auto-commit: false
Spring Boot 2.4.x Doc for Hikari

You will have to do for each statement that the jdbcTemplate executes. Because for each jdbcTemplate.execute() etc it gets a new connection from the Datasource's connection pool. So you will have to set it for the connection that the connection the jdbcTemplate uses for that query. So you will have to do something like
jdbcTemplate.execute("<your sql query", new PreparedStatementCallback<Integer>(){
#Override
public Integer doInPreparedStatement(PreparedStatement stmt) throws SQLException, DataAccessException
{
Connection cxn = stmt.getConnection();
// set autocommit for that cxn object to false
cxn.setAutoCommit(false);
// set parameters etc in the stmt
....
....
cxn.commit();
// restore autocommit to true for that cxn object. because if the same object is obtained from the CxnPool later, autocommit will be false
cxn.setAutoCommit(true);
return 0;
}
});
Hope this helps

after 5 years still a valid question, i resolved my issue in this way :
set a connection with connection.setAutoCommit(false);
create a jbc template with that connection;
do your work and commit.
Connection connection = dataSource.getConnection();
connection.setAutoCommit(false);
JdbcTemplate jdbcTemplate =
new JdbcTemplate(newSingleConnectionDataSource(connection, true));
// ignore case in mapping result
jdbcTemplate.setResultsMapCaseInsensitive(true);
// do your stuff
connection.commit();

I just came across this and thought the solution would help someone even if it's too late.
As Yosef said, the connection that you get by calling getJdbcTemplate().getDataSource().getConnection() method may or may not be the one used for the communication with database for your operation.
Instead, if your requirement is to just test your script, not to commit the data, you can have a Apache Commons DBCP datasource with auto commit set to fault. The bean definition is given below:
/**
* A datasource with auto commit set to false.
*/
#Bean
public DataSource dbcpDataSource() throws Exception {
BasicDataSource ds = new BasicDataSource();
ds.setUrl(url);
ds.setUsername(username);
ds.setPassword(password);
ds.setDefaultAutoCommit(false);
ds.setEnableAutoCommitOnReturn(false);
return ds;
}
// Create either JdbcTemplate or NamedParameterJdbcTemplate as per your needs
#Bean
public NamedParameterJdbcTemplate dbcpNamedParameterJdbcTemplate() throws Exception {
return new NamedParameterJdbcTemplate(dbcpDataSource());
}
And use this datasource for any such operations.
If you wish to commit your transactions, I suggest you to have one more bean of the datasource with auto commit set to true which is the default behavior.
Hope it helps someone!

I needed it to do some unit testing
In fact Spring already provides the SingleConnectionDataSource implementation with the setAutoCommit method
// import org.springframework.jdbc.datasource.SingleConnectionDataSource;
SingleConnectionDataSource dataSource = new SingleConnectionDataSource();
dataSourceRX71.setAutoCommit(false);
dataSourceRX71.setDriverClassName("xxx");
dataSourceRX71.setUrl("xxx");
dataSourceRX71.setUsername("xxx");
dataSourceRX71.setPassword("xxx");

In some case you could just add #Transactional in the method, e.g. After some batch insert, execute commit at last.

Related

Getting too many connection for role when using DataSource

I have a Rest service and when it gets it has to do some insertion and updation to almost 25 database. So when I tried like the below code, it was working in my localhost but when I deploy to my staging server I was getting FATAL: too many connections for role "user123"
List<String> databaseUrls = null;
databaseUrls.forEach( databaseUrl -> {
DataSource dataSource = DataSourceBuilder.create()
.driverClassName("org.postgresql.Driver")
.url(databaseUrl)
.username("user123")
.password("some-password")
.build();
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
jdbcTemplate.update("Some...Update...Query");
});
As per my understanding DataSource need not to be closed because it is never opened.
Note:
A DataSource implementation need not be closed, because it is never
“opened”. A DataSource is not a resource, is not connected to the
database, so it is not holding networking connections nor resources on
the database server. A DataSource is simply information needed when
making a connection to the database, with the database server's
network name or address, the user name, user password, and various
options you want specified when a connection is eventually made.
Can someone tell why I am getting this issue
The problem is in DataSourceBuilder, it actually creates of the connection pools which spawns some number of connections and keeps them running:
private static final String[] DATA_SOURCE_TYPE_NAMES = new String[] {
"org.apache.tomcat.jdbc.pool.DataSource",
"com.zaxxer.hikari.HikariDataSource",
"org.apache.commons.dbcp.BasicDataSource" };
Javadoc says:
/**
* Convenience class for building a {#link DataSource} with common implementations and
* properties. If Tomcat, HikariCP or Commons DBCP are on the classpath one of them will
* be selected (in that order with Tomcat first). In the interest of a uniform interface,
* and so that there can be a fallback to an embedded database if one can be detected on
* the classpath, only a small set of common configuration properties are supported. To
* inject additional properties into the result you can downcast it, or use
* <code>#ConfigurationProperties</code>.
*/
Try to use e.g. SingleConnectionDataSource, then your problem will gone:
List<String> databaseUrls = null;
Class.forName("org.postgresql.Driver");
databaseUrls.forEach( databaseUrl -> {
SingleConnectionDataSource dataSource;
try {
dataSource = new SingleConnectionDataSource(
databaseUrl, "user123", "some-password", true /*suppressClose*/);
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
jdbcTemplate.update("Some...Update...Query");
} catch (Exception e) {
log.error("Failed to run queries for {}", databaseUrl, e);
} finally {
// release resources
if (dataSource != null) {
dataSource.destroy();
}
}
});
First thing it is very bad architecture decision to have single application managing 50 database. Anyway instead of creating DataSource in for loop, you should make use of Factory Design pattern to create DataSource for each DB. You should add some connection pooling mechanism to your system . HijariCP and TomcatPool are most widely used. Analyse logs of failure thread for any further issues.

what is the purpose of setNestedTransactionAllowed in JpaTransactionManager

I use the spring boot 1.5.2 RELEASE.
JpaTransactionManager txManager = new JpaTransactionManager();
txManager.setEntityManagerFactory(ptvEntityManagerFactory);
txManager.setDataSource(ds);
txManager.setJpaDialect(hibernateDialect);
//txManager.setNestedTransactionAllowed(true);
so what does this NestedTransactionAllowed really do?
I create code like this:
#Transactional
public void testNestTransaction() {
saveToRepository()
saveToJdbcTemplate();
throw new RuntimeException();
}
#Transactional
private void saveToRepository() {
employeeRepository.save(new MyEntity(xxx,xx,xx));
}
private void saveToJdbcTemplate() {
transactionTemplate.execute(new TransactionCallbackWithoutResult() {
// the code in this method executes in a transactional context
protected void doInTransactionWithoutResult(TransactionStatus status) {
String sql = "INSERT INTO task (id,create_by,description) VALUES
(?,?,?)";
jdbcTemplate.batchUpdate(sql, new BatchPreparedStatementSetter().....
}
}
Here is the problem. no matter NestedTransactionAllowed is true or false, the runTimeException always rollback both in saveToRepository() and saveToJdbcTemplate(). it default value is false, and there is a chunk of JavaDoc to describ this flag.
But I still do not understand what is the point of the NestedTransactionAllowed??
can you guys helps me with some scenarios to show the difference between this value in true and false?
thanks a lot
BTW: the entity manager is hibernate.
// hibernate adapter
HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
Your setNestedTransactionAllowed is not working as nested transaction support is not available for JpaTransaactionManager. Following excerpt from the official doc -
This transaction manager supports nested transactions via JDBC 3.0
Savepoints. The "nestedTransactionAllowed" flag defaults to false
though, since nested transactions will just apply to the JDBC
Connection, not to the JPA EntityManager and its cached entity objects
and related context. You can manually set the flag to true if you want
to use nested transactions for JDBC access code which participates in
JPA transactions (provided that your JDBC driver supports Savepoints).
Note that JPA itself does not support nested transactions! Hence, do
not expect JPA access code to semantically participate in a nested
transaction.

Why is Ebean ORM throwing java.sql.SQLException: IJ031021: You cannot rollback during a managed transaction

I have a Jax-RS Rest service that uses Ebean to query the database. On any query I make this exception is thrown.
For example.
User currentUser = new QUser().where().id.eq(currentUserID)).findUnique();
Logs
ERROR [io.ebeaninternal.server.transaction.JdbcTransaction] (default task-10) Error when ending a query only transaction via ROLLBACK: java.sql.SQLException: IJ031021: You cannot rollback during a managed transaction
Now the query returns the appropriate user and doesn't interfere with the Jax-RS.
But I can't ignore the large code-smell
And the huge log that is created because it gets thrown on every query.
My Configuration
ServerConfig config = new ServerConfig();
config.setDataSource(ds);
config.setName("db");
config.setAutoCommitMode(false);
config.setDatabasePlatform(new PostgresPlatform());
config.setRegister(true);
config.setDefaultServer(true);
config.setTransactionRollbackOnChecked(true);
config.addPackage(User.class.getPackage().getName());
EbeanServer es = EbeanServerFactory.create(config);
When using ebean inside Java EE you need to configure the EbeanServer before it is used. A typical place to do it is in a #PostConstruct method in a #Startup #Singleton bean-managed transaction ejb. And you need to configure it to use the JTA transaction manager so it doesn't try to begin/commit the transactions on its own.
#Singleton
#Startup
#TransactionManagement(TransactionManagementType.BEAN)
public class AtStartup {
#Resource(mappedName = "java:jboss/datasources/EbeanTestDS")
private DataSource ds;
#SneakyThrows
#PostConstruct
public void startup() {
new MigrationRunner(new MigrationConfig()).run(ds); // begin/commits transaction for the migration...
ServerConfig config = new ServerConfig();
config.setDataSource(ds);
config.addPackage(Customer.class.getPackage().getName());
config.setUseJtaTransactionManager(true); // This is important !
config.setAutoCommitMode(false);
EbeanServerFactory.create(config);
}

jdbc connection pool using ThreadpoolExecutor in spring boot

I have an application that runs through multiple databases and for each database runs select query on all tables and dumps it to hadoop.
My design is to create one datasource connection at a time and use the connection pool obtained to run select queries in multiple threads. Once done for this datasource, close the connection and create new one.
Here is the Async code
#Component
public class MySampleService {
private final static Logger LOGGER = Logger
.getLogger(MySampleService.class);
#Async
public Future<String> callAsync(JdbcTemplate template, String query) throws InterruptedException {
try {
jdbcTemplate.query(query);
//process the results
return new AsyncResult<String>("success");
}
catch (Exception ex){
return new AsyncResult<String>("failed");
}
}
Here is the caller
public String taskExecutor() throws InterruptedException, ExecutionException {
Future<String> asyncResult1 = mySampleService.callAsync(jdbcTemplate,query1);
Future<String> asyncResult2 = mySampleService.callAsync(jdbcTemplate,query2);
Future<String> asyncResult3 = mySampleService.callAsync(jdbcTemplate,query3);
Future<String> asyncResult4 = mySampleService.callAsync(jdbcTemplate,query4);
LOGGER.info(asyncResult1.get());
LOGGER.info(asyncResult2.get());
LOGGER.info(asyncResult3.get());
LOGGER.info( asyncResult4.get());
//now all threads finished, close the connection
jdbcTemplate.getConnection().close();
}
I am wondering if this is a right way to do it or do any exiting/optimized solution that out of box I am missing. I can't use spring-data-jpa since my queries are complex.
Thanks
Spring Boot docs:
Production database connections can also be auto-configured using a
pooling DataSource. Here’s the algorithm for choosing a specific
implementation:
We prefer the Tomcat pooling DataSource for its performance and concurrency, so if that is available we always choose it.
Otherwise, if HikariCP is available we will use it.
If neither the Tomcat pooling datasource nor HikariCP are available and if Commons DBCP is available we will use it, but we
don’t recommend it in production.
Lastly, if Commons DBCP2 is available we will use it.
If you use the spring-boot-starter-jdbc or
spring-boot-starter-data-jpa ‘starters’ you will automatically get a
dependency to tomcat-jdbc.
So you should be provided with sensible defaults.

Spring JDBC connection without dataSource

I had read several articles on obtaining Connection using Spring DataSource.
But in our Company Setup, connection object is obtained through already configured environment. Following the sample code:
String pool = PropertyFileReader.getPropertyValue("database.properties", "development.connectionPool");
Connection connection = RequestUtils.getConnection(pool);
Hence, After reading this tutorial
I am confused on using JDBCTemplate using connection object from above code.
I believe JdbcTemplate is not designed to work against a Connection as what you expected. As a workaround, if you are fine to create a separate JdbcTemplate for each connection you created, you may wrap your connection in a thin wrapper of DataSource, and feed it to JdbcTemplate.
I think it should work but I haven't tried it anyway...
class SingleConnectionDataSource implements DataSource {
private Connection connection;
public SingleConnectionDataSource(Connection connection) {
this.connection = connection;
}
public Connection getConnection() {
return this.connection;
}
public Connection getConnection(String username, String password) {
return this.connection;
}
}
// at the place you want to use JdbcTemplate
Connection conn = blablabla; // your own way to get it
JdbcTemplate jdbcTemplate = new JdbcTemplate(new SingleConnectionDataSource(conn));
Actually, Spring already provided SingleConnectionDataSource implementation (have seen in version 4.1.7).
It is even allows you to supress connection closing by template.

Resources