Spring-data-redis connection pool not working as I expect - spring

This is my first application using spring-data-redis and I think I get the concepts pretty well (I've used JdbcTemplate many times with RDBMS-es in the past). Here's what's happening...
The problem I'm running into is that every time I do a get(key) operation (using a ValueOperations object) a connection is opened and closed which causes an approximately 1/10th second delay (this is server code, so 1/10th second is substantial). Here's the spring XML configuration:
<!-- Redis DAO stuff -->
<bean
id="jedisPoolConfig"
class="redis.clients.jedis.JedisPoolConfig"
p:testOnBorrow="true"
p:testOnReturn="true"
p:timeBetweenEvictionRunsMillis="60000"
p:minIdle="2"
p:maxTotal="30"
p:maxIdle="10"
/>
<bean id="jedisConnectionFactory"
class="org.springframework.data.redis.connection.jedis.JedisConnectionFactory"
p:host-name="${redis.url}"
p:port="${redis.port}"
p:database="0"
p:use-pool="true"
p:pool-config-ref="jedisPoolConfig"
/>
<bean id="stringRedisSerializer"
class="org.springframework.data.redis.serializer.StringRedisSerializer"
/>
<bean id="redisTemplate"
class="org.springframework.data.redis.core.RedisTemplate"
p:connection-factory-ref="jedisConnectionFactory"
p:defaultSerializer-ref="stringRedisSerializer"
/>
and here is the relevant Java code:
#Autowired
private RedisTemplate<String, String> template;
private ValueOperations<String, String> valOps;
#PostConstruct
public void init() {
logger.debug("111111111111111111111111aaaaaaaaaaaaaaaaaaaaaaa");
valOps = template.opsForValue();
}
public String getTestVal() {
logger.debug("getTestVal() function called++++++++++++++++++++");
Object testVal2 = valOps.get("akey");
testVal2 = valOps.get("akey");
testVal2 = valOps.get("akey");
testVal2 = valOps.get("akey");
testVal2 = valOps.get("akey");
testVal2 = valOps.get("akey");
logger.debug("TestVal2 returned from REdis: " + testVal2);
return null;
}
So the value for the same key is being retrieved six times. The log output that I see is as follows:
13:46:37.011 [http-nio-8080-exec-1] DEBUG com.arrow.pricing.dao.RedisDAO - getTestVal() function called++++++++++++++++++++
13:46:37.014 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Opening RedisConnection
13:46:37.344 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Closing Redis Connection
13:46:37.416 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Opening RedisConnection
13:46:37.543 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Closing Redis Connection
13:46:37.616 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Opening RedisConnection
13:46:37.742 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Closing Redis Connection
13:46:37.812 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Opening RedisConnection
13:46:37.940 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Closing Redis Connection
13:46:38.003 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Opening RedisConnection
13:46:38.128 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Closing Redis Connection
13:46:38.201 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Opening RedisConnection
13:46:38.337 [http-nio-8080-exec-1] DEBUG o.s.d.r.core.RedisConnectionUtils - Closing Redis Connection
13:46:38.414 [http-nio-8080-exec-1] DEBUG com.arrow.pricing.dao.RedisDAO - TestVal2 returned from REdis: yo mama
I thought I had followed the docs on how to set up a connection pool, but when dealing with a performance-oriented platform such as Redis, I would not expect this delay I'm seeing.
Thank you in advance for any assistance or hints.

Based on spring-data-redis-1.7.2.RELEASE
if(!isConnectionTransactional(conn, factory)) {
if (log.isDebugEnabled()) {
log.debug("Closing Redis Connection");
}
conn.close()
}
By the log, you could see "Closing Redis Connection" in line 205 and then connection is closed by call close method.
We can find that conn is implements RedisConnection.In this case, the actual class we used is JedisConnection.
see code start at line 256.
public void close() throws DataAccessExeception {
super.close();
// return the connection to the pool
if (pool != null) {
...balabala
}
}
The connection is back to the pool.
Now we know that RedisConnectionUtils alaways show the log "Closing Redis Connection" even used pool

Related

Hikari connection pool issue with sybase

I have a java spring boot application which does bulk insert data to sybase using Spring JDBC driver . I am using Sybase JDBC driver and uses DriverManagerDataSource and HikariConnectionPool to do the bulk insert. The Datasource creation goes fine and but when i try to creat the HikariConnectionPool its giving error Connection Closed error. Below is the code .As you see i am doing an explicit call to see if connecton is closed and it returns false. I am not ble to understand what i am missing here. TIA for any suggestions.
SOURCE CODE
#Configuration
#Profile("local")
public class SpringJDBCLocalConfiguration {
Logger logger = LoggerFactory.getLogger(SpringJDBCLocalConfiguration.class);
#Bean(name = "dataSource")
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource(){{
setDriverClassName("com.sybase.jdbc4.jdbc.SybDriver");
setUrl("url");
setUsername("username");
setPassword("passwd");
}};;
try {
Connection conn = dataSource.getConnection();
logger.info("SpringJDBCLocalConfiguration conn "+conn);
logger.info("SpringJDBCLocalConfiguration conn isClosed "+conn.isClosed());
}
catch(Exception exc){
logger.error("SpringJDBCLocalConfiguration error getting connection in SpringJDBCLocalConfiguration "+exc);
}
HikariConfig config = new HikariConfig() {{
setDataSource(dataSource);
setConnectionTestQuery("SELECT 1");
}};
HikariDataSource hikariDataSource = new HikariDataSource(config);
return hikariDataSource;
}
#Bean
public JdbcTemplate jdbcTemplate(#Qualifier("dataSource") DataSource dataSource) {
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource());
return jdbcTemplate;
}
}
ERROR STACK TRACE
[ main] c.j.a.w.e.d.SpringJDBCLocalConfiguration : inside SpringJDBCLocalConfiguration.dataSource method flushStrategy
[ main] com.zaxxer.hikari.HikariConfig : sybase-pool - idleTimeout is close to or more than maxLifetime, disabling it.
[ main] com.zaxxer.hikari.HikariDataSource : sybase-pool - Starting...
[ main] com.zaxxer.hikari.pool.PoolBase : sybase-pool - Driver does not support get/set network timeout for connections. (com.sybase.jdbc4.jdbc.SybConnection.getNetworkTimeout()I)
[ main] com.zaxxer.hikari.pool.HikariPool : sybase-pool - Exception during pool initialization.
java.sql.SQLException: JZ0C0: Connection is already closed.
at com.sybase.jdbc4.jdbc.ErrorMessage.raiseError(ErrorMessage.java:779) ~[jconn4-7.07-27307.jar!/:na]
at com.sybase.jdbc4.jdbc.SybConnection.checkConnection(SybConnection.java:4010) ~[jconn4-7.07-27307.jar!/:na]
at com.sybase.jdbc4.jdbc.SybStatement.close(SybStatement.java:744) ~[jconn4-7.07-27307.jar!/:na]
at com.sybase.jdbc4.jdbc.SybStatement.close(SybStatement.java:719) ~[jconn4-7.07-27307.jar!/:na]
at com.sybase.jdbc4.jdbc.MdaManager.loadMetaData(MdaManager.java:573) ~[jconn4-7.07-27307.jar!/:na]
at com.sybase.jdbc4.jdbc.MdaManager.<init>(MdaManager.java:188) ~[jconn4-7.07-27307.jar!/:na]
at com.sybase.jdbc4.jdbc.MdaManager.<init>(MdaManager.java:171) ~[jconn4-7.07-27307.jar!/:na]
at com.sybase.jdbc4.jdbc.SybConnection.checkMDA(SybConnection.java:4155) ~[jconn4-7.07-27307.jar!/:na]
at com.sybase.jdbc4.jdbc.SybConnection.getMDA(SybConnection.java:3685) ~[jconn4-7.07-27307.jar!/:na]
at com.sybase.jdbc4.tds.Tds.setOption(Tds.java:1829) ~[jconn4-7.07-27307.jar!/:na]
at com.sybase.jdbc4.jdbc.SybConnection.setReadOnly(SybConnection.java:2586) ~[jconn4-7.07-27307.jar!/:na]
at com.zaxxer.hikari.pool.PoolBase.setupConnection(PoolBase.java:409) ~[HikariCP-2.7.9.jar!/:na]
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:370) ~[HikariCP-2.7.9.jar!/:na]
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:194) ~[HikariCP-2.7.9.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:460) [HikariCP-2.7.9.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:534) [HikariCP-2.7.9.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:115) [HikariCP-2.7.9.jar!/:na]
at com.zaxxer.hikari.HikariDataSource.<init>(HikariDataSource.java:81) [HikariCP-2.7.9.jar!/:na]
The issue was application was running in cloud and able to establish the database connection but seems firewall killing the connection so at the time of connection pool creation no active connections were found causing this error. I tried explicitly whitelisting the ip address and port but still same issue,but the root cause is the firewall blocking connection.

Mono gets OnCancel instead of OnSuccessOrError after successful request

I have a SpringBoot Webflux app which implements a WebFilter to log success/error once a Publisher completes.
My Webfilter is similar to the following:
public class MyFilter implements WebFilter {
private static final Logger LOG = LoggerFactory.getLogger(MyFilter.class);
#Override
public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
return chain.filter(exchange)
.doOnCancel(() -> {
LOG.info("Cancelled");
})
.doOnSuccess(aVoid -> {
LOG.info("Success");
})
.doOnSuccessOrError((aVoid, throwable) -> {
if (throwable == null) {
LOG.info("Completed successfully");
} else {
LOG.error("An error occurred", throwable);
}
});
}
Since SpringBoot 2.1.5, the WebFilter does not work as expected anymore. Instead of getting the doOnSuccessOrError event I now get doOnCancel. This only happens when using Netty since version 0.8.7. When using Undertow, it works as expected.
In all cases the Web requests completes successfully, returning the correct data to the client.
Please see DEBUG/TRACE logs below:
[reactor-http-nio-3] DEBUG o.s.h.codec.json.Jackson2JsonEncoder - [6bfed744] Encoding [{"errors":[]}]
[reactor-http-nio-3] DEBUG r.n.http.server.HttpServerOperations - [id: 0x6bfed744, L:/0:0:0:0:0:0:0:1:8071 - R:/0:0:0:0:0:0:0:1:55927] Decreasing pending responses, now 0
[reactor-http-nio-3] DEBUG r.n.http.server.HttpServerOperations - [id: 0x6bfed744, L:/0:0:0:0:0:0:0:1:8071 - R:/0:0:0:0:0:0:0:1:55927] Last HTTP packet was sent, terminating the channel
[reactor-http-nio-3] TRACE r.netty.channel.ChannelOperations - [id: 0x6bfed744, L:/0:0:0:0:0:0:0:1:8071 - R:/0:0:0:0:0:0:0:1:55927] Disposing ChannelOperation from a channel
java.lang.Exception: ChannelOperation terminal stack
at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:391)
at reactor.netty.http.server.HttpServerOperations.cleanHandlerTerminate(HttpServerOperations.java:519)
at reactor.netty.http.server.HttpTrafficHandler.operationComplete(HttpTrafficHandler.java:314)
at reactor.netty.http.server.HttpTrafficHandler.operationComplete(HttpTrafficHandler.java:54)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:502)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:476)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:415)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:540)
at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:529)
at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:101)
at io.netty.util.internal.PromiseNotificationUtil.trySuccess(PromiseNotificationUtil.java:48)
at io.netty.channel.ChannelOutboundBuffer.safeSuccess(ChannelOutboundBuffer.java:715)
at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:270)
at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:350)
at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:411)
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:939)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:360)
at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:906)
at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1370)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:749)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:741)
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:727)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.flush(CombinedChannelDuplexHandler.java:533)
at io.netty.channel.ChannelOutboundHandlerAdapter.flush(ChannelOutboundHandlerAdapter.java:125)
at io.netty.channel.CombinedChannelDuplexHandler.flush(CombinedChannelDuplexHandler.java:358)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:749)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:741)
at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:727)
at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:127)
at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:749)
at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:764)
at io.netty.channel.AbstractChannelHandlerContext.write(AbstractChannelHandlerContext.java:789)
at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:757)
at io.netty.channel.AbstractChannelHandlerContext.writeAndFlush(AbstractChannelHandlerContext.java:812)
at io.netty.channel.DefaultChannelPipeline.writeAndFlush(DefaultChannelPipeline.java:1036)
at io.netty.channel.AbstractChannel.writeAndFlush(AbstractChannel.java:305)
...
[reactor-http-nio-3] INFO c.m.d.MyFilter - Cancelled
[reactor-http-nio-3] DEBUG r.n.channel.ChannelOperationsHandler - [id: 0x6bfed744, L:/0:0:0:0:0:0:0:1:8071 - R:/0:0:0:0:0:0:0:1:55927] No ChannelOperation attached. Dropping: EmptyLastHttpContent
This is a known issue in Spring Framework (see spring-framework#22952) and especially Reactor Netty (reactor-netty#741). There is no known workaround for that right now.

How can I confirm that Spring Transactions are working properly?

I working on an Spring 3.2 application running in OC4j. I have a number of methods annotated with Spring's #Transactional annotation as follows:
// in MyServiceImpl
#Transactional()
public void insertPaymentData(PaymentParams paymentParams) {
// call to MyDaoImpl which in turn calls a
// Spring JDBC Stored Procedure implementation
}
// in the Stored Procedure implementation
#Transactional(propagation = Propagation.MANDATORY)
public void insertPaymentData(PaymentParams paymentParams) {
// execute procedure here
}
In the logs for my local OC4J instance I see entries such as the following which make me think that Transactions are configured correctly:
[1.2.3.4-user123] 2019-03-20 17:36:14,805 DEBUG AbstractFallbackTransactionAttributeSource::getTransactionAttribute - Adding transactional method 'MyServiceImpl.insertPaymentData' with attribute: PROPAGATION_REQUIRED,ISOLATION_DEFAULT,timeout_1800; ''
[1.2.3.4-user123] 2019-03-20 17:36:14,809 DEBUG AbstractBeanFactory::doGetBean - Returning cached instance of singleton bean 'transactionManager'
[1.2.3.4-user123] 2019-03-20 17:36:14,812 DEBUG AbstractPlatformTransactionManager::getTransaction - Creating new transaction with name [my.app.service.payments.MyServiceImpl.insertPaymentData]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT,timeout_1800; ''
[1.2.3.4-user123] 2019-03-20 17:36:14,821 DEBUG AbstractBeanFactory::doGetBean - Returning cached instance of singleton bean 'transactionManager'
[1.2.3.4-user123] 2019-03-20 17:36:14,822 DEBUG AbstractPlatformTransactionManager::handleExistingTransaction - Participating in existing transaction
[1.2.3.4-user123] 2019-03-20 17:36:14,822 DEBUG DataSourceUtils::doGetConnection - Fetching JDBC Connection from DataSource
[1.2.3.4-user123] 2019-03-20 17:36:14,823 DEBUG DataSourceUtils::doGetConnection - Registering transaction synchronization for JDBC Connection
[1.2.3.4-user123] 2019-03-20 17:38:42,550 DEBUG DataSourceUtils::doReleaseConnection - Returning JDBC Connection to DataSource
[1.2.3.4-user123] 2019-03-20 17:38:42,551 DEBUG AbstractPlatformTransactionManager::processCommit - Initiating transaction commit
Sometimes I see timeout and rollback messages there too.
However when I deploy to the development server provided by the ops department I do not see any messages like this in the logs, although the DEBUG level messages are displayed there too. I added in the following logging lines which I found somewhere on stack overflow:
logger.debug("Transaction name=" + TransactionSynchronizationManager.getCurrentTransactionName());
logger.debug("isActualTransactionActive=" + TransactionSynchronizationManager.isActualTransactionActive());
logger.debug("isSynchronizationActive=" + TransactionSynchronizationManager.isSynchronizationActive());
In the logs I now see things like this:
Transaction name=my.app.service.payments.MyServiceImpl.insertPaymentData
isActualTransactionActive=true
isSynchronizationActive=true
Are these values from TransactionSynchronizationManager telling me anything useful?
Is it telling me that transactions are working fine and to stop worrying?
I have the following in a Spring configuration file:
#Configuration
#EnableTransactionManagement
public class TransactionConfig {
#Bean
public JtaTransactionManager transactionManager() {
return new JtaTransactionManager();
}
}
And I have the following in a spring xml config file:
<tx:annotation-driven transaction-manager="transactionManager" />
Thanks!

Setting Max Idle connections with Redis in Spring Boot

I have a Simple Spring Boot 1.5.x REST API application with Redis Caching. I haven't added any custom configuratators and Spring boot automatically gets configured with Jedis. Redis properties are as follows,
spring.cache.type = redis
spring.redis.host = localhost
spring.redis.port = 6379
spring.redis.pool.max-active = 10000
spring.redis.pool.max-idle = 9000
spring.redis.pool.min-idle = 9000
Spring Boot Application Class
#SpringBootApplication
#EnableCaching
public class DemoApplication {
public static void main(String[] args) {
pringApplication.run(DemoApplication.class, args);
}
}
Usage
#RestController
public class UserController {
#Autowired
private UserService userService;
#RequestMapping(value = "/api/users/{id}", method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
#Cacheable(value = "UserDto", key = "#id")
public UserDto getUser(#PathVariable("id") Integer id){
User u = userService.getUser(id);
UserDto dto = new UserDto(u.getId(), u.getFirstName(), u.getLastName());
return dto;
}
}
If I load an API Endpoint with JMeter and monitor the number of connections maintained to Redis with a netstat continuously I can see the following output.
Netstat command :
while [ true ] ; do sudo netstat -anp | grep EST|grep "127.0.0.1:6379 ESTABLISHED"|wc -l; done
The Output
9000
9000
9006
9016
9031
9019
9000
9015
9005
9027
9046
9011
9005
9013
9011
9054
9010
9094
9061
9091
9036
9010
9004
9003
8935
8826
8387
7910
7122
6631
6524
6420
6300
6190
5776
5482
5475
5293
5187
5118
5081
5020
4992
4817
3996
3485
3582
6504
8964
9031
9136
9000
9025
9035
9000
9000
As you can clearly see the number of connections is getting decreased at the middle and then again rises to 9000 which does not seem like the desirable result.
Additionally I can see the following DEBUG logs.
2018-06-11 14:20:18.010 DEBUG 4151 --- [io-8080-exec-44] o.s.d.redis.core.RedisConnectionUtils : Opening RedisConnection
2018-06-11 14:20:18.012 DEBUG 4151 --- [io-8080-exec-19] o.s.d.redis.core.RedisConnectionUtils : Closing Redis Connection
2018-06-11 14:20:18.015 DEBUG 4151 --- [io-8080-exec-19] o.s.d.redis.core.RedisConnectionUtils : Opening RedisConnection
2018-06-11 14:20:18.015 DEBUG 4151 --- [io-8080-exec-34] o.s.d.redis.core.RedisConnectionUtils : Closing Redis Connection
2018-06-11 14:20:18.015 DEBUG 4151 --- [io-8080-exec-34] o.s.d.redis.core.RedisConnectionUtils : Opening RedisConnection
2018-06-11 14:20:18.015 DEBUG 4151 --- [io-8080-exec-41] o.s.d.redis.core.RedisConnectionUtils : Closing Redis Connection
2018-06-11 14:20:18.015 DEBUG 4151 --- [io-8080-exec-41] o.s.d.redis.core.RedisConnectionUtils : Opening RedisConnection
2018-06-11 14:20:18.015 DEBUG 4151 --- [io-8080-exec-49] o.s.d.redis.core.RedisConnectionUtils : Closing Redis Connection
2018-06-11 14:20:18.016 DEBUG 4151 --- [io-8080-exec-49] o.s.d.redis.core.RedisConnectionUtils : Opening RedisConnection
2018-06-11 14:20:18.016 DEBUG 4151 --- [io-8080-exec-29] o.s.d.redis.core.RedisConnectionUtils : Closing Redis Connection
2018-06-11 14:20:18.016 DEBUG 4151 --- [io-8080-exec-29] o.s.d.redis.core.RedisConnectionUtils : Opening RedisConnection
2018-06-11 14:20:18.016 DEBUG 4151 --- [nio-8080-exec-3] o.s.d.redis.core.RedisConnectionUtils : Closing Redis Connection
2018-06-11 14:20:18.016 DEBUG 4151 --- [nio-8080-exec-3] o.s.d.redis.core.RedisConnectionUtils : Opening RedisConnection
2018-06-11 14:20:18.016 DEBUG 4151 --- [io-8080-exec-51] o.s.d.redis.core.RedisConnectionUtils : Closing Redis Connection
2018-06-11 14:20:18.016 DEBUG 4151 --- [io-8080-exec-51] o.s.d.redis.core.RedisConnectionUtils : Opening RedisConnection
2018-06-11 14:20:18.017 DEBUG 4151 --- [io-8080-exec-45] o.s.d.redis.core.RedisConnectionUtils : Closing Redis Connection
2018-06-11 14:20:18.017 DEBUG 4151 --- [io-8080-exec-45] o.s.d.redis.core.RedisConnectionUtils : Opening RedisConnection
Based on the code in GitHub this happens due to continuous calling of doGetConnection and releaseConnection methods.
Is this desirable? If so what is the explanation for seeing a connection count less than 9000?
If this is not accurate what are the things I can try?

Hibernate doesn't save edited entities, while saves new ones

I'm facing a weird problem. I'm currently writing a web application based on Spring-MVC 3.2, and Hibernate 4.1.9. I wrote a sample controller with its TestNG unit tests, and everything is fine except for editing. I can see that when saving a new object, it works like a charm, but if I try to edit an existing object, it doesn't get saved without giving out any reason (I'm calling the same method for adding and updating).
The log of adding a new object of type Application
14:26:36.636 [main] DEBUG o.s.t.w.s.TestDispatcherServlet - DispatcherServlet with name '' processing POST request for [/app/add.json]
14:26:36.637 [main] DEBUG o.s.w.s.m.m.a.RequestMappingHandlerMapping - Looking up handler method for path /app/add.json
14:26:36.650 [main] DEBUG o.s.w.s.m.m.a.RequestMappingHandlerMapping - Returning handler method [public java.lang.Long com.wstars.kinzhunt.platform.apps.web.AppController.createApp(com.wstars.kinzhunt.platform.model.apps.Application)]
14:26:36.651 [main] DEBUG o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance of singleton bean 'appController'
14:26:36.821 [main] DEBUG o.s.w.s.m.m.a.RequestResponseBodyMethodProcessor - Reading [class com.wstars.kinzhunt.platform.model.apps.Application] as "application/json" using [org.springframework.http.converter.json.MappingJacksonHttpMessageConverter#5dc6bb75]
14:26:36.890 [main] DEBUG o.s.o.hibernate3.SessionFactoryUtils - Opening Hibernate Session
14:26:36.890 [main] DEBUG org.hibernate.impl.SessionImpl - opened session at timestamp: 13580799968
14:26:36.892 [main] DEBUG c.w.c.dao.hibernate.BaseDaoHibernate - Saving or Updating Object: com.wstars.kinzhunt.platform.model.apps.Application#54fc519b[id=<null>,name=KinzHunt,company=com.wstars.kinzhunt.platform.model.apps.Company#151c2b4[id=1,name=KinzHunt],callbackUrl=http://www.kinzhunt.com/callback/,website=http://www.kinzhunt.com,senderEmail=no-reply#kinzhunt.com,logoUrl=<null>]
14:26:36.892 [main] DEBUG o.s.o.hibernate3.SessionFactoryUtils - Opening Hibernate Session
14:26:36.893 [main] DEBUG org.hibernate.impl.SessionImpl - opened session at timestamp: 13580799968
14:26:36.893 [main] DEBUG o.s.o.hibernate3.SessionFactoryUtils - Closing Hibernate Session
14:26:36.894 [main] DEBUG o.h.e.def.AbstractSaveEventListener - executing identity-insert immediately
14:26:36.898 [main] DEBUG org.hibernate.jdbc.AbstractBatcher - about to open PreparedStatement (open PreparedStatements: 0, globally: 0)
14:26:36.899 [main] DEBUG org.hibernate.jdbc.ConnectionManager - opening JDBC connection
14:26:36.899 [main] DEBUG o.s.j.d.DriverManagerDataSource - Creating new JDBC DriverManager Connection to [jdbc:h2:mem:platform_test;DB_CLOSE_DELAY=-1]
14:26:36.901 [main] DEBUG org.hibernate.SQL - /* insert com.wstars.kinzhunt.platform.model.apps.Application */ insert into applications (id, callback_url, company_id, logo_url, name, sender_email, website) values (null, ?, ?, ?, ?, ?, ?)
14:26:36.904 [main] DEBUG o.h.id.IdentifierGeneratorHelper - Natively generated identity: 2
14:26:36.904 [main] DEBUG org.hibernate.jdbc.AbstractBatcher - about to close PreparedStatement (open PreparedStatements: 1, globally: 1)
14:26:36.905 [main] DEBUG o.s.o.hibernate3.SessionFactoryUtils - Closing Hibernate Session
14:26:36.905 [main] DEBUG org.hibernate.jdbc.ConnectionManager - releasing JDBC connection [ (open PreparedStatements: 0, globally: 0) (open ResultSets: 0, globally: 0)]
14:26:36.905 [main] DEBUG org.hibernate.jdbc.ConnectionManager - transaction completed on session with on_close connection release mode; be sure to close the session to release JDBC resources!
14:26:36.926 [main] DEBUG o.s.w.s.m.m.a.RequestResponseBodyMethodProcessor - Written [2] as "application/json;charset=UTF-8" using [org.springframework.http.converter.json.MappingJacksonHttpMessageConverter#5dc6bb75]
14:26:36.927 [main] DEBUG o.s.t.w.s.TestDispatcherServlet - Null ModelAndView returned to DispatcherServlet with name '': assuming HandlerAdapter completed request handling
14:26:36.928 [main] DEBUG o.s.t.w.s.TestDispatcherServlet - Successfully completed request
While the log for saving an edited object is
14:27:03.398 [main] DEBUG o.s.t.w.s.TestDispatcherServlet - DispatcherServlet with name '' processing POST request for [/app/1/edit.json]
14:27:03.398 [main] DEBUG o.s.w.s.m.m.a.RequestMappingHandlerMapping - Looking up handler method for path /app/1/edit.json
14:27:03.401 [main] DEBUG o.s.w.s.m.m.a.RequestMappingHandlerMapping - Returning handler method [public java.lang.Long com.wstars.kinzhunt.platform.apps.web.AppController.editApp(com.wstars.kinzhunt.platform.model.apps.Application,java.lang.Long)]
14:27:03.401 [main] DEBUG o.s.b.f.s.DefaultListableBeanFactory - Returning cached instance of singleton bean 'appController'
14:27:03.404 [main] DEBUG o.s.w.s.m.m.a.RequestResponseBodyMethodProcessor - Reading [class com.wstars.kinzhunt.platform.model.apps.Application] as "application/json" using [org.springframework.http.converter.json.MappingJacksonHttpMessageConverter#5dc6bb75]
14:27:03.409 [main] DEBUG o.s.o.hibernate3.SessionFactoryUtils - Opening Hibernate Session
14:27:03.410 [main] DEBUG org.hibernate.impl.SessionImpl - opened session at timestamp: 13580800234
14:27:03.411 [main] DEBUG c.w.c.dao.hibernate.BaseDaoHibernate - Saving or Updating Object: com.wstars.kinzhunt.platform.model.apps.Application#1ba4f8a6[id=1,name=KinzHunt,company=com.wstars.kinzhunt.platform.model.apps.Company#6bc06877[id=1,name=KinzHunt],callbackUrl=http://www.kinzhunt.com/callback/,website=http://www.wstars.com/KinzHunt/,senderEmail=no-reply#kinzhunt.com,logoUrl=<null>]
14:27:03.412 [main] DEBUG o.s.o.hibernate3.SessionFactoryUtils - Opening Hibernate Session
14:27:03.412 [main] DEBUG org.hibernate.impl.SessionImpl - opened session at timestamp: 13580800234
14:27:03.413 [main] DEBUG o.s.o.hibernate3.SessionFactoryUtils - Closing Hibernate Session
14:27:03.422 [main] DEBUG o.s.o.hibernate3.SessionFactoryUtils - Closing Hibernate Session
14:27:03.424 [main] DEBUG o.s.w.s.m.m.a.RequestResponseBodyMethodProcessor - Written [1] as "application/json;charset=UTF-8" using [org.springframework.http.converter.json.MappingJacksonHttpMessageConverter#5dc6bb75]
14:27:03.424 [main] DEBUG o.s.t.w.s.TestDispatcherServlet - Null ModelAndView returned to DispatcherServlet with name '': assuming HandlerAdapter completed request handling
14:27:03.425 [main] DEBUG o.s.t.w.s.TestDispatcherServlet - Successfully completed request
As you can see, in the second log, no prepared statement is created, and no JDBC connection is opened. My test configuration for the database is like this:
<bean id="targetDataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="org.h2.Driver" />
<property name="url" value="jdbc:h2:mem:platform_test;DB_CLOSE_DELAY=-1" />
</bean>
<bean id="sessionFactory"
class="org.springframework.orm.hibernate3.annotation.AnnotationSessionFactoryBean">
<property name="dataSource" ref="targetDataSource" />
<property name="packagesToScan">
<list>
<value>com.mypackage.model.*</value>
</list>
</property>
<property name="namingStrategy">
<bean class="com.example.common.config.MyOwnNamingStrategy"/>
</property>
<property name="hibernateProperties">
<map>
<entry key="hibernate.dialect" value="org.hibernate.dialect.H2Dialect" />
<entry key="hibernate.max_fetch_depth" value="1" />
<entry key="hibernate.use_sql_comments" value="true" />
<entry key="hibernate.hbm2ddl.auto" value="update" />
</map>
</property>
<!-- <property key="hibernate.current_session_context_class" value="thread"/> -->
<!-- <property key="hibernate.transaction.factory_class" value="org.hibernate.transaction.JDBCTransactionFactory"/> -->
</bean>
<bean id="h2WebServer" class="org.h2.tools.Server"
factory-method="createWebServer" depends-on="targetDataSource"
init-method="start" lazy-init="false">
<constructor-arg value="-web,-webPort,11111" />
</bean>
My controller code is:
#Controller
public class AppController extends BaseAnnotatedController {
#Autowired
private AppManagementService appManagementService;
#RequestMapping(value="/app/add", method=RequestMethod.POST, consumes={"application/json"})
public #ResponseBody Long createApp(#RequestBody Application app) {
saveApp(app);
return app.getId();
}
#RequestMapping(value="/app/{appId}/edit", method=RequestMethod.POST, consumes={"application/json"})
public #ResponseBody Long editApp(#RequestBody Application app, #PathVariable Long appId) {
if (!appId.equals(app.getId())) {
WSError error = new WSError(ValidationErrorType.GENERIC, "id");
throw new ValidationException(error);
} else {
saveApp(app);
return app.getId();
}
}
#RequestMapping(value="/app/list", method=RequestMethod.GET)
public #ResponseBody List<Application> listApps() {
return appManagementService.listAllApps();
}
#RequestMapping(value="/app/{appId}/get", method=RequestMethod.GET)
public #ResponseBody Application getAppDetails(#PathVariable Long appId) {
return appManagementService.getApplication(appId);
}
private void saveApp(Application app) {
if (isValid(app)) {
appManagementService.saveApp(app);
}
}
public #ResponseBody List<Application> listAllApps() {
return appManagementService.listAllApps();
}
}
My test class is:
#Test
public class AppControllerIntegrationTests extends AbstractContextControllerTests {
private MockMvc mockMvc;
#Autowired
AppController appController;
#Autowired
private AppManagementService appManagementService;
#Autowired
private BaseDao baseDao;
#BeforeClass
public void classSetup() {
Company comp = new Company();
comp.setName("Some Company");
baseDao.saveObject(comp);
}
#BeforeMethod
public void setup() throws Exception {
this.mockMvc = webAppContextSetup(this.wac).build();
}
#Test
public void testAddInvalidAppWebJson() throws Exception {
String appJson = getInvalidApp().toJsonString();
RequestBuilder requestBuilder = MockMvcRequestBuilders.post("/app/add.json")
.contentType(MediaType.APPLICATION_JSON).content(appJson);
ResultActions resultAction = this.mockMvc.perform(requestBuilder);
resultAction.andExpect(status().isForbidden());
MvcResult mvcResult = resultAction.andReturn();
Exception resolvedException = mvcResult.getResolvedException();
assertTrue(resolvedException instanceof ValidationException);
ValidationException validationException = (ValidationException) resolvedException;
assertEquals(validationException.getErrors().size(), 3);
}
#Test
public void testAddAppWebJson() throws Exception {
Application app = getMockApp();
String appJson = app.toJsonString();
RequestBuilder requestBuilder = MockMvcRequestBuilders.post("/app/add.json")
.contentType(MediaType.APPLICATION_JSON).content(appJson);
this.mockMvc.perform(requestBuilder).andExpect(status().isOk());
}
#Test
public void testEditAppWithWrongIdWebJson() throws Exception {
String appJson = getMockAppWithId().toJsonString();
RequestBuilder requestBuilder = MockMvcRequestBuilders.post("/app/2/edit.json")
.contentType(MediaType.APPLICATION_JSON).content(appJson);
this.mockMvc
.perform(requestBuilder)
.andExpect(status().isForbidden())
.andExpect(
content()
.string("{\"errors\":[{\"errorType\":\"-100\",\"field\":\"id\",\"constraint\":null}],\"objects\":null}"));
}
#Test(dependsOnMethods={"testAddApp", "testAddAppWebJson"})
public void testEditAppWebJson() throws Exception {
Application app = getMockAppWithId();
setAppWebsiteToDifferentOne(app);
String appJson = app.toJsonString();
RequestBuilder requestBuilder = MockMvcRequestBuilders.post("/app/1/edit.json")
.contentType(MediaType.APPLICATION_JSON).content(appJson);
this.mockMvc.perform(requestBuilder).andExpect(status().isOk());
}
#Test
public void testEditInvalidAppWebJson() throws Exception {
Application app = getMockAppWithId();
app.setWebsite("Saba7o 3asal");
String appJson = app.toJsonString();
RequestBuilder requestBuilder = MockMvcRequestBuilders.post("/app/1/edit.json")
.contentType(MediaType.APPLICATION_JSON).content(appJson);
this.mockMvc
.perform(requestBuilder)
.andExpect(status().isForbidden())
.andExpect(
content()
.string("{\"errors\":[{\"errorType\":\"-114\",\"field\":\"website\",\"constraint\":null}],\"objects\":null}"));
}
#Test(dependsOnMethods="testEditAppWebJson")
public void testGetAppDetails() throws Exception {
RequestBuilder requestBuilder = MockMvcRequestBuilders.get("/app/1/get.json");
Application app = getMockAppWithId();
setAppWebsiteToDifferentOne(app);
this.mockMvc.perform(requestBuilder).andExpect(status().isOk())
.andExpect(content().string(app.toJsonString()));
}
}
My service method is:
#Override
#Transactional(readOnly=false)
public void saveApp(Application app) {
baseDao.saveObject(app);
}
All test methods pass, except for the last one, since it's expecting the website of the app to be the one which was edited. Where have I gone wrong?
Need to know which hibernate method is called in saveApp() also make sure your service is annotated with #Transactional.

Resources