I have implemented spring data redis caching for my spring boot project. All my caches work except when I try to cache an item which contains a field of List<Enum> type (not the raw Enum class but any enum class I have created in my project eg. List<MyEnum>).
Caches work without a problem if there is an Enum field or a List field, but when there is a list of enums, upon requesting a cached response the json I receive abruptly ends at the enum list field.
I've even implemented a custom json converter for List<Enum> to List<String> conversion but it did not solve my problem.
Redis Config
#Configuration
#EnableCaching
public class RedisConfig {
#Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory connectionFactory) {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(connectionFactory);
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new GenericJackson2JsonRedisSerializer());
template.setHashKeySerializer(new StringRedisSerializer());
template.setHashValueSerializer(new GenericJackson2JsonRedisSerializer());
template.setEnableTransactionSupport(true);
template.afterPropertiesSet();
return template;
}
}
Expected JSON Response
{
"field1": "value1",
"list1": [
"element1",
"element2"
],
"enum1": "enumValue1",
"enumList": [
"enumValue2",
"enumValue3",
"enumValue4"
]
}
Received JSON Response
{
"field1": "value1",
"list1": [
"element1",
"element2"
],
"enum1": "enumValue1",
"enumList"
}
EDIT
Turns out that my List<Enum> fields are using PersistentBag from Hibernate instead of ArrayList. Right now using ArrayList<>(enumList) when assigning a response field seems to solve the problem. Are there any better solutions to this?
Related
I want to add Caching to my Spring Boot Backend. Saving the entries to the Cache seems to work since I can see the json list in Redis after my first request but once I send my second request (which would read the Cache) to the backend Spring throws an internal error and the request fails:
WARN 25224 --- [nio-8080-exec-2] .w.s.m.s.DefaultHandlerExceptionResolver :
Resolved [org.springframework.http.converter.HttpMessageNotWritableException:
Could not write JSON: java.lang.ClassCastException#291f1fc4;
nested exception is com.fasterxml.jackson.databind.JsonMappingException:
java.lang.ClassCastException#291f1fc4
(through reference chain: java.util.ArrayList[0]->java.util.LinkedHashMap["id"])]
My backend looks as it follows:
Config:
#Configuration
class RedisConfig {
#Bean
fun jedisConnectionFactory(): JedisConnectionFactory {
val jedisConnectionFactory = JedisConnectionFactory()
return jedisConnectionFactory
}
#Bean
fun redisTemplate(): RedisTemplate<String, Any> {
val myRedisTemplate = RedisTemplate<String, Any>()
myRedisTemplate.setConnectionFactory(jedisConnectionFactory());
return myRedisTemplate;
}
#Bean
fun cacheManager(): RedisCacheManager {
return RedisCacheManager.RedisCacheManagerBuilder.fromConnectionFactory(jedisConnectionFactory()).cacheDefaults(
RedisCacheConfiguration.defaultCacheConfig().disableCachingNullValues()
.serializeKeysWith(RedisSerializationContext.SerializationPair.fromSerializer(RedisSerializer.string()))
.serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(GenericJackson2JsonRedisSerializer(redisMapper())))
).build()
}
private fun redisMapper(): ObjectMapper {
return ObjectMapper() //.enableDefaultTyping(DefaultTyping.NON_FINAL, As.PROPERTY)
.setSerializationInclusion(JsonInclude.Include.NON_EMPTY)
}
}
Controller:
fun getPrivateRecipes(#RequestParam(required = false) langCode: String?): List<PrivateRecipeData> {
val lang = langCode ?: "en"
val userId = getCurrentUser().userRecord.uid
return privateRecipeCacheService.getPrivateRecipesCached(lang, userId)
}
Caching-Service
#Cacheable("privateRecipes")
fun getPrivateRecipesCached(lang: String, userId: String): List<PrivateRecipeData> {
return privateRecipeService.getPrivateRecipes(lang, userId)
}
I played around with the Cachable annotation, added keys, but it does not change the problem. The import and export of the list seems to be done with different classes. How to solve this?
In your ObjectMapper tell Jackson to use an ArrayList to hold collections of PrivateRecipeData instances like:
objectMapper.getTypeFactory().constructCollectionType(ArrayList.class, PrivateRecipeData.class);
One possible way to set it up:
One possible way is, in your cacheManager config, pass it a RedisTemplate configure with the right object mapper. Off the top of my head:
public RedisTemplate<String, Object> redisTemplate(...) {
Jackson2JsonRedisSerializer serializer = new ...
ObjectMapper objectMapper = new ObjectMapper();
//...
serializer.setObjectMapper(objectMapper);
RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
// ...
redisTemplate.setValueSerializer(serializer);
redisTemplate.afterPropertiesSet();
return redisTemplate;
}
What I did in the end was just using the default ObjectMapper (providing no arguments to GenericJackson2JsonRedisSerializer):
#Bean
fun cacheManager(): RedisCacheManager {
return RedisCacheManager.RedisCacheManagerBuilder.fromConnectionFactory(jedisConnectionFactory()).cacheDefaults(
RedisCacheConfiguration.defaultCacheConfig().disableCachingNullValues()
.serializeKeysWith(RedisSerializationContext.SerializationPair.fromSerializer(RedisSerializer.string()))
.serializeValuesWith(
RedisSerializationContext.SerializationPair.fromSerializer(
GenericJackson2JsonRedisSerializer()
)
)
).build()
}
The serialization in Json now looks a bit different (containing class names as well) but thats totally fine since it is still human readable :)
[
"java.util.ArrayList",
[
{
"#class": "com.my.package.data.PrivateRecipeData",
"id": 1,
"img_src": "user/xxxxxx/recipeImgs/32758c1c-35cf-4f92-9e8c-0057f4447d6c.jpg",
"name": "VeggieBurger",
"instructions": [
"java.util.ArrayList",
[
{
"id": 6,
"recipeId": 1,
}
]
],
...
}
],
...
I worked with spring boot and redis to caching.I can cache my data that fetch from database(oracle) use #Cacheable(key = "{#input,#page,#size}",value = "on_test").
when i try to fetch data from key("on_test::0,0,10") with redisTemplate the result is 0
why??
Redis Config:
#Configuration
public class RedisConfig {
#Bean
JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration("localhost", 6379);
redisStandaloneConfiguration.setPassword(RedisPassword.of("admin#123"));
return new JedisConnectionFactory(redisStandaloneConfiguration);
}
#Bean
public RedisTemplate<String,Objects> redisTemplate() {
RedisTemplate<String,Objects> template = new RedisTemplate<>();
template.setStringSerializer(new StringRedisSerializer());
template.setValueSerializer(new StringRedisSerializer());
template.setConnectionFactory(jedisConnectionFactory());
return template;
}
//service
#Override
#Cacheable(key = "{#input,#page,#size}",value = "on_test")
public Page<?> getAllByZikaConfirmedClinicIs(Integer input,int page,int size) {
try {
Pageable newPage = PageRequest.of(page, size);
String fromCache = controlledCacheService.getFromCache();
if (fromCache == null && input!=null) {
log.info("cache is empty lets initials it!!!");
Page<DataSet> all = dataSetRepository.getAllByZikaConfirmedClinicIs(input,newPage);
List<DataSet> d = redisTemplate.opsForHash().values("on_test::0,0,10");
System.out.print(d);
return all;
}
return null;
The whole point of using #Cacheable is that you don't need to be using RedisTemplate directly. You just need to call getAllByZikaConfirmedClinicIs() (from outside of the class it is defined in) and Spring will automatically check first if a cached result is available and return that instead of calling the function.
If that's not working, have you annotated one of your Spring Boot configuration classes with #EnableCaching to enable caching?
You might also need to set spring.cache.type=REDIS in application.properties, or spring.cache.type: REDIS in application.yml to ensure Spring is using Redis and not some other cache provider.
I see Lettuce can do compression for Redis serialized objects: https://lettuce.io/core/release/reference/#codecs.compression
Any way to set this config within Spring Boot Data LettuceConnectionFactory or in some other bean? I've seen this question asked here as well: https://github.com/lettuce-io/lettuce-core/issues/633
I'd like to compress all serialized objects being sent to Redis to reduce network traffic between boxes.
Thanks
I ended up solving it the following way.
Create a RedisTemplate Spring Bean. This allows us to set a custom serializer.
#Bean
public RedisTemplate<Object, Object> redisTemplate(LettuceConnectionFactory connectionFactory) {
RedisTemplate<Object, Object> template = new RedisTemplate<>();
template.setConnectionFactory(connectionFactory);
// Set a custom serializer that will compress/decompress data to/from redis
RedisSerializerGzip serializerGzip = new RedisSerializerGzip();
template.setValueSerializer(serializerGzip);
template.setHashValueSerializer(serializerGzip);
return template;
}
Create a custom serializer. I decided to extend JdkSerializationRedisSerializer since that's what Spring was using by default for Redis. I added a compress/decompress in each respected method and use the super class serialization code.
public class RedisSerializerGzip extends JdkSerializationRedisSerializer {
#Override
public Object deserialize(byte[] bytes) {
return super.deserialize(decompress(bytes));
}
#Override
public byte[] serialize(Object object) {
return compress(super.serialize(object));
}
////////////////////////
// Helpers
////////////////////////
private byte[] compress(byte[] content) {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
try (GZIPOutputStream gzipOutputStream = new GZIPOutputStream(byteArrayOutputStream)) {
gzipOutputStream.write(content);
} catch (IOException e) {
throw new SerializationException("Unable to compress data", e);
}
return byteArrayOutputStream.toByteArray();
}
private byte[] decompress(byte[] contentBytes) {
ByteArrayOutputStream out = new ByteArrayOutputStream();
try {
IOUtils.copy(new GZIPInputStream(new ByteArrayInputStream(contentBytes)), out);
} catch (IOException e) {
throw new SerializationException("Unable to decompress data", e);
}
return out.toByteArray();
}
}
Use spring dependency injection for the RedisTemplate Bean.
Here is a sample of a Java Spring Boot Redis Cluster Data configuration.
It is an implementation with Redis Cluster and Redis Cache Manager.
Snappy Compression
Kryo Serialization
Support ttl per cache key
Gradle configuration
spring-data-redis
snappy-java
kryo
commons-codec
Link to github https://github.com/cboursinos/java-spring-redis-compression-snappy-kryo
I am using a Redis cache (via the Jedis client), and I would like to use ValueOperations#multiGet, which takes a Collection of keys, and returns a List of objects from the cache, in the same order. My question is, what happens when some of the keys are in the cache, but others are not? I am aware that underneath, Redis MGET is used, which will return nil for any elements that are not in the cache.
I cannot find any documentation of how ValueOperations will interpret this response. I assume they will be null, and can certainly test it, but it would be dangerous to build a system around undocumented behavior.
For completeness, here is how the cache client is configured:
#Bean
public RedisConnectionFactory redisConnectionFactory() {
JedisConnectionFactory redisConnectionFactory = new JedisConnectionFactory();
redisConnectionFactory.setHostName(address);
redisConnectionFactory.setPort(port);
redisConnectionFactory.afterPropertiesSet();
return redisConnectionFactory;
}
#Bean
public ValueOperations<String, Object> someRedisCache(RedisConnectionFactory cf) {
RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(cf);
redisTemplate.setDefaultSerializer(new GenericJackson2JsonRedisSerializer());
redisTemplate.afterPropertiesSet();
return redisTemplate.opsForValue();
}
I am using spring-data-redis:2.1.4
So, is there any documentation around this, or some reliable source of truth?
After some poking around, it looks like the answer has something to do with the serializer used - in this case GenericJackson2JsonRedisSerializer. Not wanting to dig too much, I simply wrote a test validating that any (nil) values returned by Redis are convereted to null:
#Autowired
ValueOperations<String, SomeObject> valueOperations
#Test
void multiGet() {
//Given
SomeObject someObject = SomeObject
.builder()
.contentId("key1")
.build()
valueOperations.set("key1", someObject)
//When
List<SomeObject> someObjects = valueOperations.multiGet(Arrays.asList("key1", "nonexisting"))
//Then
assertEquals(2, someObjects.size())
assertEquals(someObject, someObjects.get(0))
assertEquals(null, someObjects.get(1))
}
So, in Redis, this:
127.0.0.1:6379> MGET "\"key1\"" "\"nonexisting\""
1) "{\"#class\":\"some.package.SomeObject\",\"contentId\":\"key1\"}"
2) (nil)
Will results in a List of {SomeObject, null}
In my REST controllers Spring project, I want to store Session information in Redis.
In my application.properties I have defined the following:
spring.session.store-type=redis
spring.session.redis.namespace=rdrestcore
com.xyz.redis.host=192.168.201.46
com.xyz.redis.db=0
com.xyz.redis.port=6379
com.xyz.redis.pool.min-idle=5
I also have enabled Http Redis Session with:
#Configuration
#EnableRedisHttpSession
public class SessionConfig extends AbstractHttpSessionApplicationInitializer
{}
I finally have a redis connection factory like this:
#Configuration
#EnableRedisRepositories
public class RdRedisConnectionFactory {
#Autowired
private Environment env;
#Value("${com.xyz.redis.host}")
private String redisHost;
#Value("${com.xyz.redis.db}")
private Integer redisDb;
#Value("${com.xyz.redis.port}")
private Integer redisPort;
#Value("${com.xyz.redis.pool.min-idle}")
private Integer redisPoolMinIdle;
#Bean
JedisPoolConfig jedisPoolConfig() {
JedisPoolConfig poolConfig = new JedisPoolConfig();
if(redisPoolMinIdle!=null) poolConfig.setMinIdle(redisPoolMinIdle);
return poolConfig;
}
#Bean
JedisConnectionFactory jedisConnectionFactory() {
JedisConnectionFactory jedisConFactory = new JedisConnectionFactory();
if(redisHost!=null) jedisConFactory.setHostName(redisHost);
if(redisPort!=null) jedisConFactory.setPort(redisPort);
if(redisDb!=null) jedisConFactory.setDatabase(redisDb);
jedisConFactory.setPoolConfig(jedisPoolConfig());
return jedisConFactory;
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
final RedisTemplate< String, Object > template = new RedisTemplate();
template.setConnectionFactory( jedisConnectionFactorySpring());
template.setKeySerializer( new StringRedisSerializer() );
template.setValueSerializer( new GenericJackson2JsonRedisSerializer() );
template.setHashKeySerializer(new StringRedisSerializer());
template.setHashValueSerializer( new GenericJackson2JsonRedisSerializer() );
return template;
}
}
With this configuration, the session information gets stored in Redis, but, it is serialized very strangely. I mean, the keys are readable, but the values stored are not (I query the information from a program called "Redis Desktop Manager")... for example... for a new session, I get a hash with key:
*spring:session:sessions:c1110241-0aed-4d40-9861-43553b3526cb*
and the keys this hash contains are: maxInactiveInterval, lastAccessedTime, creationTime, sessionAttr:SPRING_SECURITY_CONTEXT
but their values are all they coded like something similar to this:
\xAC\xED\x00\x05sr\x00\x0Ejava.lang.Long;\x8B\xE4\x90\xCC\x8F#\xDF\x02\x00\x01J\x00\x05valuexr\x00\x10java.lang.Number\x86\xAC\x95\x1D\x0B\x94\xE0\x8B\x02\x00\x00xp\x00\x00\x01b$G\x88*
(for the creationTime key)
and if I try to access this information from code, with the redisTemplate, it rises an exception like this one:
Exception occurred in target VM: Cannot deserialize; nested exception is
org.springframework.core.serializer.support.SerializationFailedException:
Failed to deserialize payload. Is the byte array a result of
corresponding serialization for DefaultDeserializer?; nested exception
is java.io.StreamCorruptedException: invalid stream header: 73657373
org.springframework.data.redis.serializer.SerializationException: Cannot deserialize; nested exception is
org.springframework.core.serializer.support.SerializationFailedException:
Failed to deserialize payload. Is the byte array a result of
corresponding serialization for DefaultDeserializer?; nested exception
is java.io.StreamCorruptedException: invalid stream header: 73657373
at org.springframework.data.redis.serializer.JdkSerializationRedisSerializer.deserialize(JdkSerializationRedisSerializer.java:82)
I think it is some kind of problem with the serialization/deserialization of the Spring session information, but I don't know what else to do to be able to control that.
Does anyone know what Im doing wrong?
Thank you
You're on the right track, your problem is serialization indeed. Try this configuration (configure your template with these serializers only):
template.setHashValueSerializer(new JdkSerializationRedisSerializer());
template.setHashKeySerializer(new StringRedisSerializer());
template.setKeySerializer(new StringRedisSerializer());
template.setDefaultSerializer(new JdkSerializationRedisSerializer());