Clear Redis Cache on Spring Boot Application Startup - spring

On application (spring boot service) startup, need to clear the Redis cache.
Redis is running in a different docker container with own volume mapping. Since it retains the old cache, the application picks the data from Redis cache instead of Database even after the application restarts
Tried #EventListener for ContextRefreshedEvent and it is never getting called.
Tried with #PostConstruct in ApplicationMain class, but it doesn't clear the cache.
Tried using #CacheEvict(allEntries = true), but still no luck
#Component
public class ApplicationStartUp {
#Autowired
private CacheManager cacheManager;
#EventListener()
public void onApplicationEvent(ContextStartedEvent event) {
cacheManager.getCacheNames()
.parallelStream()
.forEach(n -> cacheManager.getCache(n).clear());
}
}

I was successfully able to clear the cache with ApplicationReadyEvent. As the CacheManager bean is available by the time, the cache is getting cleared properly on startup
#Autowired
private CacheManager cacheManager;
#EventListener
public void onApplicationEvent(ApplicationReadyEvent event) {
cacheManager.getCacheNames()
.parallelStream()
.forEach(n -> cacheManager.getCache(n).clear());
}

For the Redis cache manager, if you want to clear the cache on boot, I think you will need to initialize the cache manager with a set of names. See the RedisCacheManagerBuilder docs
For example:
RedisCacheManagerBuilder.fromConnectionFactory(redisConnectionFactory)
.initialCacheNames(Set.of("cacheOne", "cacheTwo"))
.build();
Then you should be able to use #PostConstruct in you cache config class, for example.
#PostConstruct
public void clearCache() {
cacheManager.getCacheNames()
.parallelStream()
.forEach(n -> cacheManager.getCache(n).clear());
}

A simple practice to clear Redis DB at earlier life cycle stage. To config CacheManager #bean with argument RedisConnectionFactory passed in, calling flushDb() to delete all keys of the currently selected database before build up.
#Bean
public CacheManager cacheManager(RedisConnectionFactory redisConnectionFactory) {
redisConnectionFactory.getConnection().flushDb(); //Delete all keys of the currently selected database
return RedisCacheManager.builder(RedisCacheWriter.nonLockingRedisCacheWriter(redisConnectionFactory)).cacheDefaults(RedisCacheConfiguration.defaultCacheConfig()).build();
}

Related

Spring Boot Transaction support using #transactional annotation not working with mongoDB, anyone have solution for this?

Spring Boot version - 2.4.4,
mongodb version - 4.4.4
In my project, I want to do entry in 2 different document of mongodb, but if one fails than it should do rollback. mongodb supports transaction after version 4.0 but only if you have at least one replica set.
In my case I don't have replica set and also cannot create it according to my project structure. I can't use transaction support of mongodb because no replica-set. So, I am using Spring Transaction.
According to spring docs, to use transaction in Spring Boot, you only need to use #transactional annotation and everything will work(i.e. rollback or commit).
I tried many things from many sources but it is not rollbacking transaction if one fail.
Demo code is here,
This is demo code, not actual project.
This is my service class.
#Service
public class UserService {
#Autowired
UserRepository userRepository;
#Autowired
UserDetailRepository userDetailRepository;
#Transactional(rollbackFor = Exception.class)
public ResponseEntity<JsonNode> createUser(SaveUserDetailRequest saveUserDetailRequest) {
try {
User _user = userRepository.save(new User(saveUserDetailRequest.getId(), saveUserDetailRequest.getFirstName(), saveUserDetailRequest.getLastName()));
UserDetail _user_detail = userDetailRepository.save(new UserDetail(saveUserDetailRequest.getPhone(), saveUserDetailRequest.getAddress()));
} catch (Exception m) {
System.out.print("Mongo Exception");
}
return new ResponseEntity<>(HttpStatus.OK);
}
}
Also tried below code but still not working,
#EnableTransactionManagement
#Configuration
#EnableMongoRepositories({ "com.test.transaction.repository" })
#ComponentScan({"com.test.transaction.service"})
public class Config extends AbstractMongoClientConfiguration{
private com.mongodb.MongoClient mongoClient;
#Bean
MongoTransactionManager transactionManager(MongoDbFactory dbFactory) {
return new MongoTransactionManager(dbFactory);
}
#Bean
public com.mongodb.MongoClient mongodbClient() {
mongoClient = new com.mongodb.MongoClient("mongodb://localhost:27017");
return mongoClient;
}
#Override
protected String getDatabaseName() {
return "test";
}
}
The transaction support in Spring is only there to make things easier, it doesn't replace the transaction support for the underlying datastore being used.
In this case, it will simply delegate the starting/committing of a transaction to MongoDB. WHen using a database it will eventually delegate to the database etc.
As this is the case, the pre-requisites for MongoDB still need to be honoured and you will still need a replica.

Unable to get cache from cachemanager unless cachemanager.getCacheNames() is invoked

My application uses Spring 4.3.x, EhCache 3.6 and javax Cache 1.1.0.
Here is how I've configured javax CacheManager in my application:
<bean id="jCacheManager" class="org.springframework.cache.jcache.JCacheCacheManager">
<property name="cacheManager" ref="appCacheManagerFactoryBean" />
</bean>
<bean id="appCacheManagerFactoryBean" class="com.example.AppCacheManagerFactoryBean"/>
The AppCacheManagerFactoryBean (which is just a customized version of JCacheManagerFactoryBean) helps me to configure a global persistence directory for my app. Here is how it looks:
public class AppCacheManagerFactoryBean implements FactoryBean<CacheManager>, InitializingBean,
DisposableBean {
#Value("${cache.persistenceDir}")
private String persistenceDir;
private CacheManager cacheManager;
#Override
public void afterPropertiesSet() {
this.cacheManager = buildCacheManager();
}
private CacheManager buildCacheManager()
{
EhcacheCachingProvider cachingProvider = (EhcacheCachingProvider) Caching.getCachingProvider();
DefaultConfiguration defaultConfiguration = new DefaultConfiguration(cachingProvider.getDefaultClassLoader(),
new DefaultPersistenceConfiguration(new File(persistenceDir)));
return cachingProvider.getCacheManager(cachingProvider.getDefaultURI(), defaultConfiguration);
}
#Override
public CacheManager getObject() {
return this.cacheManager;
}
#Override
public Class<?> getObjectType() {
return (this.cacheManager != null ? this.cacheManager.getClass() : CacheManager.class);
}
#Override
public boolean isSingleton() {
return true;
}
#Override
public void destroy() {
this.cacheManager.close();
}
}
Here's how I define caches. I use Ehcache API to create my caches as some of the features my caches need are not available through JCache API.
EhcacheManager ehcacheManager = jCacheCacheManager.getCacheManager().unwrap(EhcacheManager.class);
ehcacheManager.createCache("foo", CacheConfigurationBuilder.newCacheConfigurationBuilder(
String.class, Foo.class,
ResourcePoolsBuilder.newResourcePoolsBuilder()
.heap(1)
.offheap(1, MemoryUnit.GB)
.disk(5, MemoryUnit.GB)
));
When I try to retrieve a cache from the CacheManager elsewhere in my app, a null pointer exception is thrown.
Caching.getCachingProvider().getCacheManager().getCache("foo");
However, if I retrieve the cache after invoking the getCacheNames() method in CacheManager, the cache is fetched normally.
Caching.getCachingProvider().getCacheManager().getCacheNames();
Caching.getCachingProvider().getCacheManager().getCache("foo");
What have I missed? Please help me.
My first question would be: "Why not using the built-in support of Spring-cache?" It won't need to do that. The JCacheCacheManager will take care of everything.
Then, your problem is that the cache is created directly in Ehcache without passing through the JSR107 layer. Calling getCacheNames() causes a refresh of the cache list in JSR107 to make it work. However, I'm not sure this global behavior is intended. But it is the way it works.
The solution is to create a cache as expected which is through the JSR107 layer. It looks like this
CacheManager cacheManager = jcacheCacheManager.getCacheManager();
cacheManager.createCache("foo",
Eh107Configuration.fromEhcacheCacheConfiguration(
CacheConfigurationBuilder.newCacheConfigurationBuilder(String.class, Foo.class, ResourcePoolsBuilder.newResourcePoolsBuilder()
.heap(1)
.offheap(1, MemoryUnit.GB)
.disk(5, MemoryUnit.GB))
.build()));

Null pointer exception using Autowired annotation - Gemfire Listerner

I have moved all the Cassandra into single class. When I tried create instance of CassandraOperations in the gemfire cache listener was getting null pointer exception.Can you please assist me on this error
I have not received any null pointer exception using spring and cassandra but getting while integrating with gemfire.
#Component
public class CacheListener<K, V> extends CacheListenerAdapter<K, V> implements Declarable {
#Autowired
private CassandraOperations cassandraOperations;
#Override
public void init(Properties props) {
}
public void afterCreate(EntryEvent e) {
cassandraOperations.insert(e.getNewValue());
}
#Override
public void close() {
}
}
public class CassandraConfig {
#Autowired
private Environment environment;
private static final Logger LOGGER = LoggerFactory.getLogger(CassandraConfig.class);
#Bean
public CassandraClusterFactoryBean cluster() {
CassandraClusterFactoryBean cluster = new CassandraClusterFactoryBean();
cluster.setContactPoints(environment.getProperty("cassandra.contactpoints"));
cluster.setPort(Integer.parseInt(environment.getProperty("cassandra.port")));
return cluster;
}
#Bean
public CassandraMappingContext mappingContext() {
BasicCassandraMappingContext mappingContext = new BasicCassandraMappingContext();
mappingContext.setUserTypeResolver(new SimpleUserTypeResolver(cluster().getObject(), environment.getProperty("cassandra.keyspace"))); return mappingContext;
}
#Bean
public CassandraConverter converter() {
return new MappingCassandraConverter(mappingContext());
}
#Bean
public CassandraSessionFactoryBean session() throws Exception {
CassandraSessionFactoryBean session = new CassandraSessionFactoryBean();
session.setCluster(cluster().getObject());
session.setKeyspaceName(environment.getProperty("cassandra.keyspace"));
session.setConverter(converter());
session.setSchemaAction(SchemaAction.NONE);
return session;
}
#Bean
public CassandraOperations cassandraTemplate() throws Exception {
return new CassandraTemplate(session().getObject());
}
}
Exception
[error 2017/05/05 11:16:04.874 CDT <http-nio-7878-exec-1> tid=0x5b] Exception occurred in CacheListener
java.lang.NullPointerException
at CacheListener.afterCreate(CacheListener.java:27)
at com.gemstone.gemfire.internal.cache.EnumListenerEvent$AFTER_CREATE.dispatchEvent(EnumListenerEvent.java:97)
at com.gemstone.gemfire.internal.cache.LocalRegion.dispatchEvent(LocalRegion.java:8897)
at com.gemstone.gemfire.internal.cache.LocalRegion.dispatchListenerEvent(LocalRegion.java:7376)
at com.gemstone.gemfire.internal.cache.LocalRegion.invokePutCallbacks(LocalRegion.java:6158)
at com.gemstone.gemfire.internal.cache.EntryEventImpl.invokeCallbacks(EntryEventImpl.java:1919)
at com.gemstone.gemfire.internal.cache.ProxyRegionMap$ProxyRegionEntry.dispatchListenerEvents(ProxyRegionMap.java:548)
at com.gemstone.gemfire.internal.cache.LocalRegion.basicPutPart2(LocalRegion.java:6012)
at com.gemstone.gemfire.internal.cache.ProxyRegionMap.basicPut(ProxyRegionMap.java:232)
at com.gemstone.gemfire.internal.cache.LocalRegion.virtualPut(LocalRegion.java:5824)
at com.gemstone.gemfire.internal.cache.LocalRegionDataView.putEntry(LocalRegionDataView.java:118)
at com.gemstone.gemfire.internal.cache.LocalRegion.basicPut(LocalRegion.java:5214)
at com.gemstone.gemfire.internal.cache.LocalRegion.validatedPut(LocalRegion.java:1597)
at com.gemstone.gemfire.internal.cache.LocalRegion.put(LocalRegion.java:1580)
at com.gemstone.gemfire.internal.cache.AbstractRegion.put(AbstractRegion.java:327)
at org.springframework.data.gemfire.GemfireTemplate.put(GemfireTemplate.java:189)
at org.springframework.data.gemfire.repository.support.SimpleGemfireRepository.save(SimpleGemfireRepository.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
What is not apparent in your code/configuration above is how you configured your application-specific, GemFire CacheListener using Spring (Data GemFire).
I see you annotated your application CacheListener using Spring's #Component stereo-type annotation, but this does nothing without help.
Are you using Spring's Classpath component scanning functionality, or perhaps Spring's Annotation-based container configuration support? If you are using the later, you know you have to still explicitly define your application CacheListener in config (JavaConfig or XML), right?
Whenever you encounter a NullPointerException on an #Autowired component/collaborator field to inject a dependency, especially when using Spring's #Autowired annotation, it is good indication you have a configuration problem, particularly since the #Autowired annotation implies that the "dependency" (e.g. CassandraOperations) is "required" (unless you explicitly set the required attribute of the #Autowired annotation to false, which you did not; required defaults to true).
Therefore, if the CacheListener component were picked up in the scan and a dependency could not be injected (auto-wired) because no (other) bean of the specified type (e.g. CassandraOperations) was defined in the Spring application context (which it is), then Spring would throw an Exception when evaluating your configuration class(es).
Although, even your CassandraConfig class must also be annotated with Spring's #Configuration annotation or with the #Component annotation when using either Spring Classpath component scanning or Annotation-based container config. Or, it must be explicitly defined as a bean in the Spring application context if using neither.
NOTE: the naming convention (i.e. CacheListener) is not very good since it clashes with GemFire's own CacheListener interface. It would be better to call your application-specific extension/implementation perhaps, "GemFireToCassandraCacheListener"
By way of example...
import ...;
#Configuration
class GemFireConfiguration {
#Bean
CacheFactoryBean gemfireCache() {
return new CacheFactoryBean();
}
#Bean("CassandraCache")
PartitionedRegionFactoryBean cassandraCacheRegion() {
PartitionedRegionFactoryBean cassandraCacheRegion =
new PartitionedRegionFactoryBean();
cassandraCacheRegion.setCache(gemfireCache());
cassandraCacheRegion.setClose(false);
cassandraCacheRegion.setCacheListeners(
new CacheListener[] { gemfireToCassandraCacheListener() });
return cassandraCacheRegion;
}
#Bean
GemFireToCassandraCacheListener gemfireToCassandraCacheListener() {
return new GemFireToCassandraCacheListener();
}
}
import ...;
#Configuration
class CassandraConfig {
// what you have above
}
I have plenty of GemFire configuration examples here, that shows GemFire native config with Spring (Data GemFire) config, XML vs. JavaConfig vs. annotations, etc, etc.
Finally...
Technically, it might be better to use a GemFire CacheWriter, attached to the Region, rather than a CacheListener, since what you are doing (updating Cassandra on a cache create) is the intended purpose of a CacheWriter.
Of course, the CacheListener is called "after" create vs. the CacheWriter which is "before" create. However, I would say it is always better to update the "primary" data source (or "source of truth") before updating the "cache" to reflect the data source. This is applicable especially if there are constraints in the primary data source that might cause an update to fail. You would not want the cache to be updated if the primary data source could not be.
A CacheWriter is configured similarly to a CacheListener, like so...
#Bean("CassandraCache")
PartitionedRegionFactoryBean cassandraCacheRegion() {
PartitionedRegionFactoryBean cassandraCacheRegion =
new PartitionedRegionFactoryBean();
cassandraCacheRegion.setCache(gemfireCache());
cassandraCacheRegion.setClose(false);
cassandraCacheRegion.setCacheWriter(gemfireToCassandraCacheWriter());
return cassandraCacheRegion;
}
#Bean
GemFireToCassandraCacheWriter gemfireToCassandraCacheWriter(
CassandraOperations cassandraOperations) {
return new GemFireToCassandraCacheWriter(cassandraOperations);
}
Where the GemFireToCassandraCacheWriter would be defined as...
class GemFireToCassandraCacheWriter extends CacheWriterAdapter {
private CassandraOperations cassandraOperations;
// Using constructor injection is better than field injection
GemFireToCassandraCacheWriter(CassandraOperations cassandraOperations) {
this.cassandraOperations = cassandraOperations;
}
public void beforeCreate(EntryEvent<?, ?> event) {
cassandraOperations.insert(event.getNewValue());
}
}
NOTE: a Region can only have 1 CacheWriter. FYI, functionally the CacheWriter is the counterpart to a CacheLoader. See the GemFire User Guide for more details. In particular, see here, here and here.
Additionally, if you are just using GemFire as a cache for state that is primarily managed in Cassandra, then you might also consider Spring's Cache Abstraction, for which Spring Data GemFire positions GemFire as a "provider" in the abstraction.
Not sure what your GemFire to Cassandra UC is all about, but food for thought.
Hope this helps!
-John

Ehcache local transactions with Spring #Transactional

I'm trying to setup a transactional ehcache, making use of Spring #Cacheable and #Transactional.
My caches work fine with #Cacheable, but as soon as i setup my cache to use a local transaction:
<cache name="currencyCodeMaps" maxElementsInMemory="100" overflowToDisk="false" timeToIdleSeconds="5" timeToLiveSeconds="600" memoryStoreEvictionPolicy="LRU" transactionalMode="local"/>
When I access the cache i get error:
net.sf.ehcache.transaction.TransactionException: transaction not started
even though the same method is annotated #Transactional.
My Spring transaction manager is:
org.springframework.orm.jpa.JpaTransactionManager
The ehcache documentation says local transactions are controlled explicitly:
Local transactions are not controlled by a Transaction Manager.
Instead there is an explicit API where a reference is obtained to a
TransactionController for the CacheManager using
cacheManager.getTransactionController() and the steps in the
transaction are called explicitly
But this will be hard, as I want to sync my ehcache transactions with DB transactions, and DB transactions are controlled by #Transactional.
Is there a way to get local Ehcache transactions to work with Spring #Transactional?
Yes, there is a way to achieve you goal.
Because you have 2 transactional resources (JTA and Ehcache) and do not use JTA you have to use compound transaction manager likeorg.springframework.data.transaction.ChainedTransactionManager from spring-data project
#Bean
public PlatformTransactionManager transactionManager() {
return new ChainedTransactionManager(ehcacheTransactionManager(), jpaTransactionManager());
}
#Bean
public EhcacheTransactionManager ehcacheTransactionManager() {
return new EhcacheTransactionManager(ehcacheManager().getTransactionController());
}
#Bean
public PlatformTransactionManager jpaTransactionManager() {
return new JpaTransactionManager(entityManagerFactory());
}
You need to specify which transaction manager should be use by default:
#Configuration
public class Configuration implements TransactionManagementConfigurer {
...
#Override
public PlatformTransactionManager annotationDrivenTransactionManager() {
return transactionManager();
}
...
}
EhcacheTransactionManager implementation
import net.sf.ehcache.TransactionController;
import net.sf.ehcache.transaction.local.LocalTransactionContext;
import org.springframework.transaction.TransactionDefinition;
import org.springframework.transaction.TransactionException;
import org.springframework.transaction.support.AbstractPlatformTransactionManager;
import org.springframework.transaction.support.DefaultTransactionStatus;
public class EhcacheTransactionManager extends AbstractPlatformTransactionManager {
private TransactionController transactionController;
public EhcacheTransactionManager(TransactionController transactionController) {
this.transactionController = transactionController;
}
#Override
protected Object doGetTransaction() throws TransactionException {
return new EhcacheTransactionObject(transactionController.getCurrentTransactionContext());
}
#Override
protected void doBegin(Object o, TransactionDefinition transactionDefinition) throws TransactionException {
int timeout = transactionDefinition.getTimeout();
if (timeout != TransactionDefinition.TIMEOUT_DEFAULT) {
transactionController.begin(timeout);
} else {
transactionController.begin();
}
}
#Override
protected void doCommit(DefaultTransactionStatus defaultTransactionStatus) throws TransactionException {
transactionController.commit();
}
#Override
protected void doRollback(DefaultTransactionStatus defaultTransactionStatus) throws TransactionException {
transactionController.rollback();
}
public class EhcacheTransactionObject {
private LocalTransactionContext currentTransactionContext;
public EhcacheTransactionObject(LocalTransactionContext currentTransactionContext) {
this.currentTransactionContext = currentTransactionContext;
}
}
}
source code and test case can be found here
This solution has a significant drawback transaction coordinator of ehcache does not support suspend/resume operations so inner transactions (PROPAGATION_REQUIRES_NEW) are not possible. That is why I had to find another one.
Another option is not to use local ehcache transactions at all and use org.springframework.cache.transaction.AbstractTransactionSupportingCacheManager#setTransactionAware which decorates caches to postpone operations until the transaction end. But it has following drawbacks:
Evicted keys stay accessible inside transaction until transaction commit
putIfAbsent operation is not postponed
It was a problem for me, so I implemented this functionality in different way. Check 'me.qnox.springframework.cache.tx.TxAwareCacheManagerProxy', there problems described above was solved, in the same repository
You do not want local transactions, you want XA transactions, which are supported by Ehcache.
Have a look at the documentation for Ehcache 2.10.x or Ehcache 2.8.x.

Caching objects in EJB 3.0

Which programming pattern (and product) I should use to create a generic in-memory object cache in EJB 3.0 stateless session bean?
Does use of static member variables or singleton pattern cause any side-effects in clustered environment, if cache is not required to be coherent?
Data to be cached is acquired from JDBC datasources (JPA is not an option) and web services. The amount of data to be cached is few megabytes. The cache must provide very fast reads and therefore I would prefer that the solution should be running inside the JVM that the application is using.
I guess the only solution is to create a singleton which you store as member variable.
#Stateless
public class MyBean {
private CacheManager cacheManager;
#PostConstruct
public void initialize() {
cacheManager = CacheManager.getInstance();
}
}
public class CacheManager {
private static CacheManager instance;
public synchronized getInstance() {
if (instance == null) {
instance = new CacheManager()
}
return instance;
}
}

Resources