Scenario : I need to modify maxAttempts value of #Retryable at runtime, so that the number of retries can be driven from database
#Service
public class PropertyHolder {
private int retryCount= 2;
public int retryCount() { return retryCount; }
#Scheduled(fixedRate=3600000) //Query DB and update retryCount every 60mins
private void updateRetryCount(int in) {
this.retryCount = 0; //Fetch retryCount from DB and update retryCount
}
}
#Service
public class SimpleService{
#Retryable( value={ Throwable.class }, maxAttemptsExpression="#{#propertyHolder.retryCount()}")
private void performTask() {
// do some opertion that throws Exception
}
}
PropertyHolder will update retryCount once in every 60 minutes.
This PropertHolder#retryCount needs to be wired to #Retryable in SimpleService#performTask .At present, it takes only the initial value of retryCount (2).Is this a right approach or Am I making some terrible mistake?
No; currently the expression is only evaluated during context initialization; there is an open feature request to add runtime evaluation.
https://github.com/spring-projects/spring-retry/issues/184
Currently you have to wire up your own interceptor with a mutable retry policy and configure it via the interceptor property in #Retryable.
Related
Below is my requirement
begin trans
insertData();
updateData();
end trans
Also lets say insertDta method throws some error then I need to retry 5 times .Same with updateData().I should not retry both the methods at the same time i.e if i retry m2() 5 times them m1() should not be retried.
Is below the correct way to do it? I call m3() from another class .
My concern is that interceptors are added in correct and deterministic order.
#Repository
DaoA
{
void insertData();
}
#Repository
DaoB
{
void updateData();
}
Below is my Service class .
#Service
ServiceA
{
#Retryable( maxAttempts = 5)
public void m1 ()
{
daoA.insertData();
}
#Retryable( maxAttempts = 5)
public void m2 ()
{
daoB.updateData();
}
#Transactional
public void m3 ()
{
m1();
m2();
}
m3() needs to be in a different bean - calling m1() and m2() directly within the class bypasses the proxy and they won't be retried.
In any case, the transaction should be within the retry logic, not the other way around; you need to start a new transaction for each attempt.
If I got your requirement right, this should work for you.
#Service
ServiceA {
public void m1 () {
daoA.insertData();
}
public void m2 () {
daoB.updateData();
}
#Transactional
#Retryable(value = {Exception.class}, maxAttempts = 5)
public void m3 () {
m1();
m2();
}
}
This will make sure the total number of retries is maxAttempts = 5.
I am trying to cache Kafka Records within 3 minutes of interval post that it will get expired and removed from the cache.
Each incoming records which is fetched using kafka consumer written in springboot needs to be updated in cache first then if it is present i need to discard the next duplicate records if it matches the cache record.
I have tried using Caffeine cache as below,
#EnableCaching
public class AppCacheManagerConfig {
#Bean
public CacheManager cacheManager(Ticker ticker) {
CaffeineCache bookCache = buildCache("declineRecords", ticker, 3);
SimpleCacheManager cacheManager = new SimpleCacheManager();
cacheManager.setCaches(Collections.singletonList(bookCache));
return cacheManager;
}
private CaffeineCache buildCache(String name, Ticker ticker, int minutesToExpire) {
return new CaffeineCache(name, Caffeine.newBuilder().expireAfterWrite(minutesToExpire, TimeUnit.MINUTES)
.maximumSize(100).ticker(ticker).build());
}
#Bean
public Ticker ticker() {
return Ticker.systemTicker();
}
}
and my Kafka Consumer is as below,
#Autowired
CachingServiceImpl cachingService;
#KafkaListener(topics = "#{'${spring.kafka.consumer.topic}'}", concurrency = "#{'${spring.kafka.consumer.concurrentConsumers}'}", errorHandler = "#{'${spring.kafka.consumer.errorHandler}'}")
public void consume(Message<?> message, Acknowledgment acknowledgment,
#Header(KafkaHeaders.RECEIVED_TIMESTAMP) long createTime) {
logger.info("Recieved Message: " + message.getPayload());
try {
boolean approveTopic = false;
boolean duplicateRecord = false;
if (cachingService.isDuplicateCheck(declineRecord)) {
//do something with records
}
else
{
//do something with records
}
cachingService.putInCache(xmlJSONObj, declineRecord, time);
and my caching service is as below,
#Component
public class CachingServiceImpl {
private static final Logger logger = LoggerFactory.getLogger(CachingServiceImpl.class);
#Autowired
CacheManager cacheManager;
#Cacheable(value = "declineRecords", key = "#declineRecord", sync = true)
public String putInCache(JSONObject xmlJSONObj, String declineRecord, String time) {
logger.info("Record is Cached for 3 minutes interval check", declineRecord);
cacheManager.getCache("declineRecords").put(declineRecord, time);
return declineRecord;
}
public boolean isDuplicateCheck(String declineRecord) {
if (null != cacheManager.getCache("declineRecords").get(declineRecord)) {
return true;
}
return false;
}
}
But Each time a record comes in consumer my cache is always empty. Its not holding the records.
Modifications Done:
I have added Configuration file as below after going through the suggestions and more kind of R&D removed some of the earlier logic and now the caching is working as expected but duplicate check is failing when all the three consumers are sending the same records.
`
#Configuration
public class AppCacheManagerConfig {
public static Cache<String, Object> jsonCache =
Caffeine.newBuilder().expireAfterWrite(3, TimeUnit.MINUTES)
.maximumSize(10000).recordStats().build();
#Bean
public CacheLoader<Object, Object> cacheLoader() {
CacheLoader<Object, Object> cacheLoader = new CacheLoader<Object, Object>() {
#Override
public Object load(Object key) throws Exception {
return null;
}
#Override
public Object reload(Object key, Object oldValue) throws Exception {
return oldValue;
}
};
return cacheLoader;
}
`
Now i am using the above cache as manual put and get.
I guess you're trying to implement records deduplication for Kafka.
Here is the similar discussion:
https://github.com/spring-projects/spring-kafka/issues/80
Here is the current abstract class which you may extend to achieve the necessary result:
https://github.com/spring-projects/spring-kafka/blob/master/spring-kafka/src/main/java/org/springframework/kafka/listener/adapter/AbstractFilteringMessageListener.java
Your caching service is definitely incorrect: Cacheable annotation allows marking the data getters and setters, to add caching through AOP. While in the code you clearly implement some low-level cache updating logic of your own.
At least next possible changes may help you:
Remove #Cacheable. You don't need it because you work with cache manually, so it may be the source of conflicts (especially as soon as you use sync = true). If it helps, remove #EnableCaching as well - it enables support for cache-related Spring annotations which you don't need here.
Try removing Ticker bean with the appropriate parameters for other beans. It should not be harmful as per your configuration, but usually it's helpful only for tests, no need to define it otherwise.
Double-check what is declineRecord. If it's a serialized object, ensure that serialization works properly.
Add recordStats() for cache and output stats() to log for further analysis.
I wish to test the following scenarios:
Set the hystrix.command.default.execution.isolation.thread.timeoutInMillisecond value to a low value, and see how my application behaves.
Check my fallback method is called using Unit test.
Please can someone provide me with link to samples.
A real usage can be found bellow. The key to enable Hystrix in the test class are these two annotations:
#EnableCircuitBreaker
#EnableAspectJAutoProxy
class ClipboardService {
#HystrixCommand(fallbackMethod = "getNextClipboardFallback")
public Task getNextClipboard(int numberOfTasks) {
doYourExternalSystemCallHere....
}
public Task getNextClipboardFallback(int numberOfTasks) {
return null;
}
}
#RunWith(SpringRunner.class)
#EnableCircuitBreaker
#EnableAspectJAutoProxy
#TestPropertySource("classpath:test.properties")
#ContextConfiguration(classes = {ClipboardService.class})
public class ClipboardServiceIT {
private MockRestServiceServer mockServer;
#Autowired
private ClipboardService clipboardService;
#Before
public void setUp() {
this.mockServer = MockRestServiceServer.createServer(restTemplate);
}
#Test
public void testGetNextClipboardWithBadRequest() {
mockServer.expect(ExpectedCount.once(), requestTo("https://getDocument.com?task=1")).andExpect(method(HttpMethod.GET))
.andRespond(MockRestResponseCreators.withStatus(HttpStatus.BAD_REQUEST));
Task nextClipboard = clipboardService.getNextClipboard(1);
assertNull(nextClipboard); // this should be answered by your fallBack method
}
}
Fore open the circuit in your unit test case just before you call the client. Make sure fall back is called. You can have a constant returned from fallback or add some log statements.
Reset the circuit.
#Test
public void testSendOrder_openCircuit() {
String order = null;
ServiceResponse response = null;
order = loadFile("/order.json");
// use this in case of feign hystrix
ConfigurationManager.getConfigInstance()
.setProperty("hystrix.command.default.circuitBreaker.forceOpen", "true");
// use this in case of just hystrix
System.setProperty("hystrix.command.default.circuitBreaker.forceOpen", "true");
response = client.sendOrder(order);
assertThat(response.getResultStatus()).isEqualTo("Fallback");
// DONT forget to reset
ConfigurationManager.getConfigInstance()
.setProperty("hystrix.command.default.circuitBreaker.forceOpen", "false");
// use this in case of just hystrix
System.setProperty("hystrix.command.default.circuitBreaker.forceOpen", "false");
}
I want to limit concurrent method invocation in spring application.
There is interceptor for this and here the example of using this interceptor.
But the problem is that method(which need to be limited) is not in a bean, I am creating new object every time I need to call method.
Is there is possibility to achieve limitation in this case?
You can use Load-time weaving with AspectJ and write a custom aspect which does the throttling.
Example
#Aspect
public class ThrottlingAspect {
private static final int MAX_CONCURRENT_INVOCATIONS = 20;
private final Semaphore throttle = new Semaphore (MAX_CONCURRENT_INVOCATIONS, true);
#Around("methodsToBeThrottled()")
public Object profile(ProceedingJoinPoint pjp) throws Throwable {
throttle.acquire ();
try {
return pjp.proceed ();
}
finally {
throttle.release ();
}
}
#Pointcut("execution(public * foo..*.*(..))")
public void methodsToBeThrottled(){}
}
I'm trying to unit test the custom events that I've created in Spring and am running into an interesting problem. If I create a StaticApplicationContext and manually register and wire the beans I can trigger events and see the program flow through the publisher (implements ApplicationEventPublisherAware) through to the listener (implements ApplicationListener<?>).
Yet when I try to create a JUnit test to create the context using the SpringJunit4ClassRunner and #ContextConfiguration everything works well except that the ApplicationEvents are not showing up in the listener (I have confirmed that they are getting published).
Is there some other way to create the context so that ApplicationEvents will work correctly? I haven't found much on the web about unit testing the Spring events framework.
The events will not fire because your test classes are not registered and resolved from the spring application context, which is the event publisher.
I've implemented a workaround for this where the event is handled in another class that is registered with Spring as a bean and resolved as part of the test. It isn't pretty, but after wasting the best part of a day trying to find a better solution I am happy with this for now.
My use case was firing an event when a message is received within a RabbitMQ consumer. It is made up of the following:
The wrapper class
Note the Init() function that is called from the test to pass in the callback function after resolving from the container within the test
public class TestEventListenerWrapper {
CountDownLatch countDownLatch;
TestEventWrapperCallbackFunction testEventWrapperCallbackFunction;
public TestEventListenerWrapper(){
}
public void Init(CountDownLatch countDownLatch, TestEventWrapperCallbackFunction testEventWrapperCallbackFunction){
this.countDownLatch = countDownLatch;
this.testEventWrapperCallbackFunction = testEventWrapperCallbackFunction;
}
#EventListener
public void onApplicationEvent(MyEventType1 event) {
testEventWrapperCallbackFunction.CallbackOnEventFired(event);
countDownLatch.countDown();
}
#EventListener
public void onApplicationEvent(MyEventType2 event) {
testEventWrapperCallbackFunction.CallbackOnEventFired(event);
countDownLatch.countDown();
}
#EventListener
public void onApplicationEvent(OnQueueMessageReceived event) {
testEventWrapperCallbackFunction.CallbackOnEventFired(event);
countDownLatch.countDown();
}
}
The callback interface
public interface TestEventWrapperCallbackFunction {
void CallbackOnEventFired(ApplicationEvent event);
}
A test configuration class to define the bean which is referenced in the unit test. Before this is useful, it will need to be resolved from the applicationContext and initialsed (see next step)
#Configuration
public class TestContextConfiguration {
#Lazy
#Bean(name="testEventListenerWrapper")
public TestEventListenerWrapper testEventListenerWrapper(){
return new TestEventListenerWrapper();
}
}
Finally, the unit test itself that resolves the bean from the applicationContext and calls the Init() function to pass assertion criteria (this assumes you have registered the bean as a singleton - the default for the Spring applicationContext). The callback function is defined here and also passed to Init().
#ContextConfiguration(classes= {TestContextConfiguration.class,
//..., - other config classes
//..., - other config classes
})
public class QueueListenerUnitTests
extends AbstractTestNGSpringContextTests {
private MessageProcessorManager mockedMessageProcessorManager;
private ChannelAwareMessageListener queueListener;
private OnQueueMessageReceived currentEvent;
#BeforeTest
public void Startup() throws Exception {
this.springTestContextPrepareTestInstance();
queueListener = new QueueListenerImpl(mockedMessageProcessorManager);
((QueueListenerImpl) queueListener).setApplicationEventPublisher(this.applicationContext);
currentEvent = null;
}
#Test
public void HandleMessageReceived_QueueMessageReceivedEventFires_WhenValidMessageIsReceived() throws Exception {
//Arrange
//Other arrange logic
Channel mockedRabbitmqChannel = CreateMockRabbitmqChannel();
CountDownLatch countDownLatch = new CountDownLatch(1);
TestEventWrapperCallbackFunction testEventWrapperCallbackFunction = (ev) -> CallbackOnEventFired(ev);
TestEventListenerWrapper testEventListenerWrapper = (TestEventListenerWrapper)applicationContext.getBean("testEventWrapperOnQueueMessageReceived");
testEventListenerWrapper.Init(countDownLatch, testEventWrapperCallbackFunction);
//Act
queueListener.onMessage(message, mockedRabbitmqChannel);
long awaitTimeoutInMs = 1000;
countDownLatch.await(awaitTimeoutInMs, TimeUnit.MILLISECONDS);
//Assert - assertion goes here
}
//The callback function that passes the event back here so it can be made available to the tests for assertion
private void CallbackOnEventFired(ApplicationEvent event){
currentEvent = (OnQueueMessageReceived)event;
}
}
EDIT 1: The sample code has been updated with CountDownLatch
EDIT 2: Assertions didn't fail tests so the above was updated with a different approach**
I just run my app as SpringBootTest, application events working fine:
#TestComponent
public class EventTestListener {
#EventListener
public void handle(MyCustomEvent event) {
// nothing to do, just spy the method...
}
}
#RunWith(SpringRunner.class)
#SpringBootTest
public class MyEventTest {
#SpyBean
private EventTestListener testEventListener;
#Test
public void testMyEventFires() {
// do something that fires the event..
verify(testEventListener).handle(any(MyCustomEvent.class));
}
}
use the #Captor / ArgumentCaptor to verify the content of your event.
You can create a context manually.
For example: I had needed to check if my ApplicationListener<ContextClosedEvent> closed Cassandra connections:
#Test
public void testSpringShutdownHookForCassandra(){
ConfigurableApplicationContext ctx = new AnnotationConfigApplicationContext(CassandraConfig.class);
CassandraConnectionManager connectionManager = ctx.getBean(CassandraConnectionManager.class);
Session session = connectionManager.openSession(testKeySpaceName);
Assert.assertFalse( session.isClosed() );
ctx.close();
Assert.assertTrue( session.isClosed() );
}