The idea is that I would like to first let a #Scheduled method retrieve some data and only when that process has finished enable/initialize my #KafkaListener. Currently the Kafka listener starts up immediately without waiting for the scheduler to be done.
I've tried to use #Conditional with a custom Condition, but this only is executed on context creation (aka startup). Also #ConditionalOnBean didn't work because actually my Scheduler bean is already created before it finishes the process.
This is how my setup looks like.
Kafka Listener:
#Service
class KafkaMessageHandler(private val someRepository) {
#KafkaListener(topics = ["myTopic"])
fun listen(messages: List<ConsumerRecord<*, *>>) {
// filter messages based on data in someRepository
// Do fancy stuff
}
}
Scheduler:
#Component
class Scheduler(private val someRepository) {
#Scheduled(fixedDelayString = "\${schedule.delay}")
fun updateData() {
// Fetch data from API
// update someRepository with this data
}
}
Is there any nice Spring way of waiting for the scheduler to finish before initializing the KafkaMessageHandler?
Related
I am trying out Spring Boot's Async feature, but I am having some trouble getting it to work as I need.
This is my application yml
spring:
task:
execution:
pool:
max-size: 100
queue-capacity: 5
keep-alive: "10s"
core-size: 10
Application class
#SpringBootApplication
#EnableAsync
public class ServiceApplication {
public static void main(String[] args) {
SpringApplication.run(ServiceApplication.class, args);
}
}
Service class:
for (int i=0;i< 40; i++) {
CompletableFuture.runAsync(()-> {
try {
System.out.println("------------------Starting thread------------------");
//do some action here
System.out.println("------------------Ending thread------------------");
} catch (Exception e) {
e.printStackTrace();
}
});
}
I am expecting to see the System.out print out 40 times. The operations in between take long enough, and I have tried adding Thread.sleep(), but I do not see the sysouts printed more than 8 times. Is there something wrong with my config, or does it not work the way I expect?
Completable future has no idea about the pool that is used by Spring.
From docs of runAsync() method:
Returns a new CompletableFuture that is asynchronously completed by a
task running in the ForkJoinPool.commonPool() after it runs the given
action. Params: runnable – the action to run before completing the
returned CompletableFuture Returns: the new CompletableFuture
So, those tasks are being run on ForkJoinPool, not on executor used by Spring.
About executor used by Spring with #EnableAsync:
By default, Spring will be searching for an associated thread pool
definition: either a unique TaskExecutor bean in the context, or an
Executor bean named "taskExecutor" otherwise. If neither of the two is
resolvable, a SimpleAsyncTaskExecutor will be used to process async
method invocations. Besides, annotated methods having a void return
type cannot transmit any exception back to the caller. By default,
such uncaught exceptions are only logged.
You could try autowire that executor and pass it as an argument to
public static CompletableFuture<Void> runAsync(Runnable runnable, Executor executor)
Returns a new CompletableFuture that is asynchronously completed by a
task running in the given executor after it runs the given action.
Params: runnable – the action to run before completing the returned
CompletableFuture executor – the executor to use for asynchronous
execution
I am using application events in my service and decided to go for multicaster since I can set up the error handler and get the stacktrace in the console (normally runtime exceptions are not caught and are silently surpressed). So I defined my multicaster config as following:
#Configuration
class ApplicationEventMulticasterConfig {
companion object {
private val log = LoggerFactory.getLogger(ApplicationEventMulticasterConfig::class.java)
}
#Bean(name = ["applicationEventMulticaster"])
fun simpleApplicationEventMulticaster(multicasterExecutor: TaskExecutor): ApplicationEventMulticaster {
val eventMulticaster = SimpleApplicationEventMulticaster()
eventMulticaster.setTaskExecutor(multicasterExecutor)
eventMulticaster.setErrorHandler { throwable ->
log.error(throwable.stackTraceToString())
}
return eventMulticaster
}
#Bean(name = ["multicasterExecutor"])
fun taskExecutor(): TaskExecutor {
val executor = ThreadPoolTaskExecutor()
executor.corePoolSize = 4
executor.maxPoolSize = 40
executor.initialize()
return executor
}
}
Listener Case1:
#TransactionalEventListener
fun onEvent(event: Events.Created) {
Listener Case2:
#TransactionalEventListener(fallbackExecution=true)
fun onEvent(event: Events.Created) {
Publishing with multicaster.multicastEvent(Events.Created()). This simply does not work as expected, in case 1, the listener is not started at all (if transaction commits or rolls back) and in case 2 is listener triggered EACH time (in case of commit or failure).
If I delete the whole ApplicationEventMulticasterConfig everything is working fine but I do not have error handler set. Do you have any idea what could be wrong? It might be something with the way how I set up those beans.
I have a simple Kafka producer in my spring cloud stream application. As my spring application starts, I have a #PostConstruct method which performs some reconciliation and tries sending events to the Kafka producer.
Issue is, my Kafka Producer is not yet ready when the reconciliation starts sending the enets into it, leading to the below:
org.springframework.messaging.MessageDeliveryException: Dispatcher has no subscribers for channel 'orderbook-service-1.orderbook'.; nested exception is org.springframework.integration.MessageDispatchingException: Dispatcher has no subscribers, failedMessage=GenericMessage ..
at org.springframework.integration.channel.AbstractSubscribableChannel.doSend(AbstractSubscribableChannel.java:77)
at org.springframework.integration.channel.AbstractMessageChannel.send(AbstractMessageChannel.java:445)
Is there is a way to get a notification during my application's startup that Kafka channel is initialized, so that I only kick off the rec job post it.
Here is my code snippets:
public interface OrderEventChannel {
String TOPIC_BINDING = "orderbook";
#Output(TOPIC_BINDING)
SubscribableChannel outboundEvent();
}
#Configuration
#EnableBinding({OrderEventChannel.class})
#ConditionalOnExpression("${aix.core.stream.outgoing.kafka.enabled:false}")
public class OutgoingKafkaConfiguration {
}
#Service
public class OutgoingOrderKafkaProducer {
#Autowired
private OrderEventChannel orderEventChannel;
public void onOrderEvent( ClientEvent clientEvent ) {
try {
Message<KafkaEvent> kafkaMsg = mapToKafkaMessage( clientEvent );
SubscribableChannel subscribableChannel = orderEventChannel.outboundEvent();
subscribableChannel.send( kafkaMsg );
} catch ( RuntimeException rte ) {
log.error( "Error while publishing Kafka event [{}]", clientEvent, rte );
}
}
..
..
}
#PostConstruct is MUCH too early in the context lifecycle to start using beans; they are still being created, configured and wired together.
You can use an ApplicationListener (or #EventListener) to listen for an ApplicationReadyEvent (be sure to compare the even's applicationContext to the main application context because you may get other events).
You can also implement SmartLifecycle and put your code in start(); put your bean in a late Phase so it is started after everything is wired up.
Output bindings are started in phase Integer.MIN_VALUE + 1000, input bindings are started in phase Integer.MAX_VALUE - 1000.
So if you want to do something before messages start flowing, use a phase in-between these (e.g. 0, which is the default).
I would like to invoke some code after my application start. Is there any way to handle event:
Started SomeApp in 14.905 seconds (JVM running for 16.268)
I'm going to try if another application is up. I've tried to use Retryable but not its executed before application started and exception is thrown so application exits.
#EventListener
fun handleContextRefresh(event: ContextRefreshedEvent) {
retryableInvokeConnection()
}
#Retryable(
value = [RetryableException::class, ConnectionException::class],
maxAttempts = 100000,
backoff = Backoff(delay = 5)
)
private fun retryableInvokeConnection() {
}
#Recover
private fun retryableInvokeConnectionExceptionHandler(ex: ConnectionException) {
}
Maybe I should use PostConstruct and while loop.
You can't call a #Retryable method within the same bean, it bypasses the proxy with the retry interceptor. Move the method to another bean and inject it.
The event is a better way than using #PostConstruct.
I am using activiti 5.18.
Behind the scenes : There are few task which are getting routed though a workflow. Some of these tasks are eligible for escalation. I have written my escalation listener as follows.
#Component
public class EscalationTimerListener implements ExecutionListener {
#Autowired
ExceptionWorkflowService exceptionWorkflowService;
#Override
public void notify(DelegateExecution execution) throws Exception {
//Process the escalated tasks here
this.exceptionWorkflowService.escalateWorkflowTask(execution);
}
}
Now when I start my tomcat server activiti framework internally calls the listener even before my entire spring context is loaded. Hence exceptionWorkflowService is null (since spring hasn't inejcted it yet!) and my code breaks.
Note : this scenario only occurs if my server isn't running at the escalation time of tasks and I start/restart my server post this time. If my server is already running during escalation time then the process runs smoothly. Because when server started it had injected the service and my listener has triggered later.
I have tried delaying activiti configuration using #DependsOn annotation so that it loads after ExceptionWorkflowService is initialized as below.
#Bean
#DependsOn({ "dataSource", "transactionManager","exceptionWorkflowService" })
public SpringProcessEngineConfiguration getConfiguration() {
final SpringProcessEngineConfiguration config = new SpringProcessEngineConfiguration();
config.setAsyncExecutorActivate(true);
config.setJobExecutorActivate(true);
config.setDataSource(this.dataSource);
config.setTransactionManager(this.transactionManager);
config.setDatabaseSchemaUpdate(this.schemaUpdate);
config.setHistory(this.history);
config.setTransactionsExternallyManaged(this.transactionsExternallyManaged);
config.setDatabaseType(this.dbType);
// Async Job Executor
final DefaultAsyncJobExecutor asyncExecutor = new DefaultAsyncJobExecutor();
asyncExecutor.setCorePoolSize(2);
asyncExecutor.setMaxPoolSize(50);
asyncExecutor.setQueueSize(100);
config.setAsyncExecutor(asyncExecutor);
return config;
}
But this gives circular reference error.
I have also tried adding a bean to SpringProcessEngineConfiguration as below.
Map<Object, Object> beanObjectMap = new HashMap<>();
beanObjectMap.put("exceptionWorkflowService", new ExceptionWorkflowServiceImpl());
config.setBeans(beanObjectMap);
and the access the same in my listener as :
Map<Object, Object> registeredBeans = Context.getProcessEngineConfiguration().getBeans();
ExceptionWorkflowService exceptionWorkflowService = (ExceptionWorkflowService) registeredBeans.get("exceptionWorkflowService");
exceptionWorkflowService.escalateWorkflowTask(execution);
This works but my repository has been autowired into my service which hasn't been initialized yet! So it again throws error in service layer :)
So is there a way that I can trigger escalation listeners only after my entire spring context is loaded?
Have you tried binding the class to ApplicationListener?
Not sure if it will work, but equally I'm not sure why your listener code is actually being executed on startup.
Try to set the implementation type of listeners using Java class or delegate expression and then in the class implement JavaDelegate instead of ExecutionListener.