spring integration publish subscribe between beans - spring

Thanks for reading ahead of time. In my main method I have a PublishSubscribeChannel
#Bean(name = "feeSchedule")
public SubscribableChannel getMessageChannel() {
return new PublishSubscribeChannel();
}
In a service that does a long running process it creates a fee schedule that I inject the channel into
#Service
public class FeeScheduleCompareServiceImpl implements FeeScheduleCompareService {
#Autowired
MessageChannel outChannel;
public List<FeeScheduleUpdate> compareFeeSchedules(String oldStudyId) {
List<FeeScheduleUpdate> sortedResultList = longMethod(oldStudyId);
outChannel.send(MessageBuilder.withPayload(sortedResultList).build());
return sortedResultList;
}
}
Now this is the part I'm struggling with. I want to use completable future and get the payload of the event in the future A in another spring bean. I need future A to return the payload from the message. I think want to create a ServiceActivator to be the message end point but like I said, I need it to return the payload for future A.
#org.springframework.stereotype.Service
public class SFCCCompareServiceImpl implements SFCCCompareService {
#Autowired
private SubscribableChannel outChannel;
#Override
public List<SFCCCompareDTO> compareSFCC(String state, int service){
ArrayList<SFCCCompareDTO> returnList = new ArrayList<SFCCCompareDTO>();
CompletableFuture<List<FeeScheduleUpdate>> fa = CompletableFuture.supplyAsync( () ->
{ //block A WHAT GOES HERE?!?!
outChannel.subscribe()
}
);
CompletableFuture<List<StateFeeCodeClassification>> fb = CompletableFuture.supplyAsync( () ->
{
return this.stateFeeCodeClassificationRepository.findAll();
}
);
CompletableFuture<List<SFCCCompareDTO>> fc = fa.thenCombine(fb,(a,b) ->{
//block C
//get in this block when both A & B are complete
Object theList = b.stream().forEach(new Consumer<StateFeeCodeClassification>() {
#Override
public void accept(StateFeeCodeClassification stateFeeCodeClassification) {
a.stream().forEach(new Consumer<FeeScheduleUpdate>() {
#Override
public void accept(FeeScheduleUpdate feeScheduleUpdate) {
returnList new SFCCCompareDTO();
}
});
}
}).collect(Collectors.toList());
return theList;
});
fc.join();
return returnList;
}
}
Was thinking there would be a service activator like:
#MessageEndpoint
public class UpdatesHandler implements MessageHandler{
#ServiceActivator(requiresReply = "true")
public List<FeeScheduleUpdate> getUpdates(Message m){
return (List<FeeScheduleUpdate>) m.getPayload();
}
}

Your question isn't clear, but I'll try to help you with some info.
Spring Integration doesn't provide CompletableFuture support, but it does provide an async handling and replies.
See Asynchronous Gateway for more information. And also see Asynchronous Service Activator.
outChannel.subscribe() should come with the MessageHandler callback, by the way.

Related

Validate Consumer Group is created on topic or not

I am working on project where I need to validate consumer group is created on topic or not. Is there any way in boldSpring Kafkabold to validate it
Currently, I haven't seen describeConsumerGroups supported in Spring-Kafka KafkaAdmin. So, you may need to create a Kafka AdminClient and call the method by yourself.
E.g: Here, I took advantage of the auto-configuration property class KafkaProperties and autowired it to the service.
#Service
public class KafkaBrokerService implements BrokerService {
private Map<String, Object> configs;
public KafkaBrokerService(KafkaProperties kafkaProperties) { // Autowired
this.configs = kafkaProperties.buildAdminProperties();
}
private AdminClient createAdmin() {
Map<String, Object> configs2 = new HashMap<>(this.configs);
return AdminClient.create(configs2);
}
public SomeDto consumerGroupDescription(String groupId) {
try (AdminClient adminClient = createAdmin()) {
// ConsumerGroup's members
ConsumerGroupDescription consumerGroupDescription = adminClient.describeConsumerGroups(Collections.singletonList(groupId))
.describedGroups().get(groupId).get();
// ConsumerGroup's partitions and the committed offset in each partition
Map<TopicPartition, OffsetAndMetadata> offsets = adminClient.listConsumerGroupOffsets(groupId).partitionsToOffsetAndMetadata().get();
// When you get the information, you can validate it here.
...
} catch (ExecutionException | InterruptedException e) {
//
}
}
}

How to do manual transaction management with JOOQ and Spring-boot 2.0?

Using Spring Boot 2.0.4 and JOOQ 3.11.3.
I have a server endpoint that needs fine-grained control over transaction management; it needs to issue multiple SQL statements before and after an external call and must not keep the DB transaction open while talking to the external site.
In the below code testTransactionV4 is the attempt I like best.
I've looked in the JOOQ manual but the transaction-management section is pretty light-on and seems to imply this is the way to do it.
It feels like I'm working harder than I should be here, which is usually a sign that I'm doing it wrong. Is there a better, "correct" way to do manual transaction management with Spring/JOOQ?
Also, any improvements to the implementation of the TransactionBean would be greatly appreciated (and upvoted).
But the point of this question is really just: "Is this the right way"?
TestEndpoint:
#Role.SystemApi
#SystemApiEndpoint
public class TestEndpoint {
private static Log log = to(TestEndpoint.class);
#Autowired private DSLContext db;
#Autowired private TransactionBean txBean;
#Autowired private Tx tx;
private void doNonTransactionalThing() {
log.info("long running thing that should not be inside a transaction");
}
/** Works; don't like the commitWithResult name but it'll do if there's
no better way. Implementation is ugly too.
*/
#JsonPostMethod("testTransactionV4")
public void testMultiTransactionWithTxBean() {
log.info("start testMultiTransactionWithTxBean");
AccountRecord account = txBean.commitWithResult( db ->
db.fetchOne(ACCOUNT, ACCOUNT.ID.eq(1)) );
doNonTransactionalThing();
account.setName("test_tx+"+new Date());
txBean.commit(db -> account.store() );
}
/** Works; but it's ugly, especially having to work around lambda final
requirements on references. */
#JsonPostMethod("testTransactionV3")
public void testMultiTransactionWithJooqApi() {
log.info("start testMultiTransactionWithJooqApi");
AtomicReference<AccountRecord> account = new AtomicReference<>();
db.transaction( config->
account.set(DSL.using(config).fetchOne(ACCOUNT, ACCOUNT.ID.eq(1))) );
doNonTransactionalThing();
account.get().setName("test_tx+"+new Date());
db.transaction(config->{
account.get().store();
});
}
/** Does not work, there's only one commit that spans over the long operation */
#JsonPostMethod("testTransactionV1")
#Transactional
public void testIncorrectSingleTransactionWithMethodAnnotation() {
log.info("start testIncorrectSingleTransactionWithMethodAnnotation");
AccountRecord account = db.fetchOne(ACCOUNT, ACCOUNT.ID.eq(1));
doNonTransactionalThing();
account.setName("test_tx+"+new Date());
account.store();
}
/** Works, but I don't like defining my tx boundaries this way, readability
is poor (relies on correct bean naming and even then is non-obvious) and is
fragile in the face of refactoring. When explicit TX boundaries are needed
I want them getting in my face straight away.
*/
#JsonPostMethod("testTransactionV2")
public void testMultiTransactionWithNestedComponent() {
log.info("start testTransactionWithComponentDelegation");
AccountRecord account = tx.readAccount();
doNonTransactionalThing();
account.setName("test_tx+"+new Date());
tx.writeAccount(account);
}
#Component
static class Tx {
#Autowired private DSLContext db;
#Transactional
public AccountRecord readAccount() {
return db.fetchOne(ACCOUNT, ACCOUNT.ID.eq(1));
}
#Transactional
public void writeAccount(AccountRecord account) {
account.store();
}
}
}
TransactionBean:
#Component
public class TransactionBean {
#Autowired private DSLContext db;
/**
Don't like the name, but can't figure out how to make it be just "commit".
*/
public <T> T commitWithResult(Function<DSLContext, T> worker) {
// Yuck, at the very least need an array or something as the holder.
AtomicReference<T> result = new AtomicReference<>();
db.transaction( config -> result.set(
worker.apply(DSL.using(config))
));
return result.get();
}
public void commit(Consumer<DSLContext> worker) {
db.transaction( config ->
worker.accept(DSL.using(config))
);
}
public void commit(Runnable worker) {
db.transaction( config ->
worker.run()
);
}
}
Use the TransactionTemplate to wrap the transactional part. Spring Boot provides one out-of-the-box so it is ready for use. You can use the execute method to wrap a call in a transaction.
#Autowired
private TransactionTemplate transaction;
#JsonPostMethod("testTransactionV1")
public void testIncorrectSingleTransactionWithTransactionTemplate() {
log.info("start testIncorrectSingleTransactionWithMethodAnnotation");
AccountRecord account = transaction.execute( status -> db.fetchOne(ACCOUNT, ACCOUNT.ID.eq(1)));
doNonTransactionalThing();
transaction.execute(status -> {
account.setName("test_tx+"+new Date());
account.store();
return null;
}
}
Something like that should do the trick. Not sure if the lambdas would work (keep forgetting the syntax of the TransactionCallback

Spring-Boot MQTT Configuration

I have a requirement to send payload to a lot of devices whose names are picked from Database. Then, i have to send to different topics, which will be like settings/{put devicename here}.
Below is the configuration i was using which i got from spring-boot reference documents.
MQTTConfiguration.java
#Configuration
#IntegrationComponentScan
public class MQTTConfiguration {
#Autowired
private Settings settings;
#Autowired
private DevMqttMessageListener messageListener;
#Bean
MqttPahoClientFactory mqttClientFactory() {
DefaultMqttPahoClientFactory clientFactory = new DefaultMqttPahoClientFactory();
clientFactory.setServerURIs(settings.getMqttBrokerUrl());
clientFactory.setUserName(settings.getMqttBrokerUser());
clientFactory.setPassword(settings.getMqttBrokerPassword());
return clientFactory;
}
#Bean
MessageChannel mqttOutboundChannel() {
return new DirectChannel();
}
#Bean
#ServiceActivator(inputChannel = "mqttOutboundChannel")
public MessageHandler mqttOutbound() {
MqttPahoMessageHandler messageHandler = new MqttPahoMessageHandler("dev-client-outbound",
mqttClientFactory());
messageHandler.setAsync(true);
messageHandler.setDefaultTopic(settings.getMqttPublishTopic());
return messageHandler;
}
#MessagingGateway(defaultRequestChannel = "mqttOutboundChannel")
public interface DeviceGateway {
void sendToMqtt(String payload);
}
}
Here, i am sending to only 1 topic. So i added the bean like below to send to multiple number of topics;
#Bean
public MqttClient mqttClient() throws MqttException {
MqttClient mqttClient = new MqttClient(settings.getMqttBrokerUrl(), "dev-client-outbound");
MqttConnectOptions connOptions = new MqttConnectOptions();
connOptions.setUserName(settings.getMqttBrokerUser());
connOptions.setPassword(settings.getMqttBrokerPassword().toCharArray());
mqttClient.connect(connOptions);
return mqttClient;
}
and i send using,
try {
mqttClient.publish(settings.getMqttPublishTopic()+device.getName(), mqttMessage);
} catch (MqttException e) {
LOGGER.error("Error While Sending Mqtt Messages", e);
}
Which works.
But my question is, Can i achieve the same, using output channel for better performance? If yes, any help is greatly appreciated. Thank You.
MqttClient is synchronous.
The MqttPahoMessageHandler uses an MqttAsyncClient and can be configured (set async to true) to not wait for the confirmation, but publish the confirmation later as an application event.
If you are using your own code and sending multiple messages in a loop, it will probably be faster to use an async client, and wait for the IMqttDeliveryToken completions later.

No delay happening when sending messages between message channels

I am new to Spring Integration DSL. Currently, i am trying to add a delay
between message channels- "ordersChannel" and "bookItemsChannel". But , the flow continues as though there is no delay.
Any help appreciated.
Here is the code:
#Bean
public IntegrationFlow ordersFlow() {
return IntegrationFlows.from("ordersChannel")
.split(new AbstractMessageSplitter() {
#Override
protected Object splitMessage(Message<?> message) {
return ((Order)message.getPayload()).getOrderItems();
}
})
.delay("normalMessage", new Consumer<DelayerEndpointSpec>() {
public void accept(DelayerEndpointSpec spec) {
spec.id("delayChannel");
spec.defaultDelay(50000000);
System.out.println("Going to delay");
}
})
.channel("bookItemsChannel")
.get();
}
Seems for me that mixed the init phase when you see that System.out.println("Going to delay"); and the real runtime, when the delay happens for each incoming message.
We have some delay test-case in the DSL project, but I've just wrote this one to prove that the defaultDelay works well:
#Bean
public IntegrationFlow ordersFlow() {
return f -> f
.split()
.delay("normalMessage", (DelayerEndpointSpec e) -> e.defaultDelay(5000))
.channel(c -> c.queue("bookItemsChannel"));
}
...
#Autowired
#Qualifier("ordersFlow.input")
private MessageChannel ordersFlowInput;
#Autowired
#Qualifier("bookItemsChannel")
private PollableChannel bookItemsChannel;
#Test
public void ordersDelayTests() {
this.ordersFlowInput.send(new GenericMessage<>(new String[] {"foo", "bar", "baz"}));
StopWatch stopWatch = new StopWatch();
stopWatch.start();
Message<?> receive = this.bookItemsChannel.receive(10000);
assertNotNull(receive);
receive = this.bookItemsChannel.receive(10000);
assertNotNull(receive);
receive = this.bookItemsChannel.receive(10000);
assertNotNull(receive);
stopWatch.stop();
assertThat(stopWatch.getTotalTimeMillis(), greaterThanOrEqualTo(5000L));
}
As you see it is very close to your config, but it doesn't prove that we have something wrong around .delay().
So, it would be better to provide something similar to confirm an unexpected problem.

DeferredResult in spring mvc

I have one class that extends DeferredResults and extends Runnable as shown below
public class EventDeferredObject<T> extends DeferredResult<Boolean> implements Runnable {
private Long customerId;
private String email;
#Override
public void run() {
RestTemplate restTemplate=new RestTemplate();
EmailMessageDTO emailMessageDTO=new EmailMessageDTO("dineshshe#gmail.com", "Hi There");
Boolean result=restTemplate.postForObject("http://localhost:9080/asycn/sendEmail", emailMessageDTO, Boolean.class);
this.setResult(result);
}
//Constructor and getter and setters
}
Now I have controller that return the object of the above class,whenever new request comes to controller we check if that request is present in HashMap(That stores unprocessed request at that instance).If not present then we are creating object of EventDeferredObject class can store that in HashMap and call start() method on it.If this type request is already present then we will return that from HashMap.On completion on request we will delete that request from HashMap.
#RequestMapping(value="/sendVerificationDetails")
public class SendVerificationDetailsController {
private ConcurrentMap<String , EventDeferredObject<Boolean>> requestMap=new ConcurrentHashMap<String , EventDeferredObject<Boolean>>();
#RequestMapping(value="/sendEmail",method=RequestMethod.POST)
public EventDeferredObject<Boolean> sendEmail(#RequestBody EmailDTO emailDTO)
{
EventDeferredObject<Boolean> eventDeferredObject = null;
System.out.println("Size:"+requestMap.size());
if(!requestMap.containsKey(emailDTO.getEmail()))
{
eventDeferredObject=new EventDeferredObject<Boolean>(emailDTO.getCustomerId(), emailDTO.getEmail());
requestMap.put(emailDTO.getEmail(), eventDeferredObject);
Thread t1=new Thread(eventDeferredObject);
t1.start();
}
else
{
eventDeferredObject=requestMap.get(emailDTO.getEmail());
}
eventDeferredObject.onCompletion(new Runnable() {
#Override
public void run() {
if(requestMap.containsKey(emailDTO.getEmail()))
{
requestMap.remove(emailDTO.getEmail());
}
}
});
return eventDeferredObject;
}
}
Now this code works fine if there no identical request comes to that stored in HashMap. If we give number of different request at same time code works fine.
Well, I do not know if I understood correctly, but I think you might have race conditions in the code, for example here:
if(!requestMap.containsKey(emailDTO.getEmail()))
{
eventDeferredObject=new EventDeferredObject<Boolean>(emailDTO.getCustomerId(), emailDTO.getEmail());
requestMap.put(emailDTO.getEmail(), eventDeferredObject);
Thread t1=new Thread(eventDeferredObject);
t1.start();
}
else
{
eventDeferredObject=requestMap.get(emailDTO.getEmail());
}
think of a scenario in which you have two requests with the same key emailDTO.getEmail().
Request 1 checks if there is a key in the map, does not find it and puts it inside.
Request 2 comes some time later, checks if there is a key in the map, finds it, and
goes to fetch it; however just before that, the thread started by request 1 finishes and another thread, started by onComplete event, removes the key from the map. At this point,
requestMap.get(emailDTO.getEmail())
will return null, and as a result you will have a NullPointerException.
Now, this does look like a rare scenario, so I do not know if this is the problem you see.
I would try to modify the code as follows (I did not run it myself, so I might have errors):
public class EventDeferredObject<T> extends DeferredResult<Boolean> implements Runnable {
private Long customerId;
private String email;
private ConcurrentMap ourConcurrentMap;
#Override
public void run() {
...
this.setResult(result);
ourConcurrentMap.remove(this.email);
}
//Constructor and getter and setters
}
so the DeferredResult implementation has the responsibility to remove itself from the concurrent map. Moreover I do not use the onComplete to set a callback thread, as it seems to me an unnecessary complication. To avoid the race conditions I talked about before, one needs to combine somehow the verification of the presence of an entry with its fetching into one atomic operation; this is done by the putIfAbsent method of ConcurrentMap. Therefore I change the controller into
#RequestMapping(value="/sendVerificationDetails")
public class SendVerificationDetailsController {
private ConcurrentMap<String , EventDeferredObject<Boolean>> requestMap=new ConcurrentHashMap<String , EventDeferredObject<Boolean>>();
#RequestMapping(value="/sendEmail",method=RequestMethod.POST)
public EventDeferredObject<Boolean> sendEmail(#RequestBody EmailDTO emailDTO)
{
EventDeferredObject<Boolean> eventDeferredObject = new EventDeferredObject<Boolean>(emailDTO.getCustomerId(), emailDTO.getEmail(), requestMap);
EventDeferredObject<Boolean> oldEventDeferredObject = requestMap.putIfAbsent(emailDTO.getEmail(), eventDeferredObject );
if(oldEventDeferredObject == null)
{
//if no value was present before
Thread t1=new Thread(eventDeferredObject);
t1.start();
return eventDeferredObject;
}
else
{
return oldEventDeferredObject;
}
}
}
if this does not solve the problem you have, I hope that at least it might give some idea.

Resources