Workflow -> (https://i.stack.imgur.com/vgtiD.png)
Is it possible to call a microservice from a Camunda task?
1.The start event will received a Json with a client data .
2.The service task should connect to a microservice (spring boot) that stores the data in the database.-> just need to pass the json with the info to the micro and then should complete the task.
3. if the previous task is completed this task should run.
is there a way to do it? I am very new at camunda.
External Task but it didnt work
Yes you can, check for documentation :
#Component
#ExternalTaskSubscription("scoreProvider") // create a subscription for this topic name
public class ProvideScoreHandler implements ExternalTaskHandler {
#Override
public void execute(ExternalTask externalTask, ExternalTaskService externalTaskService) {
// only for the sake of this demonstration, we generate random data
// in a real-world scenario, we would load the data from a database
String customerId = "C-" + UUID.randomUUID().toString().substring(32);
int creditScore = (int) (Math.random() * 11);
VariableMap variables = Variables.createVariables();
variables.put("customerId", customerId);
variables.put("creditScore", creditScore);
// complete the external task
externalTaskService.complete(externalTask, variables);
Logger.getLogger("scoreProvider")
.log(Level.INFO, "Credit score {0} for customer {1} provided!", new Object[]{creditScore, customerId});
}
}
Spring boot with Camunda example
Related
I have a pub/sub scenario, where I create and save to DB something in one service, publish it to a SNS topic, subscribe with SQS listener, and handle the message and save it to DB in another service. So far so good.
In one of the scenarios I create a user and subscribe it to a site. Then I send the new user to its topic, the user-site relation to another topic, and the subscribed service updates its own DB tables.
private void publishNewUserNotifications(UserEntity userEntity, List<SiteEntity> sitesToAssociateWithUser) {
iPublisherService.publishNewUserNotification(userEntity);
if (sitesToAssociateWithUser != null || !sitesToAssociateWithUser.isEmpty()) {
List<String> sitesIds = sitesToAssociateWithUser.stream().map(SiteEntity::getSiteId).collect(Collectors.toList());
iPublisherService.publishSitesToUserAssignment(userEntity.getId(), new ArrayList<>(), sitesIds);
}
}
The problem is that sometimes I have a thread race and handle the user-site relation before I created the user in the second service, get an empty result from DB when loading the User object, and fail to handle the user-site relation.
#Override
#Transactional
public void handle(UsersSitesListNotification message) {
UsersSitesNotification assigned = message.getAssigned();
List<UserEntity> userEntities = iUserRepository.findAllByUserIdIn(CollectionUtils.union(assigned.getUserIds()));
List<SiteEntity> siteEntities = iSiteRepository.findAllByIdIn(CollectionUtils.union(assigned.getSiteIds()));
List<UserSiteAssignmentEntity> assignedEntities = fromUsersSitesNotificationToUserSiteAssignmentEntities(assigned, userEntities, siteEntities);
Iterable<UserSiteAssignmentEntity> saved = iUserSiteAssignmentRepository.saveAll(assignedEntities);
}
Because of that, I consider using SQS in a synchronic way. The problem is that in order to use SQS I need to import the "spring-cloud-aws-messaging" package, and the SQS configuration inside it uses the Async client.
Is there a way to use SQS in a synchronic way? What should I change? How should I override the Async configuration that I need in the package/get some other package?
Any idea will help, tnx.
So I have a Apache Camel route that reads Data elements from a JPA endpoint, converts them to DataConverted elements and stores them into a different database via a second JPA endpoint. Both endpoints are Oracle databases.
Now I want to set a flag on the original Data element that it got copied successfully. What is the best way to achieve that?
I tried it like that: saving the ID in the context and then reading it and accessing a dao method in the .onCompletion().onCompleteOnly().
from("jpa://Data")
.onCompletion().onCompleteOnly().process(ex -> {
var id = Long.valueOf(getContext().getGlobalOption("id"));
myDao().setFlag(id);
}).end()
.process(ex -> {
Data data = ex.getIn().getBody(Data.class);
DataConverted dataConverted = convertData(data);
ex.getMessage().setBody(data);
var globalOptions = getContext().getGlobalOptions();
globalOptions.put("id", data.getId().toString());
getContext().setGlobalOptions(globalOptions);
})
.to("jpa://DataConverted").end();
However, this seems to trigger a deadlock, the dao method is stalling on the commit of the update. The only explanation could be that the Data object gets locked by Camel and is still locked in the .onCompletion().onCompleteOnly() part of the route, therefore it can't get updated there.
Is there a better way to do it?
Have you tried using the recipient list EIP where first destination is the jpa:DataConverted endpoint and the second destination will be the endpoint to set the flag. This way both get the same message and will be executed sequentially.
https://camel.apache.org/components/3.17.x/eips/recipientList-eip.html
from("jpa://Data")
.process(ex -> {
Data data = ex.getIn().getBody(Data.class);
DataConverted dataConverted = convertData(data);
ex.getIn().setBody(data);
})
.recipientList(constant("direct:DataConverted","direct:updateFlag"))
.end();
from("direct:DataConverted")
.to("jpa://DataConverted")
.end();
from("direct:updateFlag")
.process(ex -> {
var id = ((MessageConverted) ex.getIn().getBody()).getId();
myDao().setFlag(id);
})
.end();
Keep in mind, you might want to make the route transactional by adding .transacted()
https://camel.apache.org/components/3.17.x/eips/transactional-client.html
I have requirement where need to have a Spring Boot Rest Service that a client application will call every 30 minutes and service is to return
number of latest messages based on the number specified in query param e.g. http://messages.com/getNewMessages?number=10 in this case should return 10 messages
number of messages based on the number and offset specified in query param e.g. http://messages.com/getSpecificMessages?number=5&start=123 in this case should return 5 messages starting offset 123.
I have simple standalone application and it works fine. Here is what I tested and would lke some direction of incorporating it in the service.
public static void main(String[] args) {
// create kafka consumer
Properties properties = new Properties();
properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
properties.put(ConsumerConfig.GROUP_ID_CONFIG, "my-first-consumer-group");
properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
properties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
properties.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false);
properties.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, args[0]);
Consumer<String, String> consumer = new KafkaConsumer<>(properties);
// subscribe to topic
consumer.subscribe(Collections.singleton("test"));
consumer.poll(0);
//get to specific offset and get specified number of messages
for (TopicPartition partition : consumer.assignment())
consumer.seek(partition, args[1]);
ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(5000));
System.out.println("Total Record Count ******* : " + records.count());
for (ConsumerRecord<String, String> record : records) {
System.out.println("Message: " + record.value());
System.out.println("Message offset: " + record.offset());
System.out.println("Message: " + record.timestamp());
Date date = new Date(record.timestamp());
Format format = new SimpleDateFormat("yyyy MM dd HH:mm:ss.SSS");
System.out.println("Message date: " + format.format(date));
}
consumer.commitSync();
As my consumer will be on-demand wondering in Spring Boot Service how I can achieve this. Where do I specify the properties if I put in application.properties those get's injected at startup time but how do i control MAX_POLL_RECORDS_CONFIG at runtime. Any help appreciated.
MAX_POLL_RECORDS_CONFIG only impact your kafka-client return the records to your spring service, it will never reduce the bytes that the consumer poll from kafka-server
see the above picture, no matter your start offset = 150 or 190, kafka server will return the whole data from (offset=110, offset=190), kafka server even didn't know how many records return to consumer, he only know the byte size = (220 - 110)
so i think you can control the record number by yourself,currently it is controlled by the kafka client jar, they are both occupy your jvm local memory
The answer to your question is here and the answer with code example is this answer.
Both written by the excellent Gary Russell, the main or one of the main person behind Spring Kafka.
TL;DR:
If you want to arbitrarily rewind the partitions at runtime, have your
listener implement ConsumerSeekAware and grab a reference to the
ConsumerSeekCallback.
I am trying to build a simple web application using spring boot - webflux (functional endpoints) & jdbc. The app receives payload in XML format (which is some details of 1 employee). Code given below persists data for one employee as expected.
public Mono<String> createData(final Mono<Record> inputMono) {
final String someID = UUID.randomUUID().toString();
final Mono<Integer> asyncUpdate = inputMono.flatMap(record -> {
return beginUpdate(dataSource,
sqlStatementSharedAbove).withStatementEnricher(stmt -> {
stmt.setString(1, record.getFirstName());
stmt.setString(2, record.getLastName());
stmt.setInt(3, record.getAddress());
}).build();
});
return asyncUpdate.doOnSuccess(affectedRows -> LOGGER.debug("Added
{} rows with ID {}", affectedRows, someID))
.map(affectedRows -> someID);
}
Now I need to save similar data for multiple employees (modifying the XML payload to contain multiple employee records)
In non-webflux world, I would just iterate over the list of employee objects and call this function for each one of them.
How can I achieve the same in webflux?
Essentially I am looking to handle a saveAll functionality with webflux and given that I have to work with JDBC (I do understand that JDBC doesn't support non blocking paradigm and Mongo supports a saveAll API but I have certain constraints as to what DB i can use and therefore must make this work with JDBC)
Thank you.
Following code works to save multiple employee records. Essentially it needs a Flux (of Employees) to work with -
public Mono<Void> createData(final Flux<Record> inputFlux) {
return inputFlux.flatMap(record -> {
return beginUpdate(dataSource,
sqlStatementSharedAbove).withStatementEnricher(stmt -> {
stmt.setString(1, record.getFirstName());
stmt.setString(2, record.getLastName());
stmt.setInt(3, record.getAddress());
}).build().doOnSuccess(affectedRows -> LOGGER.info("Added rows{}", affectedRows));
}).then;
}
I've been trying to come up with a demo of a website that uses MassTransit with RabbitMQ to post messages to a service running on Service Fabric as a Stateful service.
Everything was going fine, my client would post a message:
IBusControl bus = BusConfigurator.ConfigureBus();
Uri sendToUri = new Uri($"{RabbitMqConstants.RabbitMqUri}" + $"{RabbitMqConstants.PeopleServiceQueue}");
ISendEndpoint endPoint = await bus.GetSendEndpoint(sendToUri);
await endPoint.Send<ICompanyRequest>(new {CompanyId = id });
My consumer in my service fabric service was defined like:
IBusControl busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
IRabbitMqHost host = cfg.Host(new Uri(RabbitMqConstants.RabbitMqUri), h =>
{
h.Username(RabbitMqConstants.UserName);
h.Password(RabbitMqConstants.Password);
});
cfg.ReceiveEndpoint(host, RabbitMqConstants.PeopleServiceQueue, e =>
{
e.Consumer<PersonInformationConsumer>();
});
});
busControl.Start();
This does allow me to consume the message in my class and I can process it fine. The problem comes when we want to use IReliableDictonary or IReliableQueue or anything that needs to reference the context that is run from the RunAsync function in the service fabric service.
So my question is, how can I configure (is it possible) MassTransit to work within a Stateful Service Fabric Service which knowledge of the service context itself?
Many thanks in advance.
Mike
Update
Ok, I've made some progress on this, if I point the register routines to my message consumer class (eg):
ServiceRuntime.RegisterServiceAsync("ServiceType", context => new PersonInformationConsumer(context)).GetAwaiter().GetResult();
ServiceEventSource.Current.ServiceTypeRegistered(Process.GetCurrentProcess().Id, typeof(PersonInformationConsumer).Name);
Then in my consumer class for my messages I can do the following:
internal sealed class PersonInformationConsumer : StatefulService, IConsumer<ICompanyRequest>
{
private static StatefulServiceContext _currentContext;
#region Constructors
public PersonInformationConsumer(StatefulServiceContext serviceContext) : base(serviceContext)
{
_currentContext = serviceContext;
}
public PersonInformationConsumer() : base(_currentContext)
{
}
I can now successfully call the service message:
ServiceEventSource.Current.ServiceMessage(this.Context, "Message has been consumed, request Id: {0}", context.Message.CompanyId);
The problem I have now is trying to store something on the IReliableDictionary, doing this causes as "Object reference not set to an instance of an object" error :( ... any ideas would be appreciated (although may not read until new year now!)
public async Task Consume(ConsumeContext<ICompanyRequest> context)
{
ServiceEventSource.Current.ServiceMessage(this.Context, "Message has been consumed, request Id: {0}", context.Message.CompanyId);
using (ITransaction tx = StateManager.CreateTransaction())
{
try
{
var myDictionary = await StateManager.GetOrAddAsync<IReliableDictionary<string, long>>("myDictionary");
This is causing the error.... HELP! :)
You'll need to do a bit more to get MassTransit and stateful services working together, there's a few issues to concern yourself here.
Only the master within a stateful partition (n masters within n partitions) will be able to write/update to the stateful service, all replicas will throw exceptions when trying to write back any state. So you'll need to deal with this issue, on the surface it sounds easy until you take in to consideration the master can move around the cluster due to re-balancing the cluster, the default for general service fabric applications is to just turn off the processing on the replicas and only run the work on the master. This is all done by the RunAsync method (try it out, run 3 stateful services with something noddy in the RunAsync method, then terminate the master).
There is also partitioning of your data to consider, due to stateful services scale with partitions, you'll need to create a way to distributing data to separate endpoint on your service bus, maybe have a separate queue that only listens to a given partition range? Say you have a UserCreated message, you might split this on country UK goes to partition 1, US goes to partition 2 etc...
If you just want to get something basic up and running, I'd limit it to one partition and just try putting your bus creation within the the RunAsync and shutdown the bus once a cancelation is requested on the cancelation token.
protected override async Task RunAsync(CancellationToken cancellationToken)
{
var busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
IRabbitMqHost host = cfg.Host(new Uri(RabbitMqConstants.RabbitMqUri), h =>
{
h.Username(RabbitMqConstants.UserName);
h.Password(RabbitMqConstants.Password);
});
cfg.ReceiveEndpoint(host, RabbitMqConstants.PeopleServiceQueue, e =>
{
// Pass in the stateful service context
e.Consumer(c => new PersonInformationConsumer(Context));
});
});
busControl.Start();
while (true)
{
if(cancellationToken.CancellationRequested)
{
//Service Fabric wants us to stop
busControl.Stop();
cancellationToken.ThrowIfCancellationRequested();
}
await Task.Delay(TimeSpan.FromSeconds(1));
}
}