spring restservice asynchronous logging functionality - spring

I had written rest services in spring that is running perfectly fine.
Now, I need to add perform some db transactions before returning response to user.
This db transaction is independent to response retrieved.
For example,
#PostMapping("login")
public TransactionResponse loginAuthentication(#Valid #RequestBody LoginRequestBody loginRequest) {
TransactionResponse transactionResponse = new TransactionResponse();
try {
transactionResponse = loginService.validateUser(loginRequest);
//independent transaction needs to be executed in a separate thread
loginSerice.addLoginLog(transactionResponse);
//return below response without waiting to compelete above log transaction
return transactionResponse;
}
catch (Exception e) {
return CommonUtils.setErrorResponse(transactionResponse, e);
}
}
I read upon async controller in spring mvc link. Although controller
executes respective functionality in a separate thread but I don't want to wait for db transaction to be completed. After getting response from service layer, it should be forwarded to user without any delay.
Any Suggestions !!
Spring version is 4.3

I posted this answer to help the fellow developers with same kind of requirement (to execute a void function in a separate thread).
Since I am not experienced in multithreading/asynchronous environment, I want to keep it simple by using spring asynchronous methods.
So, First I created the Thread Pool
#Configuration
#EnableAsync
public class ThreadConfig {
#Bean
public TaskExecutor threadPoolTaskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(4);
executor.setMaxPoolSize(4);
executor.setThreadNamePrefix("WEBAPP");
executor.initialize();
return executor;
}
}
Then I created a service that will execute my code in a separate thread.
#Async
#Service
#Transactional
public class LoggingService {
public void logintransaction() throws Exception{
System.out.println("start login loggin");
Thread.sleep(5000);
System.out.println("exit");
}
}
Lastly, I called the above service on my controller. As I can see Total Time Taken is printed first, then "start login loggin" is printed. This means my new method is executed in a new thread.
#Autowired
private LoggingService loggingService;
#PostMapping("login")
public TransactionResponse loginAuthentication(#Valid #RequestBody LoginRequestBody loginRequest) {
long startTime = System.currentTimeMillis();
TransactionResponse transactionResponse = new TransactionResponse();
try {
transactionResponse = loginService.validateUser(loginRequest);
//independent transaction needs to be executed in a separate thread
//loginSerice.addLoginLog(transactionResponse);
loggingService.logintransaction();
//return below response without waiting to compelete above log transaction
System.err.println("Total Time Taken=>"+(System.currentTimeMillis() - startTime));
return transactionResponse;
}
catch (Exception e) {
return CommonUtils.setErrorResponse(transactionResponse, e);
}
}
Thanks

Related

do some asynchronous action after current transaction commited in Quarkus

Let's say I have a rest call which creates some object "A" in the database. Method is marked with #Transactional annotation. And just after creation I need to launch another asynchronous process in another thread or through some messaging system or in some other async way. That new process depends on the object "A" and needs to see it.
How can I make sure that transaction is commited before new process starts execution?
For example in Spring there is
TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronization(){
void afterCommit(){
//do what you want to do after commit
}
})
Does Quarkus has something similar?
You can inject TransactionManager
#Inject
TransactionManager transactionManager;
and do
Transaction transaction = transactionManager.getTransaction();
transaction.registerSynchronization(new Synchronization() {
#Override
public void beforeCompletion() {
//nothing here
}
#Override
public void afterCompletion(int status) {
//do some code after completion
}
});

JdbcPollingChannelAdapter and IntegrationFlow No rolling back the update when an exception occurs in the Integraion flow messages

My use case is that I have got a spring boot application with a JdbcPollingChannelAdapter to fetch data from a postgresql database, updating the fetched rows and moving foreward with message flow (using IntegrationFlowBuilder) to process some transform to the ResultSet and publish the results to RabbitMQ.
JdbcPollingChannelAdapter is configured to fetch data each 60 seonds with a select for update query followed by an update query to flag the status form NEW to PUBLISH status:
The sql query :select * from table where status= 'NEW' order by tms_creation limit 100 for update;
The update query : update table set cod_etat = 'PUBLISH', tms_modification = now() where id in (:id)
Also, there is no Max Row per Poll to fetch data, which means that the jdbc poller will execute the sql request as many time as data (with status NEW) is present.
First issue: I stop my RabbitMQ and let my microservice running, the JdbcPollingChannelAdapter fetch the first ResultSet pass them through the Message flow and process the update. The message flow process the resultSet to send them through a channel to rabbitMQ(using spring cloud stream). The send fail and no Rollback has occured which means that the resultSet has been flagged as published.
I Have been loking around in documentation to figure out what I have missed. So any help would be appreciate.
Second issue: I run 3 instances of my application on PCF, and handle the concurrent access to the rows in the datable. My transaction and the select for update query in The JdbcPollingChannelAdapter suppose to get Row-level Lock Modes for the current transaction as per sql query (select for update). But what is happening is that more than one instance could get the same rows which is supposed to be managed by the current lock. Thus, it leads to multiple instances handling the same data and publishing them multiple times.
My code is as
#EnableConfigurationProperties(ProprietesSourceJdbc.class)
#Component
public class KafkaGuy {
private static final Logger LOG = LoggerFactory.getLogger(KafkaGuy.class);
private ProprietesSourceJdbc proprietesSourceJdbc;
private DataSource sourceDeDonnees;
private DemandeSource demandeSource;
private ObjectMapper objectMapper;
private JdbcTemplate jdbcTemplate;
public KafkaGuy(ProprietesSourceJdbc proprietesSourceJdbc, DemandeSource demandeSource, DataSource dataSource, JdbcTemplate jdbcTemplate, ObjectMapper objectMapper) {
this.proprietesSourceJdbc = proprietesSourceJdbc;
this.demandeSource = demandeSource;
this.sourceDeDonnees = dataSource;
this.objectMapper = objectMapper;
this.jdbcTemplate = jdbcTemplate;
}
#Bean
public MessageSource<Object> jdbcSourceMessage() {
JdbcPollingChannelAdapter jdbcSource = new JdbcPollingChannelAdapter(this.sourceDeDonnees, this.proprietesSourceJdbc.getQuery());
jdbcSource.setUpdateSql(this.proprietesSourceJdbc.getUpdate());
return jdbcSource;
}
#Bean
public IntegrationFlow fluxDeDonnees() {
IntegrationFlowBuilder flowBuilder = IntegrationFlows.from(jdbcSourceMessage());
flowBuilder
.split()
.log(LoggingHandler.Level.INFO, message ->
message.getHeaders().get("sequenceNumber")
+ " événements publiés sur le bus de message sur "
+ message.getHeaders().get("sequenceSize")
+ " événements lus (lot)")
.transform(Transformers.toJson())
.enrichHeaders(h -> h.headerExpression("type", "payload.typ_evenement"))
.publishSubscribeChannel(publishSubscribeSpec -> publishSubscribeSpec
.subscribe(flow -> flow
.transform(Transformers.toJson())
.transform(kafkaGuyTransformer())
.channel(this.demandeSource.demandePreinscriptionOuput()))
);
return flowBuilder.get();
}
#Bean
public KafkaGuyTransformer kafkaGuyTransformer() {
return new KafkaGuyTransformer();
}
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata defaultPoller() {
PollerMetadata pollerMetadata = new PollerMetadata();
PeriodicTrigger trigger = new PeriodicTrigger(this.proprietesSourceJdbc.getTriggerDelay(), TimeUnit.SECONDS);
pollerMetadata.setTrigger(trigger);
pollerMetadata.setMaxMessagesPerPoll(proprietesSourceJdbc.getMaxRowsPerPoll());
return pollerMetadata;
}
public class KafkaGuyTransformer implements GenericTransformer<Message, Message> {
#Override
public Message transform(Message message) {
Message<String> msg = null;
try {
DemandeRecueDTO dto = objectMapper.readValue(message.getPayload().toString(), DemandeRecueDTO.class);
msg = MessageBuilder.withPayload(dto.getTxtDonnee())
.copyHeaders(message.getHeaders())
.build();
} catch (Exception e) {
LOG.error(e.getMessage(), e);
}
return msg;
}
}
}
I am new In spring integration and sorry if is not well explained. Any help is appreciate.
Everything looks good and should be as you have described. Only the problem I see that there is no transaction configured for the IntegrationFlows.from(jdbcSourceMessage()).
Consider to PollerMetadata.setAdviceChain() with a TransactionInterceptor.
Another way is to use a PollerSpec with its transactional() option.
This way you won't use local data base transactions which are committed exactly after return from the ResultSet processing. With transaction on the application level there is not going to be a commit until you exit a thread.

Spring Boot with CXF Client Race Condition/Connection Timeout

I have a CXF client configured in my Spring Boot app like so:
#Bean
public ConsumerSupportService consumerSupportService() {
JaxWsProxyFactoryBean jaxWsProxyFactoryBean = new JaxWsProxyFactoryBean();
jaxWsProxyFactoryBean.setServiceClass(ConsumerSupportService.class);
jaxWsProxyFactoryBean.setAddress("https://www.someservice.com/service?wsdl");
jaxWsProxyFactoryBean.setBindingId(SOAPBinding.SOAP12HTTP_BINDING);
WSAddressingFeature wsAddressingFeature = new WSAddressingFeature();
wsAddressingFeature.setAddressingRequired(true);
jaxWsProxyFactoryBean.getFeatures().add(wsAddressingFeature);
ConsumerSupportService service = (ConsumerSupportService) jaxWsProxyFactoryBean.create();
Client client = ClientProxy.getClient(service);
AddressingProperties addressingProperties = new AddressingProperties();
AttributedURIType to = new AttributedURIType();
to.setValue(applicationProperties.getWex().getServices().getConsumersupport().getTo());
addressingProperties.setTo(to);
AttributedURIType action = new AttributedURIType();
action.setValue("http://serviceaction/SearchConsumer");
addressingProperties.setAction(action);
client.getRequestContext().put("javax.xml.ws.addressing.context", addressingProperties);
setClientTimeout(client);
return service;
}
private void setClientTimeout(Client client) {
HTTPConduit conduit = (HTTPConduit) client.getConduit();
HTTPClientPolicy policy = new HTTPClientPolicy();
policy.setConnectionTimeout(applicationProperties.getWex().getServices().getClient().getConnectionTimeout());
policy.setReceiveTimeout(applicationProperties.getWex().getServices().getClient().getReceiveTimeout());
conduit.setClient(policy);
}
This same service bean is accessed by two different threads in the same application sequence. If I execute this particular sequence 10 times in a row, I will get a connection timeout from the service call at least 3 times. What I'm seeing is:
Caused by: java.io.IOException: Timed out waiting for response to operation {http://theservice.com}SearchConsumer.
at org.apache.cxf.endpoint.ClientImpl.waitResponse(ClientImpl.java:685) ~[cxf-core-3.2.0.jar:3.2.0]
at org.apache.cxf.endpoint.ClientImpl.processResult(ClientImpl.java:608) ~[cxf-core-3.2.0.jar:3.2.0]
If I change the sequence such that one of the threads does not call this service, then the error goes away. So, it seems like there's some sort of a race condition happening here. If I look at the logs in our proxy manager for this service, I can see that both of the service calls do return a response very quickly, but the second service call seems to get stuck somewhere in the code and never actually lets go of the connection until the timeout value is reached. I've been trying to track down the cause of this for quite a while, but have been unsuccessful.
I've read some mixed opinions as to whether or not CXF client proxies are thread-safe, but I was under the impression that they were. If this actually not the case, and I should be creating a new client proxy for each invocation, or use a pool of proxies?
Turns out that it is an issue with the proxy not being thread-safe. What I wound up doing was leveraging a solution kind of like one posted at the bottom of this post: Is this JAX-WS client call thread safe? - I created a pool for the proxies and I use that to access proxies from multiple threads in a thread-safe manner. This seems to work out pretty well.
public class JaxWSServiceProxyPool<T> extends GenericObjectPool<T> {
JaxWSServiceProxyPool(Supplier<T> factory, GenericObjectPoolConfig poolConfig) {
super(new BasePooledObjectFactory<T>() {
#Override
public T create() throws Exception {
return factory.get();
}
#Override
public PooledObject<T> wrap(T t) {
return new DefaultPooledObject<>(t);
}
}, poolConfig != null ? poolConfig : new GenericObjectPoolConfig());
}
}
I then created a simple "registry" class to keep references to various pools.
#Component
public class JaxWSServiceProxyPoolRegistry {
private static final Map<Class, JaxWSServiceProxyPool> registry = new HashMap<>();
public synchronized <T> void register(Class<T> serviceTypeClass, Supplier<T> factory, GenericObjectPoolConfig poolConfig) {
Assert.notNull(serviceTypeClass);
Assert.notNull(factory);
if (!registry.containsKey(serviceTypeClass)) {
registry.put(serviceTypeClass, new JaxWSServiceProxyPool<>(factory, poolConfig));
}
}
public <T> void register(Class<T> serviceTypeClass, Supplier<T> factory) {
register(serviceTypeClass, factory, null);
}
#SuppressWarnings("unchecked")
public <T> JaxWSServiceProxyPool<T> getServiceProxyPool(Class<T> serviceTypeClass) {
Assert.notNull(serviceTypeClass);
return registry.get(serviceTypeClass);
}
}
To use it, I did:
JaxWSServiceProxyPoolRegistry jaxWSServiceProxyPoolRegistry = new JaxWSServiceProxyPoolRegistry();
jaxWSServiceProxyPoolRegistry.register(ConsumerSupportService.class,
this::buildConsumerSupportServiceClient,
getConsumerSupportServicePoolConfig());
Where buildConsumerSupportServiceClient uses a JaxWsProxyFactoryBean to build up the client.
To retrieve an instance from the pool I inject my registry class and then do:
JaxWSServiceProxyPool<ConsumerSupportService> consumerSupportServiceJaxWSServiceProxyPool = jaxWSServiceProxyPoolRegistry.getServiceProxyPool(ConsumerSupportService.class);
And then borrow/return the object from/to the pool as necessary.
This seems to work well so far. I've executed some fairly heavy load tests against it and it's held up.

Return job id "immediately" for spring batch job before it completes

I am working on a project where we are using Spring Boot, Spring Batch and Camel.
The batch process is started by a call to a rest endpoint. The rest controller starts a camel route that starts the spring batch job flow (via spring batch camel component).
I have no control over the external application that calls my application. My application is part of a bigger nightly work flow.
The batch job can take a long time to complete and therefore the external application periodically polls my batch job via another rest endpoint asking if the job is complete. It does this by polling a status rest endpoint with the id of the jobExecution it wants a status on.
To accomplish this flow I have implemented a rest controller that starts the camel route via a ProducerTemplate. My problem is returning the job execution id immediately after starting the camel route. I don't want the rest call to wait until the job is complete to return.
startJobViaRestCall ------> createBatchJob ----> runBatchJobUntilDone
|
|
Return jobExecutionData |
<----------------------------------
I have tried using async calls and futures, but with no luck. I have also tried to use Camels wiretap to no avail. The problem is that there is only "onComplete" events. I need an hook that returns as soon as the job has been created, but not run.
For example, the following code waits until the batch job is done before returning the JobExecution data I want to send back (as json). It makes sense as extractFutureBody will wait until the response is ready.
#RestController
#Slf4j
public class BatchJobController {
#Autowired
ProducerTemplate producerTemplate;
#RequestMapping(value = "/batch/job/start", method = RequestMethod.GET)
#ResponseBody
public String startBatchJob() {
log.info("BatchJob start called...");
String jobExecution = producerTemplate.extractFutureBody(producerTemplate.asyncRequestBody(BatchRoute.ENDPOINT_JOB_START, ""), String.class);
return jobExecution;
}
}
The camel route is a simple call to the spring-batch-component
public class BatchRoute<I, O> extends BaseRoute {
private static final String ROUTE_START_BATCH = "spring-batch:springBatchJob";
#Override
public void configure() {
super.configure();
from(ENDPOINT_JOB_START).to(ROUTE_START_BATCH);
}
}
Any ideas as to how I can return the JobExecution data as soon as it is available?
Not sure How you could do it in Camel, but here is sample Job execution using spring-rest.
#RestController
public class KpRest {
private static final Logger LOG = LoggerFactory.getLogger(KpRest.class);
private static String RUN_ID_KEY = "run.id";
#Autowired
private JobLauncher launcher;
private final AtomicLong incrementer = new AtomicLong();
#Autowired
private Job job;
#RequestMapping("/hello")
public String sayHello(){
try {
JobParameters parameters = new JobParametersBuilder().addLong(RUN_ID_KEY, incrementer.incrementAndGet()).toJobParameters();
JobExecution execution = launcher.run(job, parameters);
LOG.info("JobId {}, JobStatus {}", execution.getJobId(), execution.getStatus().getBatchStatus());
return String.valueOf(execution.getJobId());
} catch (JobExecutionAlreadyRunningException | JobRestartException | JobInstanceAlreadyCompleteException
| JobParametersInvalidException e) {
LOG.info("Job execution failed, {}", e);
}
return "Some Error";
}
}
You can make the Job async by modifying JobLauncher.
#Bean
public JobLauncher simpleJobLauncher(JobRepository jobRepository){
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(jobRepository);
jobLauncher.setTaskExecutor(new SimpleAsyncTaskExecutor());
return jobLauncher;
}
Refer the documentation for more info

#Async - In Spring independent thread closing before completing task

I have a requirement where I do some operation on GUI and once I save the data in DB I need to send a http request to a webservice. But the response to GUI should not wait for result from webservice request.
For this I am using #Async , annotation of Spring.
here is my structure
MyConroller.java
calls a method
goSaveAndCreate
(not Async) in
ServiceA.java
ServiceA has a ServiceB bean injected in it. A method ,
#Async
create()
in ServiceB is annotated with Async.
Now ServiceA.goSaveAndCreate calls a method in itself , save and calls ServiceB.create() (which is Async).
I can see in logs the a new thread is created which is executing create method. But all of a sudden logs after a particular point stop and that thread seems to have got killed or comlpeted.
#Service("MarginCalculationService")
public class ServiceA implements ServiceAI {
private static final String APPROVED = "APPROVED";
public static final String SUCCESS = "SUCCESS";
....
#Autowired
ServiceB serviceB;
public List<VV> goSaveAndCreate(String[] ids,List<XX> calList) throws Exception, DataAccessException {
try {
Pair<List<VG>,List<UU>> resultList = save(ids);
vvList = resultList.getFirst();
/*
* HKAPIL - Send Message to webService callingserviceB
*/
if(resultList.getSecond() != null){
serviceB.create(resultList.getSecond());
}
} catch (DataAccessException e) {
e.printStackTrace();
logger.error("Data Access Exception thrown during - " , e);
throw e;
} catch (Exception e){
e.printStackTrace();
logger.error("Exception thrown during " , e);
throw e;
}
System.out.println("Exiting the method in MCSERVICE");
return vvList;
}
private save(){
...
...
}
}
Second service
#Service("wireInstructionMessageService")
public class ServiceB implements ServiceBI {
#Async
#Override
public void create(List<Ralc> calcList) {
String threadName = Thread.currentThread().getName();
logger.info("Inside a different thread [" + threadName + " ] to send message ." );
..
...
otherMethod(Obj);
}
private otherMethod(Obj obj){
...
...
..
//tills this point logs are getting printed nothing after this
..
...
}
}
applciationContext.xml entry
<!-- Defines a ThreadPoolTaskExecutor instance with configurable pool size, queue-capacity, keep-alive,
and rejection-policy values. The id becomes the default thread name prefix -->
<task:executor id="MyMessageExecutor"
pool-size="5-25"
queue-capacity="100"/>
<task:annotation-driven executor="MyMessageExecutor"/>
Now I have two question
1) is there a way I can add some logs in some method which tell taht the new thread from MyExecutor is getting killed or MyExecutor is getting closed (the way we have in normal Java ExecutorSerrvice)
2) Am I using the Asyn in wrong way? Is it possible that as soon as method returns from Controller or ServiceA , ServiceB instance also is getting cleaned?
Thanks
HKapil

Resources