I am working on a springboot REST API.
I have one endpoint whih is in charge of doing several treatments.
I would like that each time a treatment is done it return it back right away to the caller.
so it will not wait that the entire treatments are done before returning a result.
#async is not exactly what I want because it will run a new thread for the whole endpoint and will return a result when all treatments will be done.
Even, I tried a threadPoolExecutor and I used one thread by treatment but also I cannot return each thread result right away. I have to wait that all threads end before returning the result.
So is there away that will take care of each treatment and return it own result right away when it is done?
Depending on your architecture you have a few option:
you would send the request as it is now; on the backend side you would generate a new id for the request, and send back that id to the client. Now the client could subscribe to the websocket and wait for the backend to push the id related result to the websocket.
you could use a messaging solution which would be similar to the previous one, but with Kafka, RabbitMQ, ...
you could implement a polling mechanism between the two side: you would return an id like in the previous options and the client would check the status of the request with the id periodically from the backend. When the status is completed, it could fetch the result from another endpoint using the id.
Related
I understand that delaying or yielding in the ESPAsyncWebServer library callbacks are a no-no. However, my callback function needs to query another device via the Serial port. This process is slow and will crash the ESP32 as a result.
Here is an example:
void getDeviceConfig(AsyncWebServerRequest *request) {
AsyncResponseStream *response =
request->beginResponseStream("application/json");
StaticJsonDocument<1024> doc;
JsonArray array = doc.createNestedArray("get");
for (size_t i = 0; i < request->params(); i++)
array.add(request->getParam(i)->value());
serializeJson(doc, Serial);
/* At this point, the remote device determines what is being asked for
and builds a response. This can take fair bit of time depending on
what is being asked (>1sec) */
response->print(Serial.readStringUntil('\n'));
request->send(response);
}
I looked into building a response callback. However, I would need to know ahead of time how much data the remote device will generate. There's no way for me to know this.
I also looked into using a chunked response. In this case, the library will continuously call my callback function until I return 0 (which indicates that there is no more data). This is a good start - but doesn't quite fit. I can't inform of the caller that there is definitely more data coming, I just haven't received a single byte yet. All I can do here is return 0 which will stop the caller.
Is there an alternative approach I could use here?
The easiest way to do this without major changes to your code is to separate the request and the response and poll periodically for the results.
Your initial request as you have it written would initiate the work. The callback handler would set global boolean variable indicating there was work to be done, and if there were any parameters for the work, would save them in globals. Then it would return and the client would see the HTTP request complete but wouldn't have an answer.
In loop() you'd look for the boolean that there was work to be done, do the work, store any results in global variables, set a different global boolean indicating that the work was done, and set the original boolean that indicated work needed to be done to false.
You'd write a second HTTP request that checked to see if the work was complete, and issue that request periodically until you got an answer. The callback handler for the second request would check the "work was done" boolean and return either the results or an indication that the results weren't available yet.
Doing it this way would likely be considered hostile on a shared server or public API, but you have 100% of the ESP32 at your disposal so while it's wasteful it doesn't matter that it's wasteful.
It would also have problems if you ever issued a new request to do work before the first one was complete. If that is a possibility you'd need to move to a queueing system where each request created a queue entry for work, returned an ID for the request, and then the polling request to ask if work was complete would send the ID. That's much more complicated and a lot more work.
An alternate solution would be to use websockets. ESPAsyncWebServer supports async websockets. A websocket connection stays open indefinitely.
The server could listen for a websocket connection and then instead of performing a new HTTP request for each query, the client would send an indication over the websocket that it wanted to the server to do the work. The websocket callback would work much the same way as the regular HTTP server callback I wrote about above. But when the work was complete, the code doing it would just write the result back to the client over the websocket.
Like the polling approach this would get a lot more complicated if you could ever have two or more overlapping requests.
We started a new project with Quarkus and Mutiny, and created a bunch of endpoints with Quarkus #Funq, everything has been working fine so far. Now we want to process something very time-consuming in one of the endpoints, and what we are expecting is, once user clicks a button to send the http request from frontend and hits this specific endpoint, we are going to return 202 Accepted immediately, leaving the time-consuming operation processing in another thread from backend, then send notification email accordingly to user once it completes.
I understand this can be done with #Async or CompletableFuture, but now we want to do this with Mutiny. Based on how I read Mutiny documentation here https://smallrye.io/smallrye-mutiny/guides/imperative-to-reactive, runSubscriptionOn will avoid blocking the caller thread by running the time-consuming method on another thread, and my testing showed the time-consuming codes did get executed on a different thread. However, the http request does not return immediately, it is still pending until the time-consuming method finishes executing (as I observe in the browser's developer tool). Did I misunderstand how runSubscriptionOn works? How do I implement this feature with Mutiny?
My #Funq endpoint looks like this
#Inject
MyService myService;
#Funq("api/report")
public Uni<String> sendReport(MyRequest request) {
ExecutorService executor = Executors.newFixedThreadPool(10, r -> new Thread(r, "CUSTOM_THREAD"));
return Uni.createFrom()
.item(() -> myService.timeConsumingMethod(request))
.runSubscriptionOn(executor);
}
Edit: I found the solution using Uni based on #Ladicek's answer. After digging deeper into Quarkus and Uni I have a follow-up question:
Currently most of our blocking methods are not returning Uni on Service level, instead we create Uni object from what they return (i.e. object or list), and return the Uni on Controller level in their endpoints like this
return Uni.createFrom().item(() -> myService.myIOBlockingMethod(request)).
As #Ladicek explained, I do not have to use .runSubscriptionOn explicitly as the IO blocking method will automatically run on a worker thread (as my method on Service level does not return Uni). Is there any downside for this? My understanding is, this will lead to longer response time because it has to jump between the I/O thread and worker thread, am I correct?
What is the best practice for this? Should I always return Uni for those blocking methods on Service level so that they can run on the I/O threads as well? If so I guess I will always need to call .runSubscriptionOn to run it on a different worker thread so that the I/O thread is not blocked, correct?
By returning a Uni, you're basically saying that the response is complete when the Uni completes. What you want is to run the action on a thread pool and return a complete response (Uni or not, that doesn't matter).
By the way, you're creating an extra thread pool in the method, for each request, and don't shut it down. That's wrong. You want to create one thread pool for all requests (e.g. in a #PostConstruct method) and ideally also shut it down when the application ends (in a #PreDestroy method).
We have a distributed application following microservice Architecture. In one of our microservice we are following producer-consumer pattern.
The producer receives requests, persists it to database, pushes the request into a BlockingQueue and sends the response back to the client. The consumer running on a separate thread is listening to the blocking queue. The moment it gets the request object it performs specific operations on it.
The request received by the producer is persisted to the database asynchronously using CompleteableFutures.
The problem here is how to forward TraceId to the methods processing the requestObject inside consumer thread. Since the consumer thread might process these objects much later after the response is sent to the consumer.
Also how to forward the traceId across Asynchronous calls?
Thanks
That's an interesting question. I think that what you can do is to persist the request together with its headers. Then on the consumer side you can use the SpanExtractor interface in a similar way as we do here - https://github.com/spring-cloud/spring-cloud-sleuth/blob/v1.3.0.RELEASE/spring-cloud-sleuth-core/src/main/java/org/springframework/cloud/sleuth/instrument/web/TraceFilter.java#L351 (Span parent = spanExtractor().joinTrace(new HttpServletRequestTextMap(request));). That means that from the HttpServletRequest we're extracting values to build a span. Then, once you've retrieved the Span, you can just use Tracer#continueSpan(Span) method before processing, and then Tracer#detach(Span) in the finally block. E.g.
Span parent = spanExtractor().joinTrace(new HttpServletRequestTextMap(request));
try {
tracer.continueSpan(parent);
// do whatever you need
} catch(Exception e) {
tracer.addTag("error", doSthWithTheExceptionMsg(e));
} finally {
tracer.detach(parent);
}
I'm experimenting with microservices, event sourcing and CQRS. However, I'm a little bit confused about how I go from issuing a command to performing a query to return the new state, specifically with regard to interactions with a web API gateway.
As an example, the simple application I am attempting to write (which probably doesn't actually need any of these; it is just something to aid my learning) creates a random-graph and then performs some long-running calculations on the graph. I've modelled this as two separate services: the GraphService and the ComputationService. The imagined process flow is as follows:
User requests new random graph.
API gateway constructs CreateGraph command and sends it to the
graph service.
GraphService command handler creates a graph and publishes a
GraphCreated event.
GraphService event handler subscribes to topic for graph events,
processes GraphCreated event and stores graph in persistent read
storage.
Client somehow gets the newly created graph.
ComputationService event handler subscribes to topic for graph
events, processes GraphCreated event and begins potentially
long-running computation, e.g. calculate diameter.
ComputationService publishes DiameterComputed event.
GraphService event handler subscribes to topic for computation
events, processed DiameterComputed event and updates the graph in
persistent read storage.
Client somehow gets updated - easier than getting the new graph, since already have an ID and can poll for changes / websockets / SSE, etc.
That seems relatively simple. However, my confusion lies in how to go about informing the API gateway, and thus the web client, of the new graph (as highlighted in bold above). In a typical CRUD process, the result of the POST request to create a new graph would be to return the URL of the new resource, for instance. However, with CQRS, commands should return nothing or an exception.
How do I pass information back to the client of the service (in this case the API gateway) about the ID of the new graph so that it can perform a query to get the representation of the new resource and send it to the user? Or at least get an ID so that the web client can ask the API gateway, etc?
As I see it at the moment, after sending a command, everyone is just left hanging. There needs to be some sort of subscription model that can be interrogated for the status of the graph creation. I considered having the API gateway generate a request ID which gets embedded with the CreateGraph command, but this then couples the service to the API.
I'm obviously missing something, but have no idea what. None of the examples I've looked at or discussions I've read address this issue and assume that the ID of whatever resource is already known. I couldn't find any discussions here addressing this issue, but if I've just missed them, please point me there rather than duplicating questions. Any pointers would be hugely welcomed.
How do I pass information back to the client of the service (in this case the API gateway) about the ID of the new graph so that it can perform a query to get the representation of the new resource and send it to the user? Or at least get an ID so that the web client can ask the API gateway, etc?
By listening for the echo.
The basic idea behind at least once delivery is that I'm going to send you a message, and keep sending it over and over until I receive a message that proves you've received at least one copy of my message.
Therefore, my protocol looks something like
Establish a mail box where I can collect messages
Encode into the message instructions for delivering to my mailbox
Send the message to you
Check my mailbox
if the answer is there, I'm done
otherwise, I send you another copy of the message
The mail box could be implemented any number of ways -- it could be a callback; it could be a promise, it could be a correlation identifier. You could have the signal dispatched by the command handler, when it gets acknowledgement of the write by the book of record, or by the "read model" when the new resource is available.
I am facing the following error while using spring websockets:
Use case: On our server side code we have a fucntionality to search values in data base..if the values are not present in database..it will hit a servlet and get the data..The second part i.e., hitting servlet and getting the data is asynchronous call.
So for one request there are multiple things we have to search for in data base..
Example: In request we got a some parameter channel: 1
This channel is mapped to multiple ids say 1 is mapped to 1,2,3,4,5
In websocket once the request comes to server i will extract the channel and get all the id's mapped and run a loop over id's as follows:
for(int i=0;i<ids.length;i++)
{
SomeObject databaseRespObj=callToDatabase(i); //SomeObject contains two fields value exists and string values
if(!databaseRespObj.valuesExists)
{
AsynchronouscallToServelt(i);
//once response received it will send message immediately using session
}
}
While execution of above server side code,some times only i am facing the below error.
java.lang.IllegalStateException: Blocking message pending 10000 for BLOCKING
at org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint.lockMsg(WebSocketRemoteEndpoint.java:130) ~[websocket-common-9.3.8.v20160314.jar:9.3.8.v20160314]
at org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint.sendString(WebSocketRemoteEndpoint.java:379) ~[websocket-common-9.3.8.v20160314.jar:9.3.8.v20160314]
at org.springframework.web.socket.adapter.jetty.JettyWebSocketSession.sendTextMessage(JettyWebSocketSession.java:188) ~[spring-websocket-4.2.4.RELEASE.jar:4.2.4.RELEASE]
at org.springframework.web.socket.adapter.AbstractWebSocketSession.sendMessage(AbstractWebSocketSession.java:105) ~[spring-websocket-4.2.4.RELEASE.jar:4.2.4.RELEASE]
Sorry if the above framing of the question is not clear.Will spring support sending asynchronous messages like normal javax websocket does Session.getAsyncRemote().sendText(String text)
What is the configuration made in spring to send asynchronous messages using websocket session
From what I understand, you have multiple threads sending messages on the same RemoteEndpoint when the asynchronous technique kicks in.
Seems very similar to this :
WebSocket async send can result in blocked send once queue filled
I don't thing you necessarily have to use Futures or mechanisms described in the above post.
What I don't really get is : why doing asynchronous call to servlets ? Ofcourse several could send messages on the same RemoteEndPoint..
But can't you simply make synchronous calls to the relevant classes and keep the same request-response flow that you use when records are found in your database ? :)
UPDATE
Since you added in comments the fact that you need to focus on speed, and since it seems that your current solution is not applicable, let's maybe have a look at the problem from a different angle.
I'm not a websocket expert but as far as I understand what you try to achieve with the asynch servlet calls is not possible.
However, if you change the design/config of your project, this should be achievable.
Personally I use Websockets to be able to send a message to an arbitrary user that did not necessarily made a request - as long as he is connected, he must get the message.
To do this, I simply use the SimpMessagingTemplate class offered by Spring in their websocket support. To send a message to ANY USER THAT I WANT, I do this :
#Autowired
SimpMessagingTemplate smt;
(.......)
smt.convertAndSendToUser(recipient.getUsername(), "/queue/notify", payload);
So in your case, you could, in your loop :
make class instance method calls (instead of a servlet, no network transit, you cannot be faster ! Just a call to your biz-logic / service / whatever)
every time a method returns data, use the SimpMessagingTemplate like in the snippet here above :)
you can still do it asynchronously if you want ! :)
By doing this, you reduce latency (calling servlets adds alot), and have a reliable technique.
You can easily and quickly send thousands of messages to one user or to several users, at your own discretion, without stumbling upon the "10000 for BLOCKING" problem, which probably comes from more than 1 servlet "answering the same question" ;)
To obtain a SimpMessagingTemplate from Spring, you will need to use the <websocket:message-broker> tag or the equivalent non-xml-java-config.
I suggest to check this doc which has more info :
http://docs.spring.io/spring/docs/current/spring-framework-reference/html/websocket.html
and the following post I created on SO where I am using it (I've got another problem in the post, related to spring configuration and context hierarchies, but at least you have some template code to look at, the code is working) :
Spring Websocket : receiving nothing from SimpMessagingTemplate
Spring : how to expose SimpMessagingTemplate bean to root context ?