I want to see a "proof" that a controller returning a DeferredResult is really better scaling up to many parallel request, than return just a plain value. To this end I came up with an example like this:
#RestController
#RequestMapping("weather")
public class WeatherController {
ExecutorService executorService;
public WeatherController() {
executorService = Executors.newFixedThreadPool(100);
}
#GetMapping("deferred")
public DeferredResult<Weather> getWeatherDeferred() {
DeferredResult<Weather> result = new DeferredResult<>();
ListenableFutureTask<Weather> futureWeatherList = new ListenableFutureTask<>(
() -> {
Thread.sleep(1000);
return new Weather();
});
futureWeatherList.addCallback((weather) -> {
result.setResult(weather);
}, (throwable) -> {
result.setErrorResult(throwable);
});
executorService.submit(futureWeatherList);
return result;
}
#GetMapping("sync")
public Weather getWeather() throws InterruptedException {
Thread.sleep(1000);
return new Weather();
}
}
When I send requests to the deferred and the sync getters from JMeter, I see no difference between the two methods whatsoever. I simulate 100 parallel users and run the requests 100 times. Absolutely no difference.
Can anybody give me an explanation? Does anybody have a better "proof" for the "superiority" of DeferredResults?
Related
I have a spring Webflux Annotated controller as below,
#RestController
public class TestBlockingController {
Logger log = LoggerFactory.getLogger(this.getClass().getName());
#GetMapping()
public Mono<String> blockForXSeconds(#RequestParam("block-seconds") Integer blockSeconds) {
return getStringMono();
}
private Mono<String> getStringMono() {
Integer blockSeconds = 5;
String type = new String();
try {
if (blockSeconds % 2 == 0) {
Thread.sleep(blockSeconds * 1000);
type = "EVEN";
} else {
Thread.sleep(blockSeconds * 1000);
type = "ODD";
}
} catch (Exception e) {
log.info("Got Exception");
}
log.info("Type of block-seconds: " + blockSeconds);
return Mono.just(type);
}
}
How do I make getStringMono run in a different thread than Netty server threads. The problem I am facing is that as I am running in server thread I am getting basically less throughput (2 requests per second). How do I go about making running getStringMono in a separate thread.
You can use subscribeOn operator to delegate the task to a different threadpool:
Mono.defer(() -> getStringMono()).subscribeOn(Schedulers.elastic());
Although, you have to note that this type of blocking should be avoided in a reactive application at any cost. If possible, use a client which supports non-blocking IO and returns a promise type (Mono, CompletableFuture, etc.). If you just want to have an artificial delay, then use Mono.delay instead.
You can use Mono.defer() method.
The method signature is as:
public static <T> Mono<T> defer(Supplier<? extends Mono<? extends T>> supplier)
Your Rest API should look like this.
#GetMapping()
public Mono<String> blockForXSeconds(#RequestParam("block-seconds") Integer blockSeconds) {
return Mono.defer(() -> getStringMono());
}
The defer operator is there to make this source lazy, re-evaluating the content of the lambda each time there is a new subscriber. This will increase your API throughput.
Here you can view the detailed analysis.
Thanks for reading ahead of time. In my main method I have a PublishSubscribeChannel
#Bean(name = "feeSchedule")
public SubscribableChannel getMessageChannel() {
return new PublishSubscribeChannel();
}
In a service that does a long running process it creates a fee schedule that I inject the channel into
#Service
public class FeeScheduleCompareServiceImpl implements FeeScheduleCompareService {
#Autowired
MessageChannel outChannel;
public List<FeeScheduleUpdate> compareFeeSchedules(String oldStudyId) {
List<FeeScheduleUpdate> sortedResultList = longMethod(oldStudyId);
outChannel.send(MessageBuilder.withPayload(sortedResultList).build());
return sortedResultList;
}
}
Now this is the part I'm struggling with. I want to use completable future and get the payload of the event in the future A in another spring bean. I need future A to return the payload from the message. I think want to create a ServiceActivator to be the message end point but like I said, I need it to return the payload for future A.
#org.springframework.stereotype.Service
public class SFCCCompareServiceImpl implements SFCCCompareService {
#Autowired
private SubscribableChannel outChannel;
#Override
public List<SFCCCompareDTO> compareSFCC(String state, int service){
ArrayList<SFCCCompareDTO> returnList = new ArrayList<SFCCCompareDTO>();
CompletableFuture<List<FeeScheduleUpdate>> fa = CompletableFuture.supplyAsync( () ->
{ //block A WHAT GOES HERE?!?!
outChannel.subscribe()
}
);
CompletableFuture<List<StateFeeCodeClassification>> fb = CompletableFuture.supplyAsync( () ->
{
return this.stateFeeCodeClassificationRepository.findAll();
}
);
CompletableFuture<List<SFCCCompareDTO>> fc = fa.thenCombine(fb,(a,b) ->{
//block C
//get in this block when both A & B are complete
Object theList = b.stream().forEach(new Consumer<StateFeeCodeClassification>() {
#Override
public void accept(StateFeeCodeClassification stateFeeCodeClassification) {
a.stream().forEach(new Consumer<FeeScheduleUpdate>() {
#Override
public void accept(FeeScheduleUpdate feeScheduleUpdate) {
returnList new SFCCCompareDTO();
}
});
}
}).collect(Collectors.toList());
return theList;
});
fc.join();
return returnList;
}
}
Was thinking there would be a service activator like:
#MessageEndpoint
public class UpdatesHandler implements MessageHandler{
#ServiceActivator(requiresReply = "true")
public List<FeeScheduleUpdate> getUpdates(Message m){
return (List<FeeScheduleUpdate>) m.getPayload();
}
}
Your question isn't clear, but I'll try to help you with some info.
Spring Integration doesn't provide CompletableFuture support, but it does provide an async handling and replies.
See Asynchronous Gateway for more information. And also see Asynchronous Service Activator.
outChannel.subscribe() should come with the MessageHandler callback, by the way.
I have a doubt about this code:
#Async
public CompletableFuture<String> doFoo() {
CompletableFuture<String> fooFuture = new CompletableFuture<>();
try {
String fooResult = longOp();
fooFuture.complete(fooResult);
} catch (Exception e) {
fooFuture.completeExceptionally(e);
}
return fooFuture;
}
The question is: does doFoo return fooFuture only after longOp has finished (either correctly or exceptionally) and is therefore returning already completed futures or is Spring doing some magic and returning before executing the body? If the code is blocking on longOp(), how would you express that the computation is being fed to an executor?
Perhaps this? Any other way?
#Async
public CompletableFuture<String> doFoo() {
CompletableFuture<String> completableFuture = new CompletableFuture<>();
CompletableFuture.runAsync(() -> {
try {
String fooResult = longOp();
completableFuture.complete(fooResult);
} catch (Exception e) {
completableFuture.completeExceptionally(e);
}
});
return completableFuture;
}
Spring actually does all of the work behind the covers so you don't have to create the CompletableFuture yourself.
Basically, adding the #Async annotation is as if you called your original method (without the annotation) like:
CompletableFuture<User> future = CompletableFuture.runAsync(() -> doFoo());
As for your second question, in order to feed it to an executor, you can specify the exectutor bean name in the value of the #Async annotation, like so:
#Async("myExecutor")
public CompletableFuture<User> findUser(String usernameString) throws InterruptedException {
User fooResult = longOp(usernameString);
return CompletableFuture.completedFuture(fooResult);
}
The above would basically be the following as if you called your original method, like:
CompletableFuture<User> future = CompletableFuture.runAsync(() -> doFoo(), myExecutor);
And all of your exceptionally logic you would do with the returned CompletableFuture from that method.
The idea of using CompletableFuture is because it offers a chain, while the first several steps encapsulate beans before the last step uses it. Because any exception may happen in these steps and exceptionally is used to handle error. However, exceptionally only accepts Throwable argument and so far I haven't found a way to grab those encapsulated beans.
CompletableFuture.supplyAsync(this::msgSource)
.thenApply(this::sendMsg).exceptionally(this::errorHandler).thenAccept(this::saveResult)
public List<Msg> msgSource() // take message from somewhere.
public List<Msg> sendMsg(List<Msg>) // exceptions may happen like 403 or timeout
public List<Msg> errorHandler() // set a success flag to false in Msg.
public void saveResult(List<Msg>) // save send result like success or false in data center.
In the above example, comments are the working flow. However, since errorHandler neither accepts List<Msg> nor passes it on, so the chain is broken. How to get the return from msgSource?
EDIT
public class CompletableFutureTest {
private static Logger log = LoggerFactory.getLogger(CompletableFutureTest.class);
public static void main(String[] args) {
CompletableFutureTest test = new CompletableFutureTest();
CompletableFuture future = new CompletableFuture();
future.supplyAsync(test::msgSource)
.thenApply(test::sendMsg).exceptionally(throwable -> {
List<String> list = (List<String>) future.join(); // never complete
return list;
}).thenAccept(test::saveResult);
try {
future.get();
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
private List<String> saveResult(List<String> list) {
return list;
}
private List<String> sendMsg(List<String> list) {
throw new RuntimeException();
}
public List<String> msgSource() {
List<String> result = new ArrayList<>();
result.add("1");
result.add("2");
return result;
}
}
A chain implies that each node, i.e. completion stage, uses the result of the previous one. But if the previous stage failed with an exception, there is no such result. It’s a special property of your sendMsg stage that its result is just the same value as it received from the previous stage, but that has no influence on the logic nor API design. If sendMsg fails with an exception, it has no result that the exception handler could use.
If you want to use the result of the msgSource stage in the exceptional case, you don’t have a linear chain any more. But CompletableFuture does allow to model arbitrary dependency graphs, not just linear chains, so you can express it like
CompletableFuture<List<Msg>> source = CompletableFuture.supplyAsync(this::msgSource);
source.thenApply(this::sendMsg)
.exceptionally(throwable -> {
List<Msg> list = source.join();
for(Msg m: list) m.success = false;
return list;
})
.thenAccept(this::saveResult);
However, there is no semantic difference nor advantage over
CompletableFuture.runAsync(() -> {
List<Msg> list = msgSource();
try {
list = sendMsg(list);
} catch(Throwable t) {
for(Msg m: list) m.success = false;
}
saveResult(list);
});
which expresses the same logic as an ordinary code flow.
I'd like to stream out my content using OutputStream or Writer from Spring MVC controller method using async solution, i.e. not to block base threadpool used for http requests. As far as I could find is to use DefferedResult<?> for async in general. It's fine when you return a view string name but can't think of a way it would work with stream. Could not find anything helpful.
Thanks
As described here you can perform computation in another thread thereby unloded http thread pool.
You can try to combine DefferedResult and byte[] (DefferedResult) return type(previously registering ByteArrayHttpMessageConverter). So the final method will look like this:
#ResponseBody
public DefferedResult<byte[]> foo(HttpServlet response) {
//set headers using response
response.setContentType("someContentType");
...
DefferedResult<byte[]> r = new DefferedResult<>();
executionService.submit(() -> {
r.setResult(getBytes());
});
return r;
}
Another option is to combine Defferedresult and ResponseEntity. Do not forget to use it in servlet 3.0+ container
It may be that what you are looking for is the following. Not sure if it blocks the http thread pool though.
#Controller
public class TestController {
#RequestMapping("/")
public StreamingResponseBody handleRequest () {
return new StreamingResponseBody() {
#Override
public void writeTo (OutputStream out) throws IOException {
for (int i = 0; i < 1000; i++) {
out.write((Integer.toString(i) + " - ")
.getBytes());
out.flush();
try {
Thread.sleep(5);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
};
}
}