I Use JMeter to make load to my WebApi. 100 user + 4000 loop for every user.
Simple OWIN: avg 5-10ms, max response 200 ms II. OWIN
Microsoft Unity container: avg 15 ms, max response 9000ms.
Performance charts
I have OWIN self-hosted WebApi + Microsoft unity container.
With VS performance explorer I investigated hot Paths:
public IDependencyScope BeginScope()
{
var childContainer = Container.CreateChildContainer(); //47% CPU
return new UnityDependencyScope(childContainer);
}
public object GetService(Type serviceType)
{
if (typeof(IHttpController).IsAssignableFrom(serviceType))
{
return Container.Resolve(serviceType); //35% CPU
}
try
{
return Container.Resolve(serviceType);
}
catch
{
return null;
}
}
public void Dispose()
{
Container.Dispose(); //16%CPU
}
How to optimize Unity container use or where is problem?
If your controller depends on too many objects for construction, it may take some time building all object graph.
For every request that hit your controller, you are going to instantiate all of the dependencies even if you don't need any of them at all.
You can try splitting your one big controller into smaller multiple controllers and refactor your object graph (dependency tree) to keep it minimum per controller.
Related
Hello, Stackoverflow Community.
I have a Spring Boot application that uses Jcache with Hazelcast implementation as a cache Framework.
Each Hazelcast node has 5 caches with the size of 50000 elements each. There are 4 Hazelcast Instances that form a cluster.
The problem that I face is the following:
I have a very heavy call that reads data from all four caches. On the initial start, when all caches are yet empty, this call takes up to 600 seconds.
When there is one Hazelcast instance running and all 5 caches are filled with data, then this call happens relatively fast, it takes on average only 4 seconds.
When I start 2 Hazelcast instances and they form a cluster, then the response time gets worse, and the same call takes already 25 seconds on average.
And the more Hazelcast instances I add in a cluster, the longer the response time gets. Of course, I was expecting to see some worse delivery time when data is partitioned among Hazelcast nodes in a cluster. But I did not expect that just by adding one more hazelcast instance, the response time would get 6 - 7 times slower...
Please note, that for simplicity reasons and for testing purposes, I just start four Spring Boot Instances with each Hazelcast embedded node embedded in it on one machine. Therefore, such poor performance cannot be justified by network delays. I assume that this API call is so slow even with Hazelcast because much data needs to be serialized/deserialized when sent among Hazelcast cluster nodes. Please correct me if I am wrong.
The cache data is partitioned evenly among all nodes. I was thinking about adding near cache in order to reduce latency, however, according to the Hazelcast Documentation, the near cache is not available for Jcache Members. In my case, because of some project requirements, I am not able to switch to Jcache Clients to make use of Near Cache. Is there maybe some advice on how to reduce latency in such a scenario?
Thank you in advance.
DUMMY CODE SAMPLES TO DEMONSTRATE THE PROBLEM:
Hazelcast Config: stays default, nothing is changed
Caches:
private void createCaches() {
CacheConfiguration<?, ?> cacheConfig = new CacheConfig<>()
.setEvictionConfig(
new EvictionConfig()
.setEvictionPolicy(EvictionPolicy.LRU)
.setSize(150000)
.setMaxSizePolicy(MaxSizePolicy.ENTRY_COUNT)
)
.setBackupCount(5)
.setInMemoryFormat(InMemoryFormat.OBJECT)
.setManagementEnabled(true)
.setStatisticsEnabled(true);
cacheManager.createCache("books", cacheConfig);
cacheManager.createCache("bottles", cacheConfig);
cacheManager.createCache("chairs", cacheConfig);
cacheManager.createCache("tables", cacheConfig);
cacheManager.createCache("windows", cacheConfig);
}
Dummy Controller:
#GetMapping("/dummy_call")
public String getExampleObjects() { // simulates a situatation where one call needs to fetch data from multiple cached sources.
Instant start = Instant.now();
int i = 0;
while (i != 50000) {
exampleService.getBook(i);
exampleService.getBottle(i);
exampleService.getChair(i);
exampleService.getTable(i);
exampleService.getWindow(i);
i++;
}
Instant end = Instant.now();
return String.format("The heavy call took: %o seconds", Duration.between(start, end).getSeconds());
}
Dummy service:
#Service
public class ExampleService {
#CacheResult(cacheName = "books")
public ExampleBooks getBook(int i) {
try {
Thread.sleep(1); // just to simulate slow service here!
} catch (InterruptedException e) {
e.printStackTrace();
}
return new Book(Integer.toString(i), Integer.toString(i));
}
#CacheResult(cacheName = "bottles")
public ExampleMooks getBottle(int i) {
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
return new Bottle(Integer.toString(i), Integer.toString(i));
}
#CacheResult(cacheName = "chairs")
public ExamplePooks getChair(int i) {
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
return new Chair(Integer.toString(i), Integer.toString(i));
}
#CacheResult(cacheName = "tables")
public ExampleRooks getTable(int i) {
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
return new Table(Integer.toString(i), Integer.toString(i));
}
#CacheResult(cacheName = "windows")
public ExampleTooks getWindow(int i) {
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
return new Window(Integer.toString(i), Integer.toString(i));
}
}
If you do the math:
4s / 250 000 lookups is 0.016 ms per local lookup. This seems rather high, but let's take that.
When you add a single node then the data gets partitioned and half of the requests will be served from the other node. If you add 2 more nodes (4 total) then 25 % of the requests will be served locally and 75 % will be served over network. This should explain why the response time grows when you add more nodes.
Even simple ping on localhost takes twice or more time. On a real network the read latency we see in benchmarks is 0.3-0.4 ms per read call. This makes:
0.25 * 250k *0.016 + 0.75 * 250k * 0.3 = ~57 s
You simply won't be able to make so many calls serially over the network (even local one), you need to either
parallelize the calls - use javax.cache.Cache#getAll to reduce the number of calls
you can try enabling reading local backups via com.hazelcast.config.MapConfig#setReadBackupData so there is less requests over the network.
The read backup data feature is only available for IMap, so you would need to use Spring caching with hazelcast-spring module and its com.hazelcast.spring.cache.HazelcastCacheManager:
#Bean
HazelcastCacheManager cacheManager(HazelcastInstance hazelcastInstance) {
return new HazelcastCacheManager(hazelcastInstance);
}
See documentation for more details.
We are working with spring boot 2.0.0.BUILD_SNAPSHOT and spring boot webflux 5.0.0 and currently we cant transfer a flux to a client on request.
Currently I am creating the flux from an iterator:
public Flux<ItemIgnite> getAllFlux() {
Iterator<Cache.Entry<String, ItemIgnite>> iterator = this.getAllIterator();
return Flux.create(flux -> {
while(iterator.hasNext()) {
flux.next(iterator.next().getValue());
}
});
}
And on request I am simply doing:
#RequestMapping(value="/all", method=RequestMethod.GET, produces="application/json")
public Flux<ItemIgnite> getAllFlux() {
return this.provider.getAllFlux();
}
When I now locally call localhost:8080/all after 10 seconds I get a 503 status code. Also as at client when I request /all using the WebClient:
public Flux<ItemIgnite> getAllPoducts(){
WebClient webClient = WebClient.create("http://localhost:8080");
Flux<ItemIgnite> f = webClient.get().uri("/all").accept(MediaType.ALL).exchange().flatMapMany(cr -> cr.bodyToFlux(ItemIgnite.class));
f.subscribe(System.out::println);
return f;
}
Nothing happens. No data is transferred.
When I do the following instead:
public Flux<List<ItemIgnite>> getAllFluxMono() {
return Flux.just(this.getAllList());
}
and
#RequestMapping(value="/allMono", method=RequestMethod.GET, produces="application/json")
public Flux<List<ItemIgnite>> getAllFluxMono() {
return this.provider.getAllFluxMono();
}
It is working. I guess its because all data is already finished loading and just transferred to the client as it usually would transfer data without using a flux.
What do I have to change to get the flux streaming the data to the web client which requests those data?
EDIT
I have data inside an ignite cache. So my getAllIterator is loading the data from the ignite cache:
public Iterator<Cache.Entry<String, ItemIgnite>> getAllIterator() {
return this.igniteCache.iterator();
}
EDIT
adding flux.complete() like #Simon Baslé suggested:
public Flux<ItemIgnite> getAllFlux() {
Iterator<Cache.Entry<String, ItemIgnite>> iterator = this.getAllIterator();
return Flux.create(flux -> {
while(iterator.hasNext()) {
flux.next(iterator.next().getValue());
}
flux.complete(); // see here
});
}
Solves the 503 problem in the browser. But it does not solve the problem with the WebClient. There is still no data transferred.
EDIT 3
using publishOn with Schedulers.parallel():
public Flux<ItemIgnite> getAllFlux() {
Iterator<Cache.Entry<String, ItemIgnite>> iterator = this.getAllIterator();
return Flux.<ItemIgnite>create(flux -> {
while(iterator.hasNext()) {
flux.next(iterator.next().getValue());
}
flux.complete();
}).publishOn(Schedulers.parallel());
}
Does not change the result.
Here I post you what the WebClient receives:
value :[Item ID: null, Product Name: null, Product Group: null]
complete
So it seems like he is getting One item (out of over 35.000) and the values are null and he is finishing after.
One thing that jumps out is that you never call flux.complete() in your create.
But there's actually a factory operator that is tailored to transform an Iterable to a Flux, so you could just do Flux.fromIterable(this)
Edit: in case your Iterator is hiding complexity like a DB request (or any blocking I/O), be advised this spells trouble: anything blocking in a reactive chain, if not isolated on a dedicated execution context using publishOn, has the potential to block not only the entire chain but other reactive processes has well (as threads can and will be used by multiple reactive processes).
Neither create nor fromIterable do anything in particular to protect from blocking sources. I think you are facing that kind of issue, judging from the hang you get with the WebClient.
The problem was my Object ItemIgnite which I transfer. The system Flux seems not to be able to handle this. Because If I change my original code to the following:
public Flux<String> getAllFlux() {
Iterator<Cache.Entry<String, ItemIgnite>> iterator = this.getAllIterator();
return Flux.create(flux -> {
while(iterator.hasNext()) {
flux.next(iterator.next().getValue().toString());
}
});
}
Everything is working fine. Without publishOn and without flux.complete(). Maybe someone has an idea why this is working.
Basic Problem:
If I make an AJAX call to a controller method that performs a long computation (60 seconds or greater), I get a duplicate thread that comes in and follows the same path of execution (as best as I can tell from stack trace dumps -- and only one, this doesn't continue happening with the second thread). This appears to only happen when the controller action is called via AJAX. This can easily be replicated by creating a dummy controller method with nothing in it by a Thread.sleep() call that returns when finished.
I've tested this in a method that's loaded without an AJAX call and it doesn't produce the rogue thread. I tried various forms of AJAX calls (several forms of jQuery methods and base JavaScript) and got the same result with each. I initially thought it might be a thread problem so I implemented the dummy method using Promise(s) (http://www.playframework.com/documentation/2.1.x/JavaAsync, and http://www.playframework.com/documentation/2.1.x/JavaAkka) and AsyncResult, but it had no effect.
I know that the two threads are using the same execution context. Is that causing the problem here? Is it avoidable by moving the long computation to another context? Any ideas as to where this second, duplicate thread is coming from?
Controller Method (Long Computation):
public static Result test()
{
Logger.debug("*** TEST Controller entry: threadId=" + Thread.currentThread().getId());
StackTraceElement[] stack = Thread.currentThread().getStackTrace();
for(StackTraceElement e : stack)
{
Logger.debug("***" + e.toString());
}
Promise<Void> promiseString = Akka.future(
new Callable<Void>() {
public Void call() {
try
{
Logger.debug("*** going to sleep: threadId=" + Thread.currentThread().getId());
Thread.sleep(90000);
}
catch(InterruptedException e)
{
//swallow it whole and move on
}
return null;
}
}
);
Promise<Result> promiseResult = promiseString.map(
new Function<Void, Result>() {
public Result apply(Void voidParam) {
return ok("done");
}
}
);
return async(promiseResult);
}
We have a Windows Phone 7 application which uses a set of 3 service methods using Reactive Extensions, defined as follows:
public static class ServiceClient
{
public static IObservable<string> LookupImage(byte[] image) {...}
public static IObservable<XDocument> GetDefinition(string id) {...}
public static IObservable<Dictionary<string, byte[]>> GetFiles(string id, string[] fileNames) {...}
}
We need the WP7 application to keep calling LookupImage in the above client (each time with different set of byte[] image data) until the returned IObservable<string> is nonempty. After we get the Observable string we have to call GetDefinition and GetFiles methods (in that order).
The calls to LookupImage should happen as often as the service response is returned as opposed to being controlled by a timer as it will vary depending on network connection speed and we need to be able to send as many of these as possible.
I'd appreciate any pointers to what might be a solution to the above. As a start I have the following
private void RunLookupAndRenderLogic()
{
byte[] imageBytes = GetImageBytes();
// There are some cases where the image was not 'interesting' enough in which case GetImageBytes() returns null
if (pictureBytes != null)
{
// Where we have image data, send this to LookupImage service method
var markerLookup = ServiceClient.LookupImage(imageBytes);
markerLookup.Subscribe(id =>
{
// If the id is empty, we need to call this again.
if (String.IsNullOrEmpty(id))
{
???
}
// If we have an id, call GetDefinition and GetFiles methods of the service. No further calls to LookupImage should take place.
RenderLogic(id);
});
}
else
// If no interesting image was returned, try again
RunRecognitionAndRenderLogic();
}
Apologies if I get this wrong, but if I understand it correctly you want to Retry the call to LookupImage with the exact same argument, until it returns a value?
A naive way of solving this would be to simply call repeat and then take(1):
ServiceClient.LookupImage(imageBytes)
.Repeat()
.Take(1)
.Subscribe(id => ....);
However as Rx is single threaded by default, there is no point in this context that allows us to inject our disposal call (implicit from the Take(1)-->OnComplete()-->Auto disposal of subscription).
You can dodge this by offering some breathing space between subsequent re-subscriptions by using the CurrentThread Scheduler.
Observable.Defer(()=>
ServiceClient.LookupImage(imageBytes)
.ObserveOn(Scheduler.CurrentThread)
)
.Repeat()
.Take(1)
.Subscribe(id => ....);
There are other ways of achieving this with some good understanding of Rx and some creativity. (Most I would imagine a Scheduler)
To give you some inspriation check out the chapter on Scheduling and Threading. It covers recursion and building your own iterator which is effectively what you are trying to do.
Full code sample:
private void RunLookupAndRenderLogic()
{
byte[] imageBytes = GetImageBytes();
// There are some cases where the image was not 'interesting' enough in which case GetImageBytes() returns null
if (pictureBytes != null)
{
// Where we have image data, send this to LookupImage service method
var subscription = Observable
.Defer(()=>
ServiceClient.LookupImage(imageBytes)
.ObserveOn(Scheduler.CurrentThread)
)
.Where(id=>!String.IsNullOrEmpty(id))
.Repeat()
.Take(1)
.Subscribe(id =>
{
// If we have an id, call GetDefinition and GetFiles methods of the service. No further calls to LookupImage should take place.
RenderLogic(id);
});
//TODO: You dont offer any way to cancel this (dispose of the suscription).
//This means you could loop forever :-(
}
else
{
// If no interesting image was returned, try again
RunRecognitionAndRenderLogic();
}
}
(Disclosure: I am the author of IntroToRx.com)
Search several blogs about it but always are same examples.
I dunno if misunderstood or'm not knowing use but see no parallel process when work with AsyncController.
The following tests performed
Create a new project of type Asp.net MVC
HomeController.cs
public void IndexAsync()
{
AsyncManager.OutstandingOperations.Increment();
var bg = new BackgroundWorker();
bg.DoWork += (o, e) => GetEntriesBlog();
bg.RunWorkerCompleted += (o, e) =>
{
AsyncManager.Parameters["items"] = e.Result;
AsyncManager.OutstandingOperations.Decrement();
};
bg.RunWorkerAsync();
ViewBag.Message = "Modify this template to kick-start your ASP.NET MVC application.";
}
public ActionResult IndexCompleted(IEnumerable<SyndicationItem> items)
{
return View(items);
}
[NonAction]
public IEnumerable<SyndicationItem> GetEntriesBlog(int page = 0)
{
using (var reader = XmlReader.Create("http://blog.bindsolution.com/rss"))
{
Thread.Sleep(20000);
var rssData = SyndicationFeed.Load(reader);
if (rssData != null)
{
return (from item in rssData.Items
orderby item.PublishDate descending
select item).Take(3).Skip(3 * page).ToList();
}
return null;
}
}
Always delay 20 seconds browsing the site!
I was thinking of using PartialView AsyncController in to perform this task. Work?
I think you are misunderstanding what the Asynchronous Background worker would do.
If the operation takes 20 seconds using a background worker will not reduce that time or make the view render any faster. Using an asynchronous operations will free up the worker process on the server to process other requests while this long running request keep chugging along.
In your case I think you should create a very simple view that returns quickly to the user and kick of the long running operation as an asynch request from the client. For example, render the fast portions of your page (e.g. header, menus, et cetera) and make an AJAX request for the blog entries.
Depending on the nature of the code in GetEntriesBlog you might not need to make the controller operation asynchronous. In theory, since most of the time in this method will be spend waiting for the HTTP GET request to http://blog.bindsolution.com/rss to complete, it might be a good idea but in practice those things need to be bench marked (and perhaps under heavy load) to make sure you are getting the benefit that you expect. Keep in mind that your server code side will be more complex (and harder to maintain) if you make it asynch. I would suggest you go this route only if you do get a significant benefit.