I'm an absolute Gatling beginner, and have mostly written my simulations by copying bits and pieces of other simulations in my org's code. I searched around a bunch, and didn't see a similar question (and couldn't find anything in the gatling docs), so now I'm here.
I know Gatling has an after{} hook that will run code after the simulation finishes. I need to know how to multi-thread the after{} hook the same way the simulation is multi-threaded. Basically, can I ramp up users within the after{} hook?
My issue is: My simulation ramps up 100 users, logs them into a random account (from a list of 1000 possible accounts), and then creates 500 projects within that account. This is to test the performance of a project creation endpoint.
The problem is that we have another simulation that tests the performance of an endpoint that counts the number of projects in a given account. That test is starting to suffer because of the sheer volume of projects in these accounts (they're WAYYY more loaded than even our largest real-world account -- by orders of magnitude), so I want my "project creation" simulation to clean up the accounts when it's done.
Ideally, the after would do something like this:
after {
//ramp up 1000 users
//each user should then....
//log into an account
// delete all but N projects (where N is a # of projects close to our largest user account)
}
I have the log in code, and I can write the delete code... but how do I ramp up users within the after {} hook?
Is that even doable? Every example I've seen of the after{} hook (very few) has been something simple like printing text that the test is complete or something.
Thanks in advance!
You can use a global ConcurrentHashMap to store data from exec(session => session) blocks.
Then, you would be able to access this data from an after hook. But then, as explained in the doc, you won't be able to use the Gatling DSL in there. If you're dealing with REST APIs, you can directly use AsyncHttpClient, which Gatling is built upon.
Some people would also log in a file instead of in memory, and delete from a second simulation that would use this file as a feeder.
Anyway, I agree this is a bit cumbersome, but we don't currently consider that setting/cleaning up the system under test is the responsibility of the load test tool.
Related
I see Cypress lets us get back to the application state during a test to debug using time-travel. Is it possible to use this state snapshot as a starting point for other tests?
Imagine a UI where options in a stepper depend on previous selections in earlier steps, and many of these rely on requests to an API. To run different tests in the last step I would need to complete the earlier steps in exactly the same way each time. This can be added to the before block to make the code simpler but we still have the delay and overheads of API requests each time to get to this exact same state. Given that Cypress already stores the state at various points, can I seed future tests with the state from previous ones?
I have a web test where my requirements need a handful of different polling requests to be going on in the background. I have created a WebTestPlugin that looks for a certain context parameter to be set, and once it is, it kicks off a thread that just loops (every X seconds) firing off the configured request.
My issue is that this is not done in the context of the test, therefore the results (# of calls, duration, etc) is not part of the final report.
Is there a way to insert this data?
Rather than starting your own thread to run the background requests I suggest using the facilities of the load test. That way the results will be properly recorded. Another reason is that the threading regime of a load test is not specified by Microsoft and adding your own thread may cause issues.
You could have one scenario for the main test. Another scenario has one or more simple tests for the background polling activity. These tests could be set with a "think time between iterations" or with "test mix based on user pace" to achieve the required background rate. To get the background web tests starting at the correct time start the test with a constant load of 0 (zero) users and use a load test plugin that adjusts the number of users whenever needed. The plugin writes the required number into m_loadTest.Scenarios[N].CurrentLoad for a suitable N. This would probably be done in the Heartbeat plugin but potentially could be in any load test plugin. If may be that the TestFinished plugin can better detect when the number of users should increase.
I am working on a web application that provides its users to optionally execute long-running processes 'in background'. An example would be some long-running report generation, or deleting thousands of objects simultaneously.
I've implemented this using an ExecutorService defined as FixedThreadPool using a ThreadFactory. The ThreadFactory is built like this:
ThreadFactoryBuilder()
.setNameFormat(clientId + "-BackgroundTask-%d")
.setDaemon(true)
.setPriority(Thread.MIN_PRIORITY)
.build()
I execute the task like this:
Future<TaskStatus> future = clientExecutors.get(clientId).submit(
backgroundTask::execute);
taskFutures.put(backgroundTask.getTaskId(), future);
How can I enforce my webserver to always priorize handling new incoming requests (as fast as possible) over executing background tasks?
In other words: It should never ever happen, that a user has to wait long time while browsing the site, just because there are a lot of background-tasks executing. As you can see from above, I tried to do this by setting .setPriority(Thread.MIN_PRIORITY). However that does not seem to be sufficient.
Furthermore, as for now, I've set some arbitrary value for the FixedThreadPool size (10) and use it globally for the entire background-handling of the application (and all its customers).
Instead I would like to define a threadpool for each customer, to make sure each customer has the same privilege to run a certain amount of tasks in the background. Say, each customer has a FixedThreadPool of size 5, and on the server I'll have a max. of 50 different customers. That would add up to 250 running background tasks at the same time.
The most important requirement here is: it does not matter, how long these background-tasks need to execute (say 2 minutes, or 20 minutes). What is important, is that each customer has the ability to send 5 tasks to be executed in background, and each of those are worked on equally.
I've tested running 30 cpu-intensive background tasks and it turns out that while these are running and cpu is near 100%, new incoming requests take a very long time to be handled.
So obviously, I am doing it wrong.
Update 12.09.2017
I've read about microservices and while it sounds great I see a great challenge in splitting the necessary parts from our monolithic application. Mostly because nearly every operation might turn into a long running process given a big enough data selection.
Furthermore, wouldn't I run into the same problem with my microservice, i.e. the server running the microservice would suffer the same performance degradation. Well the only good thing would, that the rest of the web app would not suffer from it anymore.
I've read some posts about introducing Thread.sleep(1) or Thread.sleep in general into CPU-heavy operations to reduce the amount of CPU used in these operations. I've also read about someone who introduced this as an aspect so that he can even change the amount of time waited dynamically in order to have some control about how much cpu would be used.
However, my gut tells me that ain't right either. What do you think about introducing Thread.sleep to lower the amount of CPU used for a task? Is this common practice? If not, what would be the right approach?
I would highly consider changing your system architecture to offload these long-running requests to a separate instance instead of running them in-process with the general request-service application. In general I think it is an anti-pattern to handle both batch / online (or long / short running) processing in the same application instance.
Ideally you'd build a standalone microservice to handle these requests, but you could also simply just deploy X instances of your existing application, and configure your load balancer to route requests to the long running invocation paths (e.g. POST /myapp/longrunningjob) only to the instances dedicated to running these long-running processes.
I have recorded my web application through template & just to confirm that load test result which i am getting is correct? Just by increasing No of users does it give proper results? Is it enough for load testing of web application?
First of all you need to ensure that your test does what it is supposed to be doing. Recorded tests can rarely be successfully replayed, so normally you should be acting as follows:
Add View Results Tree listener and run your test with 1 user. Inspect request and response details to verify your test steps.
Perform correlation and parametrization if required.
Correlation: the process of identifying and handling any dynamic parameters. Most often people use Regular Expression Extractor for it.
Parametrization: the process of making your test data driven. For example, if your application assumes multiple authenticated users you need to store the credentials somewhere. Most commonly used test element for this is CSV Data Set Config
Make your test realistic. Virtual users simulated by JMeter need to represent real users using real browsers as close as possible with all the related stuff: cookies, headers, cache, etc. See How To Make JMeter Behave More Like A Real Browser to learn how to configure JMeter to act closer to real users. Also real users need some time to "think" between operations so make sure you are using Timers to simulate this behaviour as well.
Only after you apply the above points you should add more virtual users. Again, run your test with 2-3 users and iterations to ensure your test funcitons as designed. Once you are happy with it you can increase the load, but don't overkill your server, increase the load gradually and check the impact of the increasing load on your application, i.e. how response time, throughput and number of errors change as you increase the load. The same is applicable for decreasing the load, don't turn it off at once, decrease the number of virtual users gradually.
Building a Web Test Plan
Building an Advanced Web Test Plan
I am load testing (baseline, capacity, longevity) a bunch of APIs (eg. user service, player service, etc) using JMeter. Each of these services have several endpoints (eg. create, update, delete, etc). I am trying to figure out a good way to organize my test plans in JMeter so that I can load test all of these services.
1) Is it a good idea to create a separate JMeter Test Plan (jmx) for each of the APIs rather than creating one JMeter test plan and adding thread groups like "Thread Group for User Service", "Thread Group for Player Service", etc? I was thinking about adding one test plan per API, and then adding several Thread Groups for different types of load testing (baseline, capacity, longevity, etc).
2) When JMeter calculates the Sample Time (Response Time), does it also include the time taken by the BeanShell Processors?
3) Is it a good idea to put a Listener inside of each Simple Controller? I am using JMeter Plugins for reporting. I wanted to view the reports for each endpoint.
Answers to any or all of the questions would be much appreciated :)
I am using a structure like below for creating a test plan in JMeter.
1) I like a test plan to look like a test suite. JMeter has several ways of separating components and test requirements, so it can be hard to set a rule. One test plan is likely to be more efficient than several, and can be configured to satisfy most requirements. I find there can be alot of repetition between plans, which often means maintaining the same code in different places. Better to use modules and includes on the same plan to reduce code duplication, but includes are equivalent and can be used with test fragments to reduce duplication.
Threadgroups are best used as user groups, but can be used to separate tests any way you please. Consider the scaling you need for different pages/sites. ie User/Administrator tests can be done in different Thread Groups, so you can simulate say 50 users and 2 admins testing concurrently. Or you may distinguish front-end/back-end or even pages/sites.
2) It does not include beanshell pre- and post-processing times. (But if you use a beanshell sampler, it depends on the code)
3) listeners are expensive, so fewer is better. To separate the results, you can give each sampler a different title, and the listeners/graphs can then group these as required. You can include timestamps or indexes as part of your sampler title using variables, properties and ${__javaScript}, etc. This will cause more or less grouping depending on the implementation you choose.