I have a HTTP Request in a Test Plan. I am sending a POST Request with a very large body (10 MB).
I am reading the body data from a file using ${__FileToString(data.json)} and testing it with 300 threads
Running it is giving out of memory error
Each thread is storing the same (10 MB) file in RAM, which is causing the error.
Is it possible to make all the threads access the same copy instead of creating duplicates?
As of JMeter 5.5 default maximum Java Heap size is 1 GB
You would need 10 * 300 == 3 GB at least so you will need to increase the Java Heap space accordingly. See 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article for more details.
As of now there is no way to re-use the pointer to the large object between threads, you can consider using HTTP Raw Request sampler instead, it has nice feature of direct streaming of the request body to the backend without loading it into memory first
Related
I am testing REST API, each of which returns a 10 MB response body.
Now during load test, JMeter gives Out of Memory exception at 20 threads.
Concluded that this is due to APIs have huge response body. When other APIs with fairly low size Response Body are tested I am able to scale up to 500 Threads
Have tried all options shared under hacks to avoid Out of Memory exception:
Running in Non GUI mode
All Listeners Off - generating JTL from command line
Disabled all assertions, relying on application logs
Groovy used as scripting language in JSR223, using to emulate Pacing between requests
Heap Size increased to 12 GB
JMeter 5.4.1 and JDK used
Going for distributed load testing from multiple JMeter host machines also seems to have problem, as when I reduced No. of Threads to 10 for same APIs, Out of Memory Exception still came up.
How to effectively handle huge response body to a request made from JMeter ?
If you don't need to read or assert the response then you can reduce disk space usage, check Save response as MD5 hash in your HTTP Request Advanced tab
Save response as MD5 hash? If this is selected, then the response is not stored in the sample result. Instead, the 32 character MD5 hash of the data is calculated and stored instead. This is intended for testing large amounts of data.
I'm trying to test my https endpoint with JMeter. I want to make at least 10000 requests per second, but when I set the number of threads to 10000 I get way less rps, around 500.
I've tried setting the number of threads to 1000 and 100, surprisingly I get this same number of rps. I'm using HTTP Sampler and "use Keep-Alive" is set to true. When I look in the statistics I see that when using 100 threads, it makes use of Keep-Alive and connect_time is around 100 ms, but when the number of threads is higher connect_time grows, it's like it stops reusing the connections.
I know this isn't a server issue, because I've tried testing that same endpoint with Yandex.Tank and phantom and it can easily maintain 10 000 requests per second, the problem is it can't use response data to make furhter requests, that's why I have to use JMeter for this task.
This can be done by using "Stepping thread group". It will allow you to send 10000 request per second upto specified time. Refer below image.
Stepping Thread Group
Download jar from below link.
https://jmeter-plugins.org/wiki/SteppingThreadGroup/
I hope you are trying to achieve this using one machine. Try with multiple machine or jmeter distributed mode.
https://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.pdf
https://www.blazemeter.com/blog/how-to-perform-distributed-testing-in-jmeter/
https://blazemeter.com/blog/3-common-issues-when-running-jmeter-scripts-and-how-solve-them/
I am assuming that it is the issue with machine which is not able to generate that much load. Usually, i have use max 300 threads per machine but it depend on the machine config. Just check if the machine is having issue and multiple machine is able to generate more load, considering server is not having any issue.
Hope this helps.
Update:-Usually 200-500 can be handled my modern machines.
Please check the below link to have some more info:-
1.How do threads and number of iterations impact test and what is JMeter’s max. thread limit
2.https://www.blazemeter.com/blog/what%e2%80%99s-the-max-number-of-users-you-can-test-on-jmeter/ .
I am running jmeter with only one thread and it is still eventually consuming all available memory. The test is running gets, heads and puts from a large csv file >3mil rows. Some of these files are large (> 1gb) but most are average sizes, regardless of that, it seems to die well before it gets to those huge files anyway.
I have all listeners disabled and running from the console, which should also reduce the overhead. I have also given the heap 10G of memory which I would think is plenty?
I have attached the jmx file if that helps:
jmx runfile
Is there some process i can run to cleanup/purge as its processing, or am I doing something wrong in the plan its self.
sample csv:
PUT,500,path_name_here,filename_1,replication=false
GET,1500,path_name_here,filename_2,allowredirect=true
GET,500,path_name_here,filename_3,allowredirect=true
You seem to be suffering from the problem described here, try switching to the HTTP Raw Request which can upload large files without loading them fully into memory.
You can install HTTP Raw Request sampler using JMeter Plugins Manager:
JMeter machines with versions: 2.13 r13365067, 2.11.20140918 |
Java: OpenJDK 1.7.0_79 |
OS: Debian 8.1
I'm having a problem where some HTTP requests seem to be processed far too long on a load injector that isn't really under load.
Examples from result files from tests with 20 vUs (with caching, on weaker load injector, JMeter v2.11) and 40vUs (without caching, on much higher spec'd load injector, JMeter v2.13):
<time_stamp>,3257,<request_name>,200,<thread_name>,true,28537,20,20,437
<time_stamp>,5158,<request_name>,304,<thread_name>,true,138,40,40,0
Memory is at 75% in the first case, and below 50% in the second. CPU doesn't seem to spike (measured in 1 sec intervals) and goes up to 20% max in both examples.
Checked the JVM's garbage collection, and it doesn't seem like the GC is at its limits at the time of the requests either (actually at no point during the test).
I noticed this in the case where I had caching (via Cache Manager with "Use Cache-Control/Expires headers..." checked) enabled, and, like in the second example above, get the unrealistic response time of 5158 ms.
This only happens at some steps during an iteration and to more than one thread, but not all.
It seems like JMeter is somehow processing the result too long, but I can't really see that my load injectors are under heavy load, to cause processing times of seconds.
Clearly this is messing up the performance statistics so I would like to know how this is happening.
Hope someone can help.
EDIT:
#First example: Case where ResponseTime >> Latecy > 0, happens on both JMeter machines (JMeter v2.11, JMeter v2.13).
#Second example: Case where ResponseTime >> Latecy = 0 happens only on the machine with JMeter v2.13.
2nd EDIT:
Turns out it doesn't matter what JMeter version I run (or on which node).
Regex'd my result file:
Of the same requested resources, cached (latency=0), with header check, about 10% took 1 second or multiple seconds. Without header check it is 6%.
You should run same JMeter version on all nodes. If this won't solve the problem, monitor your JMeter instance resource utilisation with jconsole.
How to send HTTP post using jmeter for approx 16mb of ofx data? for one thread data is posted successfully but for multiple threads load test stops after execution of first thread. I am sending this data in the body of POST request. Also I need parameterized few of the data in ofx file data.
Just to clarify, I know how to post HTTP request with huge ofx data. But problem I am facing is- for one thread it works fine. My application takes around 25 mins to process that much data. But, in case I increase the number of threads in JMeter testplan with some ramp up time only one thread executes and jmeter stops executing next thread. Pl suggest the solution.
Did you try setting the HTTP request connect/response timeouts?
If not, this could be the solution for your issue!
This issue generally occurs due to insufficient memory/heap
The best way to overcome this issue is to increase the heap size in jmeter.bat file
From:
"set HEAP=-Xms512m -Xmx512m"
To:
set HEAP=-Xms512m -Xmx2048m
Hope this will help.