Is this a valid testing for checking of how much time to load a web under test with 500 concurrent user.
I run jmeter with a 500 thread user , ramp-up period = 50 and loop count forever. with a listener with "results in table" that also record the latency.
While jmeter is running, i try to load/browse the web under test using actual browser(in my case IE8) ,
and it loads in 7 secs. but based on the latency the majority of result is 50k++.
is the 7 secs load time in actual browser is consider a "response time result"? since it is load in actual browser.
another question:
is the latency 50k is converted to sec? means 50secs. to load the web under test if we based on the jmeter result?
kindly clarify this to me please :)
In simple words, Latency is network delay (time taken by network while transferring data)
In JMeter latency is time between, when request is sent to server till first byte of response reaches the client/Jmeter. If response time is very low enough then you wont get precise measure of latency. If Response time is high then probably you will get correct measure.
In Jmeter Latency shares the measure as response time i.e. ms/seconds.
Your 7sec in browser is (Response time (Processing time + Latency) + Rendering time). In Jmeter rendering time is not present (As it is not a browser). Though your rendering is very low as compared to response time but in cases heavy content websites rendering time is comparable. Thus should be considered.
I hope this clears your doubts :)
Related
I am testing a scenario with 400 threads. Although I am almost getting no errors, I have very high average response. What can bring about this problem? Seems like server gives no time-out but gives response so late. I've addded the summary report. It is as follows:
This table doesn't tell the full story, if response time seems "so high" to you - this is definitely the bottleneck and you can report it already.
What you can do to localize the problem is:
Consider using a longer ramp-up period, i.e. start with 1 user and add 1 more user every 5 seconds (adjust these numbers according to your scenario) so you would have arrival phase, the "plateau" and the load decrease phase. This approach will allow you to correlate increasing load and increasing response time by looking at Active Threads Over Time and Response Times Over Time charts. This way you will be able to state that:
response time remains the same up to X concurrent users
after X concurrent users it starts growing so throughput is going down
after Z concurrent users response time exceeds acceptable threshold
It would also be good to see CPU, RAM, etc. usage on the server side as increased response time might be due to lack of resources, you can use JMeter PerfMon Plugin for this
Inspect your server configuration as you might need to tune it for high loads (same applies to JMeter, make sure to follow JMeter Best Practices)
Use a profiler tool on server side during the next test execution, it will show you the slowest places in your application code
I am running my jmeter script for almost a week and observed an interesting thing today. Below is the scenario:
Overview: I am gradually increasing the load on the application. In my last test I gave load of 100 users on the app and today I increased the load to 150 users.
Result of 150 users test:
Response time of the requests decreased compared to the last test. (Which is a good sign)
Throughput decreased drastically to half of what I got in the previous test with less load.
Received 225 errors while executing the test.
My questions are:
What could be the possible reason for such strange behavior of throughput? Why did throughput decrease instead of increasing with the increasing load?
Did I get good response time as many of my requests failed?
NOTE: Till 100 users test throughput was increasing with the increasing load of users.
Can anyone please help me with this question. I am a new bee in performance testing. Thanks in Advance!!
Also, would like to request if anyone can suggest good articles/site etc on finding performance bottleneck and learning crucial things in performance.
Most probably these 225 requests which failed returned failure immediately therefore average response time decreased, that's why you should be looking into i.e. Response Times Over Time chart and pay more attention to percentiles as mean response time can mask the real problem.
With regards to the bottleneck discovery, make sure to collect as much information from the server side as you can, i.e.
CPU, RAM, Network, Disk usage from JMeter PerfMon Plugin
Slow queries log from the database
"heaviest" functions and largest objects from the profiling tool for your application
How is this possible that in Jmetetr increasing number of users (threads) in my test did not changed the latency (Response time)?
I got the same latency for 100 threads and for 300 threads.
Latency is the difference between the time when a request was sent and time when the response has started to be received.
As per JMeter Glossary
JMeter measures the latency from just before sending the
request to just after the first response has been received. Thus the
time includes all the processing needed to assemble the request as
well as assembling the first part of the response, which in general
will be longer than one byte. Protocol analysers (such as Wireshark)
measure the time when bytes are actually sent/received over the
interface. The JMeter time should be closer to that which is
experienced by a browser or other application client.
Response time (= Sample time = Load time = Elapsed time) is a difference between the time when the request was sent and time when the response has been fully received.
As per JMeter Glossary
JMeter measures the elapsed time from just before sending the request
to just after the last response has been received. JMeter does not
include the time needed to render the response, nor does JMeter
process any client code, for example, Javascript.
So Response time always >= latency.
So it is possible that you may have same Latency for 100 and 300 threads but Response time will be different or increased.
If you have stable network connectivity between JMeter and Application Under Test it is expected that Latency wouldn't change not matter how many threads you kick off. It is "pure" network metric which tells how long did it take for the request to reach to the server.
Check out How to Analyze the Results of a Load Test article to see the impact of Latency for the end user
I am doing API load testing by sending 250 requests at once.
1. Configuration
Naturally, server takes longer to respond when a lot of users requests it simultaneously, this is what it says here.. As per http://jmeter-plugins.org/wiki/ResponseTimesVsThreads/. However when testing this is what I found..
2. Test
The plot above starts from right to left and as the number of active threads decrease, the response time increases.
Is active threads same as number of user requests, if so why this is happening on a consistent basis?
Update-1
Ran another test and increased the ramp-up period this time
No of threads: 200
Ramp-Up Period: 200 secs
Loop Count: 200
There are at least 2 possible explanations:
you don't have a problem, and your improvement in response times comes from caching effect related to your data being in cache after some time. Only you can validate as we don't know if you are using a large enough dataset and how long is your test lasting
you have a problem, your server is rejecting connections under load, so you have very rapid failed responses that have a very good response time. To know if it's your problem, check the response code over time or transactions over time as long as error percentage
My website is hosted on cloud. I am running JMeter from my office. Now I want to know if the throughput that I get in Summary Report contain network latency also.
I have this kind of API details in my log file.
GET mywebsite/getBday 200 67
So for all getBday requests it gives me processing time of 67ms. But my JMeter show throughput 1.20 reqs/sec and latency here is 8.5 secs (latency = Average field from Summary Report).
Can you tell me if the throughput that I get in Summary Report contain network latency also. If so, how can I exclude it?
Response time includes network latency. It measures the time the request was made to the time the response was received.
How can jmeter know how long the request spent in transit, unless the server can respond with a time the request was received?
The only way to exclude network latency from jmeter results is to measure it at the server and send back the information in the response (or by some other method).
Most servers should have monitoring software running anyway, like carbon/graphite. You can use that to measure the true server response times, and show network latency.
As I am most testing Java stacks, I use jconsole as well on the same machine as jmeter for side by side comparison of graphs to determine real server capability.
"Can you tell me if the throughput that I get in Summary Report contain network latency also."?
The answer is no - throughput is a measure of the completion rate of requests and the formula for calculating it does not include latency. See below.
Probably worth looking up a definition for throughput. JMeter provides its own :
"Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server. The formula is: Throughput = (number of requests) / (total time)."
https://jmeter.apache.org/usermanual/glossary.html