I am using Locust and my code looks as below
class RecommenderTasks(TaskSet):
#task
def test_recommender_multiple_platforms(self):
start = round(time.time() * 1000)
self.client.get('recommendations', name='Test')
end = round(time.time() * 1000)
print(end - start)
class RecommenderUser(FastHttpUser):
tasks = [RecommenderTasks]
wait_time = constant(1)
host = "https://my-host.com/"
When I test with this code, I get the following output times
374
62
65
68
64
I am not sure why the very first task time alone is about 300+ ms and the rest are as expected. With this, my overall average time also increases. Could you please help me here?
Locust response times are measured from the time the initial request is sent to the server to the time a response is received. By default Locust reuses socket connections when available but creates new ones if an existing one isn't available. When connecting via HTTPS, there are a number of things that need to be done to set up the connection initially. Generally performance of that connection set up is dependent on things the server is doing. You could look into ways of reducing your connection setup time. How to do that will vary widely depending on your stack but you can find general principles in SO answers like this one:
how to reduce ssl time of website
Related
I created a test in JMETER
Add > Sampler > HTTP Request = Get
Server Name = dainikbhaskar.com
No. of threads(users) = 1
Ramp-up period (seconds) = 1,
Loop Count = 1
(My internet connection is a broadband one with the speed 50 MBPS)
I ran the test, ran successful, latency comes as 127 & sometimes less than 100 in subsequent executions.
I switched off my Wi-Fi, connected my laptop with mobile hotspot & executed the same test.
Now the latency is 607, 932, 373, 542, 915
I believe it's happening due to INTERNET CONNECTION SPEED as rest of the inputs are same.
Please confirm whether my perception is correct ? :)
It is correct.
You can also get network latency from https://www.speedtest.net/ or https://fast.com/
Latency is the time from sending the request until first byte of response arrives, so called "Time to first byte"
In JMeter's world:
Latency is Connect Time + Time to send the request + time to get the first byte of response
Elapsed time is Latency + time to get the last byte of the response.
More information:
JMeter Glossary
Understanding Your Reports: Part 1 - What are KPIs?
If you get 5x times higher latency for other connection it means that the majority of time is spend for the packets to travel back and forth. You can see the more precise picture by looking at Network tab of your browser developer tools or using a special solution like Lighthouse
I'm running load tests in gatling, and noticed when I ramp 250 users over 10 seconds, the report gives me an average of 31 req/s:
val combinedScenario = scenario("Combined")
.feed(UuidFeeder.feeder)
.exec(_.set("token", token))
.exec(saveData)
.exec(processDocumentRequest)
)
val scn = List(OAuthRequest.inject(atOnceUsers(1)),
combinedScenario.inject(nothingFor(5 seconds),
rampUsers(250) over (10 seconds)));
setUp(scn).protocols(httpConf).maxDuration(60 minutes)
However, when I surround the scenario in a forever loop and put a 60 second pace in between each set of requests, the report then says I average about 8 req/s:
val combinedScenario = scenario("Combined")
.feed(UuidFeeder.feeder)
.exec(_.set("token", token))
.forever(
pace(60 seconds)
.exec(saveData)
.exec(processDocumentRequest)
)
Is this simply because the report factors in the 50 seconds in between iterations where 0 requests are being sent? Can I assume that it's still sending around 31 req/s for the short bursts of requests being sent each minute?
Yes - the reports just show what the actual throughout during the scenario was, not some hypothetical maximum. The number you get could be constrained by your scenario or by the application under test. You would need to run some experiments to confirm.
With the pace in the scenario, you should also be able to increase the number of concurrent users, based on your initial testing
I want to test a rate-limiting app with Ruby where I define different behavior based on the number of requests per second.
For example, if I see 300 request per second or more, I want it to respond with a block.
But how would I test this by generating 300 requests per second in Ruby? I understand there are hard limitations based on CPU for example, but if I kept the number well below that limitation, how would I still send something that both exceeds the threshold and stays below?
Just looping N-times doesn't guarantee me the throughput.
The quick and dirty way is to spin up 300 threads that each do one request per second. The more elegant way is to use something like Eventmachine to create requests at the required rate. With the right non-blocking HTTP library it can easily generate that level of activity.
You also might try these tools:
ab the Apache benchmarking tool, common many systems. It's very good at abusing your system.
Seige for load testing.
How about a minimal homebrew solution:
OPS_PER_SECOND = 300
count = 0
duration = 10
start = Time.now
while true
elapsed = Time.now - start
break if elapsed >= duration
delay = (count - (elapsed / OPS_PER_SECOND)) / OPS_PER_SECOND
sleep(delay) if delay > 0
do_request
count += 1
end
I am running a load testing with Jmeter and python Requests package, but get different result when I try to access the same website.
target website: http://www.somewebsite.com/
request times: 100
avg response time for Jmeter: 1965ms
avg response time for python Requests: 4076ms
I have checked response html content of jmeter and python Requests are the same. So it means they all got the correct response from website. but not sure why it has 2 times difference with each other. Is there anyone know is there any deep reason for that?
the python Requests sample code:
repeat_time = 100
url = 'http://www.somewebsite.com/'
base_time = datetime.datetime.now()
time_cost = base_time
for i in range(repeat_time):
start_time = datetime.datetime.now()
r = requests.get(url, headers=headers)
end_time = datetime.datetime.now()
print str(r.status_code) + ';time cost: %s' % (end_time - start_time)
time_cost += (end_time - start_time)
print 'total time: %s' % (time_cost - base_time)
print 'average time: %s' % ((time_cost - base_time).total_seconds() / repeat_time)
Without your JMeter code, I can't tell you what the difference is, but let me give you an idea of what's happening in that one call to requests:
We create a Session object, plus the urllib3 connection pools we use
We do a DNS look-up for 'www.somewebsite.com' which shouldn't be too negatively affecting this request
We open a socket for 'www.somewebsite.com:80'
We send the request
We receive the first byte of the response
We determine if the user wanted to stream the body of the response, if not we read all of it and cache it locally.
Keep in mind that the three most intensive parts (usually) are:
DNS lookup (for various reasons, but as I already said, it shouldn't be a problem here)
Socket creation (this is always an expensive operation)
Reading the entirety of the body and caching it locally.
That said, each response object should have an attribute, elapsed which will give you the time to the first byte of the response body. In other words, it will measure the time between when the request is actually sent and when the end of the headers is found.
That might give you far more accurate information than what you're measuring now, which is the time to the last byte of the message.
That said, keep in mind that what you're doing in that for-loop is also invoking the garbage collector a lot:
Create Session, it's adapters, the adapters connection pools, etc.
Create socket
Discard socket
Discard Session
Goto 1
If you create a session once, your script will perform better in general.
I'm noticing some weird snmp communication behavior when using MS SNMP Mgmt Api in terms of timeout and retries. I was wondering if mgmt api is supported on Win Server 2008 R1 x64. My program is a C++ 64bit snmp extension agent that uses the mgmt api to communicate with other agents as well.
This is my pseudo code:
SnmpMgrOpen(ip address, 150ms timeout, 3 retries)
start = getTickCount()
result = SnmpMgrRequest(get request with 3 or 4 OIDs)
finish = getTickCount()
if (result == some error)
{
log Error including total time (i.e finish - start ticks)
}
SnmpMgrClose()
When the snmpMgrRequest call times out, the total time equals anywhere from 1014ms to 5000ms. If, I set retries to 0, the total time is still 1014ms to 5000ms.
I would expect, with retries to 0 that the SnmpMgrRequest would timeout within 150ms. The documentation seems to imply this. Am I missing something is there a minimum timeout period of at least a second? What could be causing this behavior?
Any help would be greatly appreciated. I'm at a lost here.
ballerstyle_98#hotmail.com
From my experience with SNMP on Windows platforms the minimum timeout value is 1 second. So even if you set it to any value lower than that, it will default to 1 second.
Also the timeout value used is doubled for every retry. So with a 150ms 3 retry configuration in the worst case you will have a failed response to a request in 1+2+2+2 =7 seconds.
I hope this helps.