I want to profile a server written in Go. I am using "net/http/pprof", but the default behaviour is utterly useless, as it seems to only profile the goroutine running the server that serves the profiling data.
I put my server under load with siege. With 1000 concurrent users I got the profiling data that I wanted.
Related
I want to profile a simple webserver that I wrote in Go. It accepts requests, maps the request to avro object and sends it to Kafka in a go routine. The requirement is that it answers immediately and sends an object to the Kafka later. Locally it answers in under 1 ms on average. I have been trying to profile it by starting the script with davecheney/profile package and sending test requests with jmeter. I can see in the output that the profile file is generated but it remains empty, no matter how long jemeter is sending the requests. I'm running it on Mac El Capitan. I read that there were some issues with profiling on Mac but it would be working on El Capitan. Do you have any tips?
Firstly, I'm not sure whether you're trying to do latency profiling. If so, be aware that Go's CPU profiler only reports time spent by a function executing on the CPU and doesn't include time spent sleeping, etc. If CPU Profiling really is what you're looking for, read on.
If you're running a webserver, just include the following in your imports (in the file where main() is) and rebuild:
import _ "net/http/pprof"
Then, while applying load through jmeter, run:
go tool pprof /path/to/webserver/binary http://<server>/debug/pprof/profile
The net/http/pprof package provides profiling hooks that allow you to profile your webserver on demand any time, even while it's running in production. You may want to use a different, firewalled port for it, though, if your webserver is meant to be exposed publicly.
I need to stress test a Server with around 3000 users conecting to it concurrently via SyncML Clients. For simulation of each user, a application needs to be launched which then connects to the server and does some operations.
Each user corresponds to each process.
The process is unix based and does http transactions based on SyncML Protocol.
I need to run the load for these 3000 processes for an hour or so.
Can you suggest best industry methods to fulfil such requirements?
Can JMeter or Locust help me in this?
Regards
You can definitely use Locust for this.
I wouldn't recommend starting processes to generate the load (even though it's possible), mainly because you won't get detailed statistics on what requests are made, how long they take to complete, etc.
Either you could just manually do the HTTP POST requests containing the SyncML data with the built in Locust HTTP client, or you could actually take something like pysyncml, and make your own SyncML client that reports the requests it does to Locust. It's fairly simple to do, you can read more about it, and see example, on the documentation page about custom clients.
Yes, JMeter can do this, though it's not clear to me what exactly the unix based processes needs to do.
JMeter can natively make HTTP POST requests and send XML data. Unless you have some very custom logic to make the requests, stick to JMeter on it's own.
If you must, you CAN execute a local process, but then you're severely limiting the number of users you can simulate per machine.
http://jmeter.apache.org/usermanual/component_reference.html#OS_Process_Sampler
Recently, I received a PC installed LoadRunner 11.03(perhaps patch 3) from my client and watched a web performance with it by long-run test.
In multiple user test, it seems not to work on proper performance because my web's performance monitor couldn't reach any limitation, usage of CPU, network bands, disk usage per minute, usage of Memory. Only waiting threads was little bad, but it was not obvious.
It seems a sequential behavior rather than a parallel access.
(No error occured.)
So I though it was not problem of servers, but the client have some problem having prevent to be acting parallel access for some reasons.
I don't have proper HP passport ID, I can't access the LoadRunner patches' website.
Please notice me if not LoadRunner patches, especially patch 4 or higher , let it show such the above behavior or not.
Ok, it sounds like you are just running a script in VUGen. If that is the case I am guessing (based on what you wrote, correct me if I'm wrong) you are running a script in the Virtual User Generator and not in the Controller. LoadRunner is actually a suite of multiple applications. The Virtual User Generator is the script development application, a development environment like Eclipse. It is single threaded and running a script there is meant only to test the script individually.
To run a multi-threaded test you need to use the Controller app and develop a test scenario, assign multiple virtual users (the LR term for concurrent threads) to each script you want to run and execute the test from the Controller. You can configure machines to be the Load Generators (another app set up to run as a process or service) and push out the test from the Controller to the Generators.
We are experiencing slow processing of requests under heavy load. When looking at the currently running requests during these bursts I can see many requests to our web-service code.
The number of requests is not that large but they appear to be stuck in a preprocessing state. Below is an example:
We are running an IIS7 app pool in classic mode due to the need to support some legacy code.
Other requests continue to be processed but these stuck requests gradually seem to fill up the available threads leading to slow processing of other pages.
Does anyone have any idea on where these requests are getting stuck.
There appears to be no resource issue with the DB and the requests state show suggest this is all preprocessing.
We have run load tests on the code involved on local machines and can not replicate the issue.
Another possible factor is we are making use of MVC and UrlRouting.
Many thanks for any help.
Some issues only happen at production servers unfortunately, as load test can never simulate real world users.
You can try to capture hang dumps when performance is bad, and then analyze them (on your own or open a support case via http://support.microsoft.com to work with Microsoft support).
Usually you might have hit the famous thread pool bottleneck, http://support.microsoft.com/kb/821268. Dump analysis can easily tell the culprit and help locate a solution.
Why not move them into their own AppPool to separate them from the Classic ASP app - you'll then have more options to tune.
For performance testing, I would like to capture some traffic from a production server and use that as a basis to replay the request to a test server in order to simulate a realistic load in our development environment. These are all stateless queries, so no issues regarding cookies, sessions, etc.
The Apache log timestamps everything down to a 1 second resolution, but that's not fine enough granularity for our peak times. What's the best way to capture more fine-grained timestamps for replay? And is there some ab-like load generating program that can use this data to replicate load?
Use jmeter.
https://serverfault.com/questions/84041/how-can-i-replay-apache-access-logs-back-at-my-servers-to-do-real-world-load-test
http://jmeter.apache.org/usermanual/component_reference.html#Access_Log_Sampler
As far as granularity with timestamps, you're not going to get better than that. However, you can randomize the time slots within jmeter. Even if your production traffic logs show hits every second, you can tell jmeter to speed that up drastically.
You could capture the network data of a production run, parse it, and then use that as a replay mechanism comparing the results of the production run and the test run (where desired). Oren Eini (Ayende Rahien) talks about something quite similar on his blog.
I know that there is (or was) a tool that allowed you to do load/performance testing based on recorded sessions, but I can't find it right now :(.
You can also use BadBoy to capture sessions to replay w/ JMeter:
http://www.badboysoftware.biz/docs/jmeter.htm