For performance testing, I would like to capture some traffic from a production server and use that as a basis to replay the request to a test server in order to simulate a realistic load in our development environment. These are all stateless queries, so no issues regarding cookies, sessions, etc.
The Apache log timestamps everything down to a 1 second resolution, but that's not fine enough granularity for our peak times. What's the best way to capture more fine-grained timestamps for replay? And is there some ab-like load generating program that can use this data to replicate load?
Use jmeter.
https://serverfault.com/questions/84041/how-can-i-replay-apache-access-logs-back-at-my-servers-to-do-real-world-load-test
http://jmeter.apache.org/usermanual/component_reference.html#Access_Log_Sampler
As far as granularity with timestamps, you're not going to get better than that. However, you can randomize the time slots within jmeter. Even if your production traffic logs show hits every second, you can tell jmeter to speed that up drastically.
You could capture the network data of a production run, parse it, and then use that as a replay mechanism comparing the results of the production run and the test run (where desired). Oren Eini (Ayende Rahien) talks about something quite similar on his blog.
I know that there is (or was) a tool that allowed you to do load/performance testing based on recorded sessions, but I can't find it right now :(.
You can also use BadBoy to capture sessions to replay w/ JMeter:
http://www.badboysoftware.biz/docs/jmeter.htm
Related
I have excecated a test and got the following report and duration the analysis I noticed that the sum of the individual request data is not matching up to the transaction enter image description here
Help me around here to identify the cause of this issue
Note: For running one or few users I am not facing this issue, only during on higher user count this issue is coming up
Most probably there is an issue with the way you "excecate" the test (whatever it means)
Given the issue is not reproducible with "few" users and only happens under the load my expectation is that JMeter doesn't have enough headroom to operate in terms of CPU/RAM/etc. or it's not properly configured for high loads.
Make sure to follow JMeter Best Practices
Make sure to set up monitoring of resources like CPU, RAM, Network, Disk IO, swap, etc. If you don't have other/better monitoring toolchain you can consider using JMeter PerfMon Plugin
If after following JMeter Best Practices the error is still there or resource consumption is way too high - consider going for Distributed Testing
If even after switching to distributing testing the issue is still there check jmeter.log for any suspicious entries
What can be the reason for difference in results of load test run at different times with SAME bandwidth?
If I run load test at midnight the response times would be better and during they are real bad. Thanks for your help.
Maybe during the day the application is being used by real users and your artificial load is being added to the natural load?
Another option is that network is more busy during the day so channel bandwidth is fully utilized.
Load testing tool itself metrics don't tell the full story, you can only make assumptions by looking at TCP connect time metric.
If you have an APM system in place you can assess what's going on with the system during the daytime and night time and detect the factors which are impacting the response time. If you don't - you can set up your own by using i.e. JMeter PerfMon Plugin
Adding to Dmitri's note, there could be multiple reasons / causes for difference in results.
As Dmitri pointed check your APM tool to see server health while tests is executing
Do you integrate with any downstream applications? Do these applications reside in a stable and dedicated perf testing environment or they are live production environments? if it is later, then you should expect a latency in response during day time
Authentication / token validation - usually gateways are configured to validate incoming bearer token. when you execute during morning time there is a possibility that your gateway could be busy serving other real users requests (assuming this a production AD / Okta / PingID servers)
Jmeter log replay taking more time .. can we able to use access log sampler in multiple thread? if so how to share the log files to all the threads?
Please help.
You can convert your access log into a CSV file with a script and use CSV Data Set Config in conjunction with the HTTP Request samplers - this way you will get it replayed with the multiple threads. See Stop Making Assumptions! Learn How to Replay Your Production Traffic With JMeter article to get the idea in more details and see the example implementation.
However be aware that this way most likely you won't be able to replay complex end-to-end scenarios, only simple HTTP GET requests which are not authenticated can be replayed this way. For quick-and-dirty test plan it might be enough, however it is much better to come up with modular test plan, reusable components, full control of workload, etc. so I would rather recommend building a web test plan from scratch.
I use JMeter for checking load testing.
I note a time with stopwatch when i check load time personally it was
8.5 seconds
when i run same case with JMeter it gave load time of 2 seconds
There is huge difference between them, How can i verify the actual time?
e.g : if one user taking 9 seconds to load the form while in JMeter it is given load time 2 seconds
Client time is a complex item, as you can see from the clip from the Chrome Developer tools, performance tab, above. There's lots going on at the client which does lead to a difference between the time you see with an HTTP protocol test tool, such as JMETER (and most of the other performance test tools on the planet) and the actual client render.
You can address this Delta in a number of ways:
Run a single GUI Virtual user. Name your timing records such as "Login" and "login_GUI." The delta between the two is your client weight. Make sure to run the GUI virtual user on a dedicated host to avoid resource contention
Run a test with all browsers. This was state of the art in 1995. Because of the resource cost and the skew imposed on trying to figure out the cost of the server response the entire industry shifted to protocol level virtual users. Some are trying to bring back this model as "state of the art." It is not
Ask a performance question earlier, also known as "shift left..." Every developer has these developer tools at their disposal, as does every functional tester. If you find that a client is slow for one user, be curious and use the developer tools to identify, "why?" If you are waiting to multi user performance testing to answer questions related to client weight, then you have waited too long and often will not have the time or resources to change the page architecture in meaningful ways to reduce the client page cost. This is where understanding earlier has tremendous advantages for making changes.
I picked the graphic above deliberately to illustrate the precise challenge you have. Notice, the loading of the components takes less than a tenth of a second. These are the requests that JMETER would be making. But the page takes almost five seconds to "render." Jmeter is not broken, it is working as designed. It is your understanding that needs to change on which tools can be used to pull particular stats for analysis.
You can't compare JMeter load time to browser as is, also because your browser will load JavaScript files and can call JavaScript functions on page load while JMeter doesn't execute JavaScript.
JMeter is not a browser, it works at protocol level. As far as
web-services and remote services are concerned, JMeter looks like a
browser (or rather, multiple browsers); however JMeter does not
perform all the actions supported by browsers. In particular, JMeter
does not execute the Javascript found in HTML pages. Nor does it
render the HTML pages as a browser does (it's possible to view the
response as HTML etc., but the timings are not included in any
samples, and only one sample in one thread is ever displayed at a
time).
Just a side note - you can use plugin to check exact load time in chrome.
Well-behaved JMeter test timing should be equal or similar to real user timing, if there is a 4x times difference - most probably your JMeter configuration is not correct.
Probably the most important. Make sure your HTTP Request samplers are configured to retrieve so called "embedded resources" (images, scripts, styles) which are referenced in the web page
If your application is using AJAX technology make sure you execute AJAX-driven requests as well and add their elapsed time to main sampler using i.e. Transaction Controller.
Make sure you mimic browser's:
Cookies via HTTP Cookie Manager
Headers via HTTP Header Manager
Cache via HTTP Cache Manager
Assuming all above you should be receiving similar to real user experience page load time. See How to make JMeter behave more like a real browser article for more detailed information on the above tips.
In addition to the answers provided by James and user7294900, please find these images to help you understand the reason behind the difference in time given by your stop watch and JMeter.
Below image gives the ideology behind how JMeter provides the time.
Below image gives the ideology behind how you have measured the time with
your stop watch.
Notice that there are additional actions performed by the browser when you are taking the time using your stop watch. This is the reason behind the huge difference in time between JMeter and your stop watch.
In addition to this, ensure that you are using the same test environmental conditions for both the tests (like same network conditions, same LG etc.)
Hope this helps!
Which tools can be used to measure performance of a webserver?
To test a webserver, you can use Apache Jmeter.
To see where is the bottleneck you have to flood your server application.
ApacheBench (ab) can do this. Here is a tool to get the server HTTP response code (ab) just says there is an HTTP error, and to automate test runs:
dsec.com/source/ab.c.txt
This program also gives useful tips about how to configure Linux and Windows (TCP/IP system options) to get the best possible performances.
It always depends on the setup.
Depending on the application there can be different bottlenecks.
Sometimes its the CPU, sometimtes the database connections, sometimes the sockets, sometimes the hard disc etc...
Most common practice is to use siege (simple command line tool) and increase the concurrent connections and see how many transactions per second go through.
It will increase per connection until an optimum is reached, then it will slowly decrase.
You can produce a set of urls that are randomly accessed, maby biased and/or send random data, request random ids etc to simulate more "real" clients.
Completely depends on your application whether this is relevant.