for testing purposes I measure the time it takes for parsing, db accessing, posting and rendering of one of my web php web pages in the browser (by using Firebug's network tool). When I press F5 after clearing the cache by "Delete recent data" it takes about 5 seconds, when I hit Ctrl-F5 it takes about 20 seconds.
Isn't that the same? What's the difference between them? What is the recommended way to test the performance of php code and db access?
Thank you very much in advance ...
There could be any number of reasons all of which have to do with the implementation of firebug.
You cannot test the performance on the client side since clients differ a lot and also have the network latency which is even harder rule out.
You should do this all on the server side: start a timer when the request reaches your web server and then stop it when it exits. If that is a bit difficult then in the PHP script itself you can run a wrapper script that has a start timer, a require statement for the script you want and a stop timer.
Related
I'm in the process of writing an app that builds a table of Trello card data based on multiple API calls, and while the app works I'm finding the performance degrades considerably the longer it runs. The initial calls take a couple of seconds while later calls (after 100 runs or so) take upwards of a minute.
Looking at the XHR Network tab on my Chrome console, I can see the bulk of the call is taken by the 'Content Download' phase of the Ajax call. I'm curious as to whether this means the issue is with my application or if the problem resides with the API I'm trying to call? I'm a bit of a novice so my terminology is probably not appropriate here.
The Content Download time is the time during which your content is downloaded from the server.
Very long time can be due to slow connection client-side or server-side.
As you can see TTFB (time to first byte) is about 200ms. So your server is starting sending data after 200ms. Your server process seems to be OK.
You can click on the Explanation link for further information.
I use JMeter for checking load testing.
I note a time with stopwatch when i check load time personally it was
8.5 seconds
when i run same case with JMeter it gave load time of 2 seconds
There is huge difference between them, How can i verify the actual time?
e.g : if one user taking 9 seconds to load the form while in JMeter it is given load time 2 seconds
Client time is a complex item, as you can see from the clip from the Chrome Developer tools, performance tab, above. There's lots going on at the client which does lead to a difference between the time you see with an HTTP protocol test tool, such as JMETER (and most of the other performance test tools on the planet) and the actual client render.
You can address this Delta in a number of ways:
Run a single GUI Virtual user. Name your timing records such as "Login" and "login_GUI." The delta between the two is your client weight. Make sure to run the GUI virtual user on a dedicated host to avoid resource contention
Run a test with all browsers. This was state of the art in 1995. Because of the resource cost and the skew imposed on trying to figure out the cost of the server response the entire industry shifted to protocol level virtual users. Some are trying to bring back this model as "state of the art." It is not
Ask a performance question earlier, also known as "shift left..." Every developer has these developer tools at their disposal, as does every functional tester. If you find that a client is slow for one user, be curious and use the developer tools to identify, "why?" If you are waiting to multi user performance testing to answer questions related to client weight, then you have waited too long and often will not have the time or resources to change the page architecture in meaningful ways to reduce the client page cost. This is where understanding earlier has tremendous advantages for making changes.
I picked the graphic above deliberately to illustrate the precise challenge you have. Notice, the loading of the components takes less than a tenth of a second. These are the requests that JMETER would be making. But the page takes almost five seconds to "render." Jmeter is not broken, it is working as designed. It is your understanding that needs to change on which tools can be used to pull particular stats for analysis.
You can't compare JMeter load time to browser as is, also because your browser will load JavaScript files and can call JavaScript functions on page load while JMeter doesn't execute JavaScript.
JMeter is not a browser, it works at protocol level. As far as
web-services and remote services are concerned, JMeter looks like a
browser (or rather, multiple browsers); however JMeter does not
perform all the actions supported by browsers. In particular, JMeter
does not execute the Javascript found in HTML pages. Nor does it
render the HTML pages as a browser does (it's possible to view the
response as HTML etc., but the timings are not included in any
samples, and only one sample in one thread is ever displayed at a
time).
Just a side note - you can use plugin to check exact load time in chrome.
Well-behaved JMeter test timing should be equal or similar to real user timing, if there is a 4x times difference - most probably your JMeter configuration is not correct.
Probably the most important. Make sure your HTTP Request samplers are configured to retrieve so called "embedded resources" (images, scripts, styles) which are referenced in the web page
If your application is using AJAX technology make sure you execute AJAX-driven requests as well and add their elapsed time to main sampler using i.e. Transaction Controller.
Make sure you mimic browser's:
Cookies via HTTP Cookie Manager
Headers via HTTP Header Manager
Cache via HTTP Cache Manager
Assuming all above you should be receiving similar to real user experience page load time. See How to make JMeter behave more like a real browser article for more detailed information on the above tips.
In addition to the answers provided by James and user7294900, please find these images to help you understand the reason behind the difference in time given by your stop watch and JMeter.
Below image gives the ideology behind how JMeter provides the time.
Below image gives the ideology behind how you have measured the time with
your stop watch.
Notice that there are additional actions performed by the browser when you are taking the time using your stop watch. This is the reason behind the huge difference in time between JMeter and your stop watch.
In addition to this, ensure that you are using the same test environmental conditions for both the tests (like same network conditions, same LG etc.)
Hope this helps!
I have set up a test plan using Apache JMeter, but when testing a remote server I am seeing much slower times than it takes the browser to run the same test, it's on order of 5-10x slower. For example chrome takes about 300ms to load a simple static page with some embedded assets while jmeter reports 2000+ms for the same page from the same machine.
I've tried tweaking the thread count to make sure this is not the bottleneck as well as not too much load slowing down the server, but nothing seems to change these slow numbers.
What else can I look at to get more realistic response times out of this tool?
I figured this one out, the issue was that "Retrieve all embedded resources" was checked, which I want, but I was also not using the "concurrent pool size" option, so it was cycling through each of the ~10 embedded HTTP requests on the page serially and reporting slow overall load times. Checking this and adding a realistic browser concurrency number (Chrome uses 6 these days so that's what I went with: http://sgdev-blog.blogspot.com/2014/01/maximum-concurrent-connection-to-same.html)
give me numbers that are very close to real browser testing.
Make sure you add HTTP Cache Manager as browsers download embedded stuff like images, scripts, styles, etc. but do it only once, on subsequent requests aforementioned resources are being returned from browser's cache, no actual request is being made.
See How to make JMeter behave more like a real browser guide for other recommendations on how to make your JMeter test more realistic.
Practical Challenge:
I have a LR script that runs against an app being mocked and do not have a logout button (yet).
The test runs fine With stable response time for about 10 minutes, but after that the response time peaks and the server goes into 99% memory usage and transactions start to fail.
I suspect this is due to the script does not terminate the vusers after each run anf it builds up a lot of running sessions against the server wich is not terminated. But I might be wrong.
Anyays I want to programatically close each run after it has competed the business process.
I have red somewhere that web_set_sockets_option ("SHUTDOWN_MODE", "ABRUPT") could be used for this, but I want to be sure that this function actually does what I want and what does 'ABRUPT' means?
Are there better ways of closing sessions? Clicking the close browser during recording does not result in anything being captured in the script.
It's a server issue on session aging. Your server admin for your website can adjust the timeout values where no activity has taken place on a given session. By default most places have this set at 30 minutes. Trim it to what you need rather than taking the default value on the server.
Also, you may have hit a leak situation if resources are constantly accumulated on the server side but never released.
Based on your question I assume you're using the WEB/HTML protocol. I agree that the core issue is that your app's sessions should expire more elegantly and probably sooner. But, in order to get beyond this while testing you can try this. It isn't a guarantee, but it has worked sometimes for me in the past when dealing with similar situations. Try changing your Run-time Settings for the script:
Run-time Settings > Browser > Browser Emulation
Make sure you have the box checked for "Simulate a new user on each iteration". You can also try playing with the other settings here, like clearing the cache each iteration. This could cause a new connection setting with the web page for each iteration depending on the server's session settings. Again, this isn't 100%, but it has worked for me from time to time.
try this:
web_set_sockets_option("CLOSE_KEEPALIVE_CONNECTIONS", "1");
I'm trying to decide what the best solution would be for my web application. I have a page that will fire an arbitrary number of ajax requests to retrieve data from the server. For example a page on load may fire 10 ajax requests to the server and each request may take 10 seconds (+-) to return content.
In view of this being a web app in a multi user and multi concurrency environment is it a good idea to use a traditional ajax approach or would you opt for long polling, such as SignalR.
What are the pros/cons of both approaches (Pull vs Push)? Ultimately i'm after the most resource efficient approach.
Thanks
In your stated example you are talking about a pure 'Pull' scenario. ie 'When the page loads I want X, Y, Z to happen and then I want to see the results'.
Long polling/websockets (SignalR) is useful for a Push scenario - ie 'Oh look I have finished running this super long process... I better tell any users currently connected'.
You can use SignalR to run those normal style AJAX requests... but you wont get any performance enhancements. The AJAX will run asynchronously and in parallel and once the server side process is complete you will a callback that executes. Perhaps you might get a slight increase in performance as signalR will have a continuous connection running, so you will loose the slight delay in creating a connection. On the flip side the server will have a large number of open connections running which may degrade performance (especially if you are hitting it with 10 X 10 sec computations)