I've been looking at ways people test their apps in order decide where to do caching or apply some extra engineering effort, and so far httperf and a simple sesslog have been quite helpful.
What tools and tricks did you apply on your projects?
I use httperf for a high level view of performance.
Rails has a performance script built in, that uses the ruby-prof gem to analyse calls deep within the Rails stack. There is an awesome Railscast on Request Profiling using this technique.
NewRelic have some seriously cool analysis tools that give near real-time data.
They just made it a "Lite" version available for free.
I use jmeter for session-based testing - it allows very fine-grained control over pages you want to hit, parameters to inject, loops to go through, etc. It's great for simulating how many real users your site can handle, rather than just performance testing a set of static urls. You can distribute tests over multiple machines quite easily by loading up the jmeter-server on computers with publicly accessible IP's. I have found some limitations in the number of users/threads any one machine can throw at a server at once (it depends on the test), but jmeter has helped my team improve our apps capacity for users to 6x.
It doesn't have any fancy graphing -- I actually use my own in-house graphing with gruff that can do performance analysis on request time for certain pages and actions.
I'm evaluating a new opensource web page instrumentation and measurement suite called Jiffy. It's not particularly for ruby, it works for all kind of webapps
There's also a Jiffy Firebug Extension for rendering the metrics inside the browser.
I also suggest you look at Browser Mob for load testing.
A colleague of mine has also posted some interesting thoughts on this.
Related
I am new to load testing.
So please help in learning gatling and Apache Jmeter for stress testing.
Please help in installing both on Windows and Linux.
How to implement them in my application?
Which one is better for stress testing?
You are asking very generic questions in terms of Stress/Load testing. I think it would be best if you take a look at their documentation then formulate a more specific question.
Installation documentation is best served from the creators of the software.
Implementing these load/stress testing tools into your application isn't really a thing. If you are looking for unit testing (test to utilize in validating your functions/classes/etc work then look at your languages specific go-to libraries - ie. Java is junit/jboss, Nodejs is Karma/Protractor, Python is TestCase/Nose, etc). These tools (jmeter/gatling) are used for stressing your application outside of your build process so they should be treated as end-users (meaning you run the stress testing from remote machines if it is a web service).
Either are best for the right scenario. I think jmeter clusters easier (built-in, where gatling is more manual) but gatling is more programatic and can be manipulated more.
These are opinions and shouldn't be taken as fact or the best so your milage may vary
I strongly doubt that you need them both, if you want a piece of advice in regards which one to choose take a look at Open Source Load Testing Tools: Which One Should You Use? guide.
Once you have clear vision on what tool better suits the needs - you could start ramping up on the selected tool and ask questions in its community communication channels.
I have created a website that allows users to search a database. It is a Perl script that searches oracle using Perl DBI then writes in HTML and JavaScript.
I have found many websites that will quantitatively test the initial loading of the website. I can't help but think that the figures I have are false because the test is not actually performing a search and loading any data.
Are there any tools for testing the speed and performance of the interactive operations of a site beyond its initial load?
You can look to wiring up load testing with something like WebDriver and JMeter. Lots of folks use these or similar tools for just these sorts of scenarios. They're great tools, but require a pretty significant investment in time to get up and running.
You can also use Telerik's Test Studio which makes it easier to quickly get good performance and load tests up and running. Please note I said "easier" and not "easy." Load and performance testing of websites takes anywhere from a moderate amount of work to "OMG! This is nuts!"
Disclaimer: I'm the director of engineering for Test Studio, so I'm a bit biased about it. :)
For load testing you have to user load testing tool like Jmeter or Loadrunner.
Jmeter is an open source tool and Load runner is paid tool but both are user to find load of the website there is other tool also available in the marker which is used to find the load of a website and that tool is free for 1 month.
But you have to use tool to find load of a website
YSlow, dynaTrace, HTTPWatch, Fiddler .........
All these things are really good for measuring the performance of the website and get statistics for the same. YSlow is really cool, offers good guidelines also.
However, i am very confused with so many things around (Though it's good that people already invested time and have made nice guidelines to follow and i thank them for great work done).
Following are my questions:
How much accuracy these tools have in terms or numbers they show ?
Which one(tool) is BEST to use (one for all needs)? Or i am missing name of some tool which is out of box and better than above all?
I'm suprised that you haven't mentioned JMeter. It is free, quite easy to use, has lots of features, and great for load testing your website.
As for question one, I'm not sure I can answer that. I'm sure that in general, the numbers these tools show are pretty accurate, but there are some catches. Take JMeter for example:
JMeter itself uses a lot of memory and also some substantial CPU time if you do some heavy load testing. That means that if you run the tool on the same machine as your website, some resources are lost, e.g. not available for the website
Testing it on the same machine does not out-of-the-box take in account that the data has to be sent over the internet connection, so response times are lower then the reality.
But in all, I think you should never blindly trust the results these tools give you, but they can give you a good insight into possible bottlenecks or problems.
YSlow is good to measure performance for a single user. Try to keep it grade A and it will be OK. But it actually doesn't measure performance in case of multiple concurrent users. For that you can use under each Apache JMeter. It's a good webserver/webapplication stresstest tool. So I would say, just use both YSlow (for client performance) and JMeter (for server performance).
I haven't used DynaTrace before, so I'll skip that part. The mentioned HTTP request trackers doesn't really measure performance, they are more debuggers.
As far as I am concerned, i find YSlow to be really good (have tried fiddler too) and it does help me when i need it and i do believe that it provides the correct figures thereby making me use that in the time ahead too unless there is anything unanimously accepted (which is difficult because everyone has different choices and requirements.) or even better. Oh they are right, forgot the JMeter, something you should definitely give a mention.
There is also Speed Tracer extension for Chrome. It should be usable with any JavaScript heavy website.
http://code.google.com/webtoolkit/speedtracer/
http://gtmetrix.com is a good tool and it is free. that analyzes your page's speed performance using Page Speed and YSlow
Does anyone know of any tools out there that will let me run and debug a VXML application visually? There are a ton of VXML development tools, but they all require you to build your application within them.
I have an existing application that uses JSPs to generate VXML, and I'm looking for a way to navigate through and debug the rendered VXML in much the same way that Firebug allows one to do this with HTML. I have some proxy-like tools that let me inspect the rendered code as it is sent to the VXML browser, but there's a ton of JS, which makes traversing the code by hand rather difficult.
Has anyone worked with a product that allows for this?
Thanks!
IVR Avenger
There is JigSaw Test suite - has free trial license and reasonably priced.
There is IBM's debugger - part of WebSphere Voice Toolkit.
Many other products have debuggers - a very good summary is here
Disclaimer: I am the development manager for Voiyager (www.voiyager.com), a VoiceXML testing tool. It doesn't meet your criteria nor do I believe it is the type of tool you want, but I thought it was worth mentioning it.
As far as I know, there isn't such a test tool for VoiceXML. In fact there are very few VoiceXML tools on the market and hardly any of them test or analysis. The vendors that created development tools, have all been acquired by other companies. Some of them offered did offer various forms of debugging that were specific to their tool set or stayed at the Dialog (caller input) level. From your question, I'm assuming you need much lower level debugging capabilities.
I think the alternative paths are minimal and somewhat difficult. I believe your primary goal is to debug or rewrite an existing application, but you haven't provided any specific challenges beyond the JavaScript. Some thoughts or approaches that may help:
Isolate the JavaScript and place the code into a unit test harness. That will go a long way to understanding the logic of the application. Any encapsulation of the JavaScript you perform will probably go a long way towards better code maintainability.
Attempt to run the VoiceXML through a translation layer to HTML so you could use FireBug. The largest challenge would involve caller input (ie processing the SRGS grammars). You could probably cheat this by just having the form accept a JSON string the populates the field values. There are tools on the market to test grammars. Depending on the nature of your problems, you could take a simple and light approach and attempt this over just the trouble areas.
Plumb the application with a lot of logging. This can be done through the VoiceXML LOG element, or push the variable space back to the server. By adding intermediate forms, you may be able to provide a dump from each via the VoiceXML Data element.
See if your application will run in one of the open source VoiceXML browsers (not sure of the state of the open source browsers as we've built and bought for our various product lines). If you can get it mostly working, you can use the development debugger to provide some ability to step through the logic. However, it is probably one of the more difficult paths as you'll really need to understand the browser to know when and where to stick your breakpoints and to figure out how to expose the data you want.
Good luck on the challenge. If you find another approach, I would be interested in seeing it posted.
An alternative debug env is to use something like Asterisk with a voicexml browser plugin like the one from http://www.voiceglue.org/ or for a limited licence, i6net.
You can keep all the pieces separate(dynamic html and vxml application in php/jsp/j2ee/, tts processing, and optional asr processing as separate virtual machines with something like virtualbox. If the logic can be kept the same, then it is just a matter of changing the UI based on the channel.
A softphone is all you need to call a minimal asterisk machine, which has the voicexml browser with the url of the vxml in the call plan.
I just used Zend Framework as php is used in this environment, and changed view suffixes(phtml vs vxml) based on the user-agent string.
Flite for tts is fine for debugging, and when your app is ready you can either record phrases, and there was a page on the ubuntu forums with directions for how to increase flite quality with some additional sound files.
Do you have tried Eclipse VTP or InVision Studio?
Eclipse VTP
This is Eclipse plugin. But I feel that it is user-unfriendly a little (of Japanese viewpoint).
InVision Studio *Required create user account*
This is Convergys's IVR tool. It has to edit standard VXML mode. (Unfortunately, It's not exact matching.)
For just debugging vxml, I use Nuance Cafe's VoiceXML checker. It doesn't give you a visual tree or anything, but it's pretty good at spotting syntax errors and is free. I think they might also have more advanced debugging tools if you look into it, but I haven't had the need. (Note: I have no association with them)
http://cafe.bevocal.com/tools/vxmlchecker/vxmlchecker.jsp
I'm looking for the same problem that most of the links are down. I found a document where they propose an open source solution, which works as a plugin for Asterisk (https://www.researchgate.net/publication/228873959_Open_Source_VoiceXML_Interpreter_over_Asterisk_for_Use_in_IVR_Applications) and is available at https://sourceforge.net/projects/voxy/
I would like to know if there are current options to create a VXML structure graphically, like the next image.
I am developing a Cocoa application which communicates constantly with a web service to get up-to-date data. This considerably reduces the performance of the application. The calls are made to the web service asynchronously but the number of calls is huge.
In what ways can I improve the performance of the application? Is there a good document/write up available which gives the best practices to be followed when an a Cocoa application communicates with a web service?
Thanks
You should try out Shark that comes with the Mac OS X devtools - really great for digging into your callstack and should allow you to limit to network libraries and friends.
Yes! Apple actually has some very concise guides on performance that cover a lot of tricks and techniques, I'm sure you'll find something relevant to your own application. There may be some additional guides specific to 10.5 I haven't seen yet, but here are three I've found useful in the past.
Performance Overview
Cocoa Performance Guidelines
Cocoa Drawing Performance Guidelines
The most important thing to take away though, is that you need to use performance tools to see exactly where the bottleneck is occurring. Sometimes it may be in the place you least expect it.
I think if you use Shark your just find your app is blocking waiting for answers back from the server. Code split across machines is far harder to benchmark as the standard tools can only benchmark part of the picture.
Sounds like you need to look into bundling calls up into fewer transactions.... Your bottleneck is almost certainly the network. What about supporting sending multiple calls as an array of calls? and the same for answers? Maybe you could buffer calls locally and only send them a few times a second as a single transaction?
Tony