I want to do API automation and load testing and security testing at the same time using one project. What kind of tool or technology can I use to implement that project?
For API automation and load testing almost any load testing tool will fit, if you're uncertain regarding which one to choose - take a look at i.e. Open Source Load Testing Tools: Which One Should You Use? article which highlights and compares the most advanced and popular free and open source load testing tools in the market.
With regards to security testing at least the majority of points from OWASP Web Application Penetration Checklist can be automated, moreover you can use a load testing tool to simulate a DoS attack
Related
Can anyone please compare all the features of MS-Visual Studio Load Test tool and JMeter tool?
Here I already got few comparions with both,
Price
JMeter: free and open source
MSVS: Test Professional costs around $2000 per developer.
OS supported
JMeter: any
MSVS: Windows only
Protocols supported
JMeter: HTTP, FTP, JDBC, SOAP, TCP, JMS, SMTP, POP3, IMSP
MSVS: HTTP
But I need a comparison rather than this, Also I need to prove JMeter is worthy and better than MS-VisualStudio Load Test.
The main reason of choosing JMeter is that JMeter is being actively developed and supported and Visual Studio 2019 is the last version to support Coded UI and Load Testing features, they will be removed from the next releases.
So if you are looking for a load testing tool which will live at least several years - go for JMeter.
You're being disingenuous on the protocols supported. MSVSLT supports any protocol supported in Visual Studio, which includes all standard protocols for client server, web, even raw sockets mode.
Both are developer tools built for use by Developers.
It is true that Microsoft is abandoning the performance testing market with their tools. This should be eye opening on how Microsoft views the importance of performance testing in their developer community.
Price is the absolutely last criteria you should be looking at for a delivery item on performance testing. Why? Because you can get a lot of cheap tools/free tools which may not exercise your interface, may not support your monitoring needs, will not support your analysis and reporting needs, and are flat out unusable on your reference platform and by your intended user population.
Start with your analysis and reporting. This is where the value is added. Running the load test is simply a controlled model for the production of timing records and resource measurements This will provide a subset of tools that meet your business mission goals.
Next, look at your environment, for a list of tools which will collect resource measurements while your test is running.
Once you have a subset of tools that meet your business mission needs and your technical environment needs, then take a very keen look at your intended user population. You may have the most effective tool on the planet, but if your team isn't capable of delivering high value, then you as well be deploying the least effective tool. Your return on effort will be about the same.
Lastly, look at license model. If you have no authority to authorize based upon price, then you should also lack the authority to reject based upon price. Pass or reject on technical and mission capabilities.
What are the tools available for UI independent and integrated testing with Services & UI Performance testing ?
Browser specific Tools
Performance Specific Tools
There are plenty of testing frameworks out there. You may want to look at these:
QUnit - JavaScript unit testing framework.
Selenium - Very good for cross-browser interaction testing
I am looking for a solution to simulate vxml application in a desktop/web browser.
Challenge
i have few hundreds of unit test cases in every new vxml application development.
it is not possible to always make a test call to test every single unit test cases i have. if i can test vxml application in a simulator it will be more easy to test it..
i am aware of that avaya Dialog designer/AAOD have a feature to test application in their inbuild Application simulator which comes with AAOD/DD Eclipse package but this tool can run AAOD/DD projects in workspace alone.
Please refer : Chapter 16 in Avaya dev guide
AAOD's Tool looks like this
is it possible to customize to use this tool to simulate static VXML / Nuance NDF based VXML application ?
or any other tools available ?
Unit testing of IVR applications is challenging and it is near impossible to hit ever possible path in the call flow for an application of any size. I am not aware if you can customize Avaya's tool for your needs. But there is a product out there that may fit some of your requirements. It is Voiyager by Syntellect. It looks like they have a free trial now if you want to try it out. I saw a demonstration of it at SpeechTEK a few years ago and it was pretty impressive. The only reason I did not pursue it at the time is that it was pretty pricey.
Avaya od/dd is a platform dependent tool(IDE) where you can create (vxml+java), test and debug application in the tool. But code which is created outside(Avaya OD/DD tool) the environment is not possible. Because code generated in avaya OD is java format, while executing in the voice browser it is converted into vxml for that u need to give webLM a license to avaya tool.You can use some ivr automation tools like hammer etc.But you need to pay for the software.
VXML apps are just WARs. The only session information transferred between invocations of pages is the session id. We have used Apache JMeter to create apps.
What you can do it try and see if what you want to do can be done via the web browser, by going to http://<server>/<Application/Start there fill the boxes you require (most of the time just filling the ones marked ANI and DNIS will be enough). If your call can be tested this way you can use the tools for web testing, without using Orchestration Designer.
If your app has a CTI/AES connector you may not be able to do this, because the web server tries to connect to AES upon submitting the first page.
As stated above , there are things that cannot be unit tested .
Nuance NDF and SVF applications come with test pages where you can simulate the call flow ; similar tools exit for grammars .
Can anyone help me in understanding the difference between Selenium RC and WebDriver and which one is better and why?
Selenium uses JavaScript to automate web pages. This lets it interact very tightly with web content, and was one of the first automation tools to support Ajax and other heavily dynamic pages. However, this also means Selenium runs inside the JavaScript sandbox. This means you need to run the Selenium-RC server to get around the same-origin policy, which can sometimes cause issues with browser setup.
WebDriver on the other hand uses native automation from each language. While this means it takes longer to support new browsers/languages, it does offer a much closer ‘feel’ to the browser. If you’re happy with WebDriver, stick with it, it’s the future. There are limitations and bugs right now, but if they’re not stopping you, go for it.
Selenium Benefits over WebDriver
Supports many browsers and many languages, WebDriver needs native implementations for each new languagte/browser combo.
Very mature and complete API
Currently (Sept 2010) supports JavaScript alerts and confirms better
Benefits of WebDriver Compared to Selenium
Native automation faster and a little less prone to error and browser configuration
Does not Requires Selenium-RC Server to be running
Access to headlessHTMLUnit can allow really fast tests
Great API
It's explained here.
Selenium-RC uses JavaScript to automate web pages. Therefore it is constrained by what you can do with JavaScript, specifically, it is constrained to the JavaScript sandbox. It also requires the Selenium-RC server. It supports many browsers and many languages.
WebDriver uses native automation and does not have the sandbox constraints of Selenium-RC. It's a little faster and does not require a server.
Has anyone had experience with developing web apps using ClearQuest? Looking at the features, it mentions process automation but I'm not sure how large of a process/application it can support. Our group gave up a small section of a larger application to another group that used ClearQuest to web-enabled the process but now the ClearQuest tools are being pushed on us to web-enable the entire application. I have a ASP.NET background but ClearQuest is very wizard like and the only code that I've seen is called "BASIC" and it looks a lot like VBScript.
Clearcase/Clearquest has decent integration with Rational Applicaiton Developer. I am not sure if you arsking about its strenghts or actually exposing ClearQuest to the web. You can do the obligatory and lame activity of writing CGI/PERL scripts to expose some of the features to the web. You can use perl or VBScript to write CC/CQ hooks. It has its own perl build included in the product called ratperl. It has fantastic command line support but it comes at a price of complexity. CC/CQ is very powerful provided you think about the implementation in your organization very carefully. It can support very powerful work flows and is highly customizable.
There are several APIs which allow you to do what you want, write a WebApp.
CQWeb itself is built using the CM API, which allows a Java application (web or non-web) to be build. A tutorial is here:
http://www.ibm.com/support/knowledgecenter/SSSH5A_9.0.0/com.ibm.rational.team_api_cq_tut.doc/topics/teamapi_cq_abstract.htm
You could implement a basic web application using perl-CGI, since CQ has a Perl API.
You can write an ASP/C# web application because ClearQuest has a OLE (Visual Basic) API. The original ClearQuest Web application was built using the Visual Basic API
So the answer is yes, it is possible.
Wouldn't it be nice it supported a JavaScript API? Then we could write nodeJS CQ applications.
Sure it's possible, for a VS style integration I would either wrap the cqole.dll or use the OSLC connectors to get access to the RESTful interface. While the cqole is the C++ API and it is not officially supported this will suite you better than trying to implement this with the VBScript API. If you are more comfortable with just using the web services I would check out jazz.net where the wiki explains the bridge in gory details.
You dont exactly develop webapp with ClearQuest, CQ is only a process automation tool that build on logic and state transitions, and the interactions are scripted in either VBScript or Perl.
However do note that it has always been a pain to manage CQ codes as the "Designer" itself is a completely broken tool. It supports version tracking but doesnt tell u the difference between versions. The built-in code editor doesnt have syntax highligting, and it doesnt support parallel development. I can go on and on.
In the later versions, (version 7 onwards I supposed), CQ comes with eclipse based client AND designer tools, which supposedly enhance the whole experience. But I dont have too much experience with it to comment.
As for the web component, it runs on the websphere layer and simply is a application/presentation layer for users to access CQ through a HTTP protocol. It has 90% of the features of a full CQ client, and is usually easier to maintain and deploy to a wide user group.
Check out http://open-services.net/ for information on development using Open Services for Lifecycle Collaboration.