Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
I have implemented a http proxy client/server. Currently I intended to test this proxy client/server performance. Can anybody help me what approaches exists to make these tests?
If you are looking for some tools the following will be helpful for you:
RoboHydra is a web server designed precisely to help you write and test software that uses HTTP as a communication protocol. There are many ways to use
RoboHydra, but the most common use cases are as follows: RoboHydra
allows you to combine a locally stored website front end with a back
end sat on a remote server, allowing you to test your own local
front end installation with a fully functional back end, without
having to install the back end on your local machine.
If you write a program designed to talk to a server using HTTP, you
can use RoboHydra to imitate that server and pass custom responses
to the program. This can help you reproduce different bugs and
situations that might be otherwise hard, if not impossible, to test.
https://dev.opera.com/articles/robohydra-testing-client-server-interactions/
Webserver Stress Tool simulates large numbers of users accessing a website via HTTP/HTTPS. The software can simulate up to 10.000 users that independently click their way through a set of URLs. Simple URL patterns are supported as well as complex URL patterns (via a Script file).
Webserver Stress Tool supports a number of different testing types. For example
✓ Performance Tests—this test queries single URLs of a webserver or web application to identify and discover elements that may be responsible for slower than expected performance. This test provides a unique opportunity to optimize server settings or application configurations by testing various implementations of single web pages/script to identify the fastest code or settings.
✓ Load Tests—this tests your entire website at the normal (expected) load. For load testing you simply enter the URLs, the number of users, and the time between clicks of your website traffic. This is a “real world” test.
✓ Stress Tests—these are simulations of “brute force” attacks that apply excessive load to your webserver. This type of “brute force” situation can be caused by a massive spike in user activity (i.e., a new advertising campaign). This is a great test to find the traffic threshold for your webserver.
✓ Ramp Tests—this test uses escalating numbers of users over a given time frame to determine the maximum number of users the webserver can accommodate before producing error messages.
✓ Various other tests—working with Webserver Stress Tool simply gives you more insight about your website, e.g. to determine that web pages can be requested simultaneously without problems like database deadlocks, semaphores, etc.
http://www.paessler.com/tools/webstress/features
To better understand what is client-server and web based testing and how to test these applications you may read this post http://www.softwaretestinghelp.com/what-is-client-server-and-web-based-testing-and-how-to-test-these-applications/
Related
I have Two Question related to Native App Performance Testing?
1)I have a Payment App, and it comes with bank security which is installed at the time of app installation. It sends an token number and rest of the data in encrypted format. Is it possible to handle such kind of request using Jmeter or any other performance testing tool, do i need to change some setting in app server or jmeter to get this done ?
2)Mobile App uses Device ID, so if i simulated load on cloud server it will use same Device ID which i used while creating script? is it possible to simulate different mobile ID to make it real-time?
any Help or references will be appreciated ..:)
(1) Yes. This is why performance testing tools are built around general purpose programming languages, to allow you (as the tester) to leverage your foundation skills in programming to leverage the appropriate algorithms and libraries to represent the same behavior as the client
(2) This is why performance testing tools allow for parameterization of the sending datastream to the server/application under test
I'm not an expert in JMeter. But work a lot with Loadrunner (LR) (Performance Testing Tool from HP). Though JMeter and LR are different tools, they work under same principle and objective and so objective of performance testing.
As James Pulley mentioned, the performance testing tool may have the capability. But the question is,
Have your tried recording your app with JMeter? Since your app is a native kind, please do the recording from simulator/emulator and check the feasibility. JMeter might not be the right candidate for mobile app load testing.
Alternatively there are lot of other tools available (both commercial and opensource) in market for your objective.
Best Regards
With the raise of several mobile network technologies, load testing a mobile application has become a different ball game in comparison with normal web app load testing. This is because of the differences in the response times that occur in different mobile networks such as 2G, 3G, 4G, etc. Additionally the client being a mobile device has plenty of physical constraints such as limited CPU, RAM, internal storage etc. All of these need to be considered while conducting performance testing of a mobile application if one wants to simulate a scenario close to a real time condition.
Coming to your 2 questions,
1) Yes it is possible but the amount of manual effort that needs to be invested to make the script execution ready might vary (since you are mentioning there is data in encrypted format - some are easy to understand and some are just crude and difficult to handle using JMeter). But there might not be any app server setting that would be required to change (unless of course you are unable to handle the encryption with JMeter in which case, the encryption might have to be disabled for QA phase)
2) As rightly said by James Pulley, these values can be parameterized. However, I fear that these values will be validated by the app server and hence the values need to be appropriately fed in the requests.
You can refer to this link for reference on how to do Mobile Performance Testing for Native application http://www.neotys.com/documents/doc/neoload/latest/en/html/#4234.htm#o4237
.The same could be extrapolated to JMeter to an extent.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I am looking to capacity test my websocket server but don't really know where to start?
I am able to write a AI that will send messages to test the usage, but how would I simulate/make 100, 500, 1000 connections etc...?
I had a similar problem a little while ago when I had to load test thousands of connections against a server using the socket.io library. I was not able to find any off-the-shelf-solutions to do this so in the end I ended up building my own test using Node.js and a few for loops.
The advantage of Node is you can pretty much copy and paste the client side javascript into your server code so it's pretty simple to simulate the client and then you only need to make multiple connections to generate load. It's a quick and easy way to run the required javascript to establish the socket connection (assuming this is how you connect to your socket).
The gotcha I hit was running more than 600 listeners tended to max out the CPU on my node box but a little bit of AWS magic solved that.
Another issue is reporting results. There's not really any concept of response time with a socket connection, at least not in the classic sense, so it's hard to know when things are going wrong - at least from the client side perspective. But from monitoring the server we were able to see when connections failed and when resources started to get scarce and this was enough for us to benchmark how many connections it could support.
Autobahn Testsuite was designed to meet that need but the performance section of the tool still says "Under Development".
You could use JMeter for this purpose and get the WebSocket sampler plug-in from here: http://github.com/maciejzaleski/JMeter
For that many connections 1000 you might need to get more than one agent machine to achieve your task. This doesn't necessarily have to be dedicated server as you could deploy agents on few workstations (developers/testers machines) and used them for your test purpose. You could limit the impact by scheduling test execution to run out-of-hours.
Jmeter plugin is having severe limitations with number of concurrent users. It was working well only till ~450 users. Then I tried with artillery library(https://artillery.io/docs/testing_websockets.html) but this library also has restrictions with loops with their web socket package.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
I need to simulate 3G network and http requests from different geographical locations
API: HTTP restful - JSON
Geo Location: Europe, US
Client: Mobile device
It sounds like you're asking two separate questions:
What are the performance characteristics of your API server? Most usefully expressed as: how many concurrent users can it serve before response times exceed your acceptable level?
what is the performance experience on your client devices?
I'd encourage you to split those two concerns up, and test them independently. A device operating over 3G would struggle to generate enough load to stress a well-configured web server, and it's usually not cost efficient to commission thousands of load testing nodes around the world. Also, once the traffic arrives at your web server, it shouldn't really care whether it came from the same city, country or continent, or whether it originated from a mobile device or a PC or a load test server.
So, I would use any load testing tool you like to test the performance of your web API. Apache JMeter is free, but has a bit of a learning curve; however, it's available from several cloud providers, which allows you to take your tests and run them from different continents. Google "Jmeter cloud" for more details.
If performance is a key concern, you probably want to have a continuous testing regime, where you subject your code to performance tests ever week or so, and optimize as you go - leaving optimization to the end of the project is usually rather risky...
The next question is "okay, so I know my API server can serve 1000 concurrent requests with an average response time < 1 second" (or whatever) - how does that translate to end user experience?
It only makes sense to attack this once you are really clear that your web server isn't the bottleneck - because most of the decisions you make to optimize performance beyond this point are pretty major.
Logically, if you know that your webserver is responding at a certain level, the end user performance is affected by network latency and throughput. Latency is usually measured through the crude statistic of ping times: how long does it take a network packet to travel between client and server? It's hard or impossible to improve on ping times without revisiting your entire hosting strategy; it depends on the speed of light, the connectivity between your hosting farm and the dozens of other network segments between your client and the server.
Throughput is typically measured in bytes per second. Often, this is something you can affect - for instance, by making your API less verbose, using compression, etc.
3G devices typically have relatively poor network latency characteristics, but pretty decent throughput under normal circumstances. However, there are many circumstances which affect both latency and throughput which are entirely unpredictable - busy locations are the classic example: a football stadium full of 3G devices means individual users often have poor connectivity.
Testing this is hard. I'd split it up into testing for 3G devices, and testing geographical variation. To test the performance characteristics on a 3G device, I'd simulate the network conditions using bandwidth throttlers in front of a dedicated test suite (probably based on JMeter).
The final part of the jigsaw could be expensive - there are specialist companies who can test web performance from around the world. They have nodes around the world, where they can execute test scripts; they often write the scripts for you, and provide a web interface for you to run and measure the tests. I've used Keynote in the past, and found them to be very good. This kind of testing is expensive, so I only use it right at the end of the project, once I have excluded all other performance aspects from consideration.
For JSON load testing,
json-simple
for REST in general you can use
loadUI - "tool for Load Testing numerous protocols, such as Web Services, REST, AMF, JMS, JDBC as well as Web Sites. LoadUI uses a highly graphic interface"
I have a simple web application to conduct online exams for the college students. All questions are multiple choice questions. Around 5000 users will be taking up the exam. My backend is mysql and using PHP as the front end. I want to know the hardware configuration for the servers that will be required to host this application and work seamlessly for the required no of users.
I am also looking out for cloud solutions. If I choose Amazone EC2 instances, can some body give me advice on what type of EC2 machine I should go into for this application?
It is impossible to tell the exact specs of the servers that will be required to run your setup, because there are too many variables. However, it is definitely a good question: when I was a student at university, it happened that a professor tried to do this, and didn't do testing: on the exam date, the system got overloaded and the exam had to be cancelled!
Start with testing what you already have. You can use something like the ab tool or JMeter. It will simulate the requested load for you automatically, so you can check how your actual server performs, and act accordingly.
Application design is also important. Like you can cache all the question at web layer to avoid database query. Make client heavy app such that server payload is minimum (json response) to reduce download time load on server.
Request multiple questions at once and Batch user responses to answer question together to decrease ajax calls.
Make use of nosql solution to avoid RDMS constraints overhead.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I'm looking at stress testing our website and having trouble picking the right tool.
It looks to me like two of the most popular are JMeter and The Grinder. Can anyone help with reasons in favor of either?
Thanks!
I have worked a bunch with The Grinder and there are three main reasons I like it.
It's free. I assume from your question, you are only looking at free solutions. There are some excellent commercial products as well, but most of the time, I cannot justify the cost
It is easy to start up processes on other machines. When really trying to crank up load on a cluster, I need to easily distribute the load out to remote machines. Grinder is great for that.
The scripts are all Jython. That allows me to easily customize my scripts to programatically tweak my scripts (e.g. randomize certain paths).
I haven't used JMeter in a long time, so I cannot say authoritatively how it compares on points 2 & 3.
As of Jmeter:
It's free.
it's easy to Start with lot of documentation on its Website and on internet
it has a proxy feature to easily create test plan from browser navigation
It is easy to start up processes on other machines. It remote testing, can be done from GUI or console.
The scripts can be written in beanshell, java, or any jsr223 language ( groovy, Javascript, scala, jexl ...)
it has a lot of built- in samplers and thanks to its plugin architecture it's very Easy to add new ones or use any scripting engine to do what's missing
it has great user mailing list
it has very reactive support
it's now a top Level Apache project
...
Use gatling.
http://gatling-tool.org/
Its lovely, great DSL, and you can just edit the .scala files and rerun and it will compile for you on demand. Also emits very nice graphs
http://gatling-tool.org/sample/index.html
If you're not familiar with Jython (like I wasn't) then there is also a great little tool that comes with Grinder (the TCP Proxy or some such name) that allows you to click around in a webapp and save your actions as a ready-made Jython script for Grinder, which you can then analyse/edit/adopt as necessary.
There is an excellent blog post that describes
Load Test Tools:
Grinder 3.11
Gatling 2.0.0.M3a
Tsung 1.51
JMeter 2.11
The Grinder
The Grinder consists of two main parts:
The Grinder Console - This is GUI application which controls various Grinder agents and monitors results in real time. The console can be used as a basic IDE for editing or developing test suites.
Grinder Agents - These are headless load generators; each can have a number of workers to create the load
Key Features of the Grinder:
TCP proxy - records network activity into the Grinder test script
Distributed testing - can scale with the increasing number of agent instances
Power of Python or Closure combined with any Java API for test script creation or modification
Flexible parameterization which includes creating test data on-the-fly and the capability to use external data sources like files, databases, etc.
Post processing and assertion - full access to test results for correlation and content verification
Support of multiple protocols
Apache JMeter
Key Features of the JMeter:
Cross-platform. JMeter can be run on any operating system with Java
Scalable. When you need to create a higher load than a single machine can create, JMeter can be executed in a distributed mode - meaning one master JMeter machine will control a number of remote hosts.
Multi-protocol support. The following protocols are all supported ‘out-of-the-box’: HTTP, SMTP, POP3, LDAP, JDBC, FTP, JMS, SOAP, TCP
Multiple implementations of pre and post processors around sampler. This provides advanced setup, teardown parametrization and correlation capabilities
Various assertions to define criteria
Multiple built-in and external listeners to visualize and analyze performance test results
Integration with major build and continuous integration systems - making JMeter performance tests part of the full software development life cycle
I just went through the process of trying at both and I would totally agree with Rob here. Grinder also seemed faster, and I really like how simple and lightweight it is compared to Jmeter. The grinder.properties file is totally easy to use, especially if you're more of a console guys then a UI guy.