I am using JMeter to test load /performance of live streaming severs including WOWZA live streaming engine. But I am unable to test Live streaming of WOWZA cloud, since I am getting a lot of Time out errors. I am very well aware that the timeout is not because of the delay in response because the live streaming is running smoothly when opened from an external network. I found out that, after some period of load request being send to Wowza cloud, the domain name itself is getting changed(its dynamic). I have created the config in jmeter in such a way that all the URL path, Playlist.m3u8, chunklist.m3u8, and corresponding stream(ts) files are dynamic. But, since the domain name itself is getting changed after a period of load test, the request sending are partially getting failed(maybe because the domain name which I am sending request is not responsible to handle all the requests anymore). Can anybody suggest what to do? And is there any way to test load in WOWZA cloud?
You can use this JMeter plugin.
It is a plugin that will do URL extraction automatically from manifest without you needing to use JMeter extractors, as a consequence even when segment’s URLs will change due to Wowza Cloud scaling, this will be taken into account.
Besides, it will accurately similate how player request the server and give metrics on User Experience.
Still as written by the other answer ensure you:
ask for authorization to avoid your test being marked as DDOS
ensure you disable Java DNS caching for JMeter's JVM
Disclaimer: I work for the company that develops it.
As you are testing a multi-tenant cloud environment the first thing you must do is get permission from Wowza. Almost all Cloud applications have restrictions on the use of automation outside of their published interfaces. Your point of contact inside of Wowza will work with you for your testing window, scale, approve your performance test plan, your pacing and think times to ensure that they are reasonable and will not impact their service to other tenants on the system.
They can also provide technical insight on how to construct your tests given some unique features/capabilities/engineering for the site. They may even be able to provide you with sample code.
As a general rule of thumb, you don't point and fire tactical nuclear software at sites you don't own, manage, control or have direct written permission from those that do have those rights.
Related
I am exploring to put rate limiting functionality on rest API which are developed using spring boot.
After going through many articles, I came to know that the best way to put rate limiting functionality is with application code, rather then putting it on web servers.
My question is how do you decide that which functionality should go where. Since, its monitoring your incoming calls and nothing to do with business logic, the ideal place should be a web server.
My question is how do you decide that which functionality should go
where. Since, its monitoring your incoming calls and nothing to do
with business logic, the ideal place should be a web server.
Technically the web server could do the job but in the facts, a web server doesn't have necessarily all needed information, it is not specialized for API consuming and it may also make the testability of this feature much harder.
Some practical reasons why the webserver side could be a bad choice :
the developers don't have necessarily the configuration of the HTTP web server in local.
you want to write unit and integration test to check that the rate limitations are applied as specified. Creating a configuration for automated testing is much simpler in the scope of your Java application than with a configuration file defined on a web server.
web servers reasons in terms of HTTP request-response, not in terms of service.
Rate limitations may be applied according to the IP but not only, the username, the user roles, the type of service may influence the limitations. Not sure that you could get all of these easily from an HTTP server.
For example roles are stored on the server side or in a database.
A better option is setting these mechanisms by adding specific and specialized classes or configuration files, which simplifies their reading, their maintenance and their testability.
As you mention Spring Boot in your tags, that and that should interest you.
I recommend spring-cloud-gateway's rate limiter
you could separate this functionality from your business logic by using Filters.
https://www.baeldung.com/spring-boot-add-filter
Hi since I was not aware of load testing I got a doubt at the time of learning.
hope if it is a not valid also pls let me help.
In jmeter we can simply record and do load test right. if that is the case if I load some unknown application with lots of load from my client side it might causes the server crashes right. then what should they do if server crashes of unknown person load test.
is there any specific things to do load test or simply we can do load test on any website .pls let me know this thing even my query is not a valid one also...thanks in advance
The majority of web applications are protected from DoS attacks therefore most likely you will not be able to "crash" the server, the traffic from your IP will simply get blocked and your IP will get banned.
Moreover, your action fall under Computer Misuse Act and you may be a subject to imprisonment up to 1 year and a fine up to 5000 pounds. The above law applies to the UK however I'm pretty much sure the equivalent exists in all the countries around the globe.
So don't load test the application without explicit permission of that application owner or you will run into a trouble.
Check out Websites Forbidden to Test Using BlazeMeter for explicit list of web sites you must not test by any means. There are some sites you can use for practicing like http://newtours.demoaut.com/ or http://blazedemo.com/ however I would recommend using something you can deploy locally as this is the safest way to practice load testing, moreover you will be able to see server-side impact of your test
I want to practice performance testing at home using some load testing tool like jmeter. Can anybody tell me some links of websites or applications on which I can practice performance testing by throwing load by load testing tool from home?
It is not a LEGAL way to conduct load/stress test on any live websites/web applications without the permission.
So, you can deploy your own sample application or download sample applications available online and deploy it in your local server like Apache HTTP server or Apache Tomcat etc.
From my knowledge, You can use WebTours sample application from HP LoadRunner as an application to put your load.
Download HP Load Runner community edition from here. free for 50 vusers for life time. You need to create an account in the website.
You can download the webtours application from the HP website (comes with Apache HTTP server, which acts as a Web server).
Setup WebTours as per the instructions here
Confirm the successful installation/configuration by accessing the application from the browser (similar to http://localhost:8080/WebTours). The app is about flight booking (though not in real time ;) ).
Now, you can use the WebTours application as an AUT (Application Under Test)
Either you can continue with Load Runner or download latest JMeter version (3.1 as of now)
JMeter tutorials:
Getting Started
Component Reference
Builiding a Test plan
For Load Runner, there is documentation available in the following link to start with:
http://lrhelp.saas.hpe.com/en/12.53/help/WebHelp/Content/WelcomeContent/c_Welcome.htm
There is community support available managed by HP.
Use the sample applications which ship with the tool
Take your choice of open source application, install them on servers you own, manage and control. Use these applications as targets.
You are welcome to use api.jmeter.ninja. I built it for that purpose. A more formal API declaration is on it''s way but you can start with
http://api.jmeter.ninja/example.html.
http://api.jmeter.ninja/objects.xml
http://api.jmeter.ninja/objects/${OBJECT}.XML
Where ${OBJECT} is taken from the objects.xml page.
Or swap xml for dot json for the same in Jason format.
Exercises/Tutorial is available at http://api.jmeter.ninja/jmeter.pdf
Currently I just use this service for my own training sessions. But I hope to make it more generally available in the near future. There a currently no automatic limits so please just use your common sense and don't run high at throughput for a sustained periods of time. Anything under a total 100k requests is no problem.
Only caveat is that service is provided on a best effort basis at this point. Any abusive/problematic users may be blocked without notice.
My company has over a hundred users of a specific CRM web application, which is provided as a service by another company to us.
The users of this application are very dissatisfied with its average response time, and I need to find a way to gather metrics during a certain period of time (let's say .. a week) to prove the service provider that they are really providing a bad service.
If the application were mine, I would get the metrics from New Relic or some other equivalent monitoring service, but since it is not, I'm looking for something that could do some sort of client side monitoring.
I already checked Page Speed from Google and YSlow from Yahoo, but both are only useful when you want to test the application during a few seconds. They are not meant for the long term monitoring I need.
Would anybody know a way to get this kind of monitoring from a client side perspective?
LoadRunner is no charge for 50 users, but what you really need is not a test tool but a synthetic user monitor which runs every n number of minutes and pulls the stats. You can build it yourself using LoadRunner 12, Jmeter, or any other http sampling technology. You could also use a service like Gomez for sampling or mpulse from SOASTA for tracking every page component across all users.
Keep in mind that your developer tools will time all of the components of the request to give you some page times. As will Dynatrace for the web client.
If you have access to the web server then consider configuring the web server logs to capture the w3c time-taken field, which will track every request. Depending upon the server the level of granularity can be to the millionth of a second on each and every request.
You could also look at a service like LiteSquare which can process those web logs and provide ammunition for changes to the server to improve performance on a no-gain, no-charge model.
One (expensive) solution would be using LoadRunner endurance test feature. Check here for a demonstration.
Another tool is Oracle OATS.
JMeter is a free tool, though I'm not sure if it's reliable enough to run for a whole week.
These are load generator tools, so if you are testing as a single client, you should carefully chose your load amount (e.g. one user).
Last but not least, you could create your own webservice client, and create a cron job to run it on your specified time of day and log the access time.
If what you want is to get data from their server, this is impossible ... without hacking into it. All you can do is monitor the website as a client, using some of the above tools, make a report and present that to them. But even so they could challenge your bandwidth, your test method etc.
I recommend that you negotiate with them to give you their logs and to prove that their system can support a certain amount of load. If you are a customer to them, you can file a complain or test additional offers.
Dynatrace was already mentioned in combination with Load Testing. As you said that you want to monitor your live system I want to bring Dynatrace up again. Most of the time it is used to do live system monitoring to understand what end users are actually doing. It is also available as a 30 day trial - so - no need to buy it - but - use it for your sanity check: http://bit.ly/dttrial
I have been tasked with looking for a performance testing solution for one of our Java applications running on a Weblogic server. The requirement is to record production requests (both GET and POST including POST data) and then run these requests in a performance test environment with a copy of the production database.
The reasons for using production requests instead of a test script are:
It is a large application with no existing test scripts so it would be a a large amount of work to write scripts to cover the entire application.
Some performance issues only appear when users do a number of actions in a particular order.
To test using actual user interaction with the system not an estimation at how the users may interact with the system. We all know that users will do things we have not thought of.
I want to be able to fix performance issues and rerun the requests against the fixed code before releasing to production.
I have looked at using JMeters Access Log Sampler with server access logs however the access logs do not contain POST data and the access log sampler only looks at the request URL so it cannot simulate users submitting form data.
I have also looked at using the JMeter HTTP Proxy Server however this can record the actions of only one user and requires the user to configure their browser to use the proxy. This same limitation exist with Tsung and The Grinder.
I have looked at using Wireshark and TCReplay but recording at the packet level is excessive and will not give any useful reports at a request level.
Is there a better way to analyze production performance considering I need to be able to test fixes before releasing to production?
That is going to be a hard ask. I work with Visual Studio Test Edition to load test my applications and we are only able to "estimate" the users activity on the site.
It is possible to look at the logs and gather information on the likelyhood of certain paths through your app. You can then look at the production database to look at the likely values entered in any post requests. From that you will have to make load tests that approach the useage patterns of your production site.
With any current tools I don't think it is possible to record and playback actual user interation.
It is possible to alter your web app so that is records and logs every request and post against session and datetime. This custom logging could be then used to generate load test requests against a test website. This would be some serious code change to your existing site and would likely have performance impacts.
That said, I have worked with web apps that do this level of logging and the ability to analyse the exact series of page posts/requests that caused an error is quite valuable to a developer.
So in summary: It is possible, but I have not heard of any off the shelf tools that do it.
Please check out this Whitepaper by Impetus Technologies on this page.. http://www.impetus.com/plabs/sandstorm.html
Honestly, I'm not sure the task you're being asked to do is even possible, let alone a good idea. Depending on how complex the application's backend is, and how perfect you can recreate the state (ie: all the way down to external SOA services or the time/clock), it may not be possible to make those GET and POST requests reproduce the same behavior.
That said, performance testing against production data is always great, but it usually requires application-specific knowledge that will stress said data. Simply repeating HTTP GETs and POSTs will almost certainly not yield useful results.
Good luck!
I would suggest the following to get the production requests and simulate the accurate workload:
1) Use coremetrics: CoreMetrics provides such solutions using which you can know the application usage patterns. This would help in coming up with an accurate workload model. This model can then be converted into test scripts and executed against a masked copy of production database. This will provide you accurate results about the application performance in realtime.
2) Another option would be creating a small utility using AOP (Aspect oriented apporach) so that it can trace all the requests and corresponding method traces. This would help in identifying the production usage pattern and in turn accurate simulation of workload. AOP frameworks such as AspectJ can be used. This would not require any changes in code. The instrumentation can be done on the fly. The other benefit would be that thi cna only be enabled for a specific time window and then it can be turned off.
Regards,
batterywalam