im trying to do an load test on my page that uses Bing seach API for user queries in a search box, i wanna know who many searching users can handle my web, to do that i configure a Jmeter test, but when i run it appears to fail at 90% of searches, ¿there is any limitation on this api on multiple users that search at same time?
Bing does have API limits, the free Tier is 5,000. You should be able to see your limits at
https://datamarket.azure.com/dataset/explore/5BA839F1-12CE-4CCE-BF57-A49D98D29A44
To validate this, I create a load test to hit my 5,000 per month limit to see re-create the rate limit condition.
The error output and copy of the JMX is available at Load Test Bing. The test launched on 5 servers, each running 100 users, looping 11 times.
This first test run shows that you can quickly hit the ‘per minute’ which you can see in your error responses
Code(503) Message(The number of requests per minute for the subscription has reached the maximum threshold that is allowed. Please try again after….
Related
We have an web application which was working fine till yesterday. But since yesterday afternoon , one of our projects in google api console , all the keys started giving OVER_QUERY_LIMIT error.
And we cross checked that the quotas for that project and api are still not full. Can anybody help me to understand what may have caused this.
And after a days use also the API keys are still giving the same error.
Just to give more information we are using Geocoding API and Distance Matrix API in our application.
If you exceed the usage limits you will get an OVER_QUERY_LIMIT status code as a response. This means that the web service will stop providing normal responses and switch to returning only status code OVER_QUERY_LIMIT until more usage is allowed again. This can happen:
Within a few seconds, if the error was received because your application sent too many requests per second.
Within the next 24 hours, if the error was received because your application sent too many requests per day. The daily quotas are reset at midnight, Pacific Time.
This screencast provides a step-by-step explanation of proper request throttling and error handling, which is applicable to all web services.
Upon receiving a response with status code OVER_QUERY_LIMIT, your application should determine which usage limit has been exceeded. This can be done by pausing for 2 seconds and resending the same request. If status code is still OVER_QUERY_LIMIT, your application is sending too many requests per day. Otherwise, your application is sending too many requests per second.
Note: It is also possible to get the OVER_QUERY_LIMIT error:
From the Google Maps Elevation API when more than 512 points per request are provided.
From the Google Maps Distance Matrix API when more than 625 elements per request are provided.
Applications should ensure these limits are not reached before sending requests.
Documentation usage limits
I need to do performance testing where my scenario is 100 different user logging per day
You don't need any performance testing for 100 users as 100 users per day = 12 users per hour = one user each 5 minutes, any server will handle this "load", even if it is running on a Raspberry Pi.
In general if at some point of time you would like to know the limits of your application, i.e. what is the maximum amount of users it can handle - you will need to go for so called Stress Testing when you start with 1 user and gradually increase the load until response time starts exceeding acceptable thresholds or errors start occurring or website crashes whatever comes the first.
When it comes to using JMeter for web applications performance testing normally steps look like:
Record your test scenario(s) using JMeter's Proxy Server
Perform correlation if required (most probably you will need it as modern web applications widely use dynamic parameters for i.e. user state management or CSRF protection)
Given your test assumes > 1 virtual user you should be using different credentials for every user. Most often JMeter users go for CSV Data Set Config to read the data from external file.
References:
Building a Web Test Plan
Building an Advanced Web Test Plan
There is also an example test script available via File -> Templates -> Building an Advanced Web Test Plan entry of the JMeter main menu which you can use as a "skeleton" for your test
I have requirement of test api with 10k users. What I choose is :
Jmeter
Linux server to install Jmeter and throw load with 10k users
One node for now
Will perform API with operations like :
Login
Book hotel with post parameters
Update Booking Details
Save Booking
I am thinking to use above for API testing with 10k users, Above tools are enough or I should look for other options like loadimpact, loader or blazemeter?
If you are talking about API you should be rather considering "requests per second" rather than "users" as I strongly doubt that end users will be sending requests to API endpoints via curl or Postman.
No matter whether your goal is "users" or "requests per second" it is only you who can answer as it depends on many parameters like:
your machine hardware specifications
software specifications (OS/JVM/JMeter version and architecture)
nature of your test (request/response size, number of pre/post processors, assertions, etc)
So you should act as follows:
Make sure you're following JMeter Best Practices
Make sure you monitor baseline health metrics of the node which is running JMeter (CPU, RAM, Swap, Network, Disk), you can use JMeter PerfMon Plugin for that .
Start with minimal load (1 virtual user or 1 request per second) and increase the load until your machine starts swapping or any other health metric exceeds, say, 80% of maximum available capacity. Once it happens take a look into active threads (Active Threads Over Time listener) or throughput (Transactions Per Second) - this is the maximum number of users or hits per second you can produce on particular this host for particular this test. If it is enough - you're good to go, if not - you will have to switch to Distributed testing
See What’s the Max Number of Users You Can Test on JMeter? article for more details.
The whole answer is an elaborate it depends.
The best thing you can do is to run PerfMon agents on the server generating the load as well as on the server running the system under test.
This way you should see (in the CPU utilization and free memory statistics) whether you had maxed what the server providing the API can do or whether it is your load generator running out of steam. In the first case you got some readline based on the hardware and configuration you had run with. In the second one you have an indication to employ more than 1 box to generate the load or to investigate settings and options.
Have a closer look at PerfMon JMeter plugin for exact details.
We have a PHP application to do load test. The application team wants to know that, up to how many users that the application can be capable to withstand without any crashes.
How to do the load test on the same.. Please help us.
Thanks in advance
Its a boardroom question, application system capacity depends upon the application design and server where application is hosted. Ultimately this depends the purpose of application (public or private) and customer requirements (number of users).
You can find notes on test strategy for loading application in MSDN website Real-World Load Testing Tips to Avoid Bottlenecks When Your Web App Goes Live. My suggestion is application should able to manage atleast 10% maximum expected user simultaneously (till its popular... !!!!).
Try record and run Jmeter.
Use Summary Report, Summary Error Report and View Results Tree to see your server's health.
Keep adding thread counts till you realized all your thread group assertion starts failing.
My application includes complex data retrieval from RDC using XML. If error are less than 2% total test, i consider it as healthy. My app can handle 50 threads consecutively easily. Try yours. good luck.
You can identify the number of actual users experimentally.
At the first, make virtual users behavior like real user. Follow this link to get it.
Then increase number of users and observe application behavior. To scedule increasing number of users, you can use Throughput Shaping Timer
I have an application that needs to stay in sync with google drive. To that end, I'm using the Changes feed that is described on this page.
I know the idea is to poll the changes feed so that I don't have to request a list of files and do a comparison. Right now I have it set to query every 30 seconds, and initiate a sync operation when the latest change number is updated. But, to make the application feel a little more responsive, I would like to query API more frequently (but still initiate a sync only when necessary)
Given that, I was wondering if requests against the Changes feed count toward the API quota? I don't want to query more frequently if it's going to double my quota consumption rate.
It looks like requests to the changes API do count toward the quota. I found the Reports section in the cloud console. It gives a detailed breakdown of requests by user, location, method, and more. Looking through the methods, I found that drive.changes.list accounts for the majority of my usage.
It's unfortunate, but better than burning through the quota with multiple calls to get the status of every file.