I have a script at parse.com which imports a fresh information about recent videos from ~10 channels with YouTube v3 API. It worked well for some time, but about a month ago it started to fail once or twice per day with an error:
Error: can't fetch https://www.googleapis.com/youtube/v3/... "error": { "errors": [ { "domain": "usageLimits", "reason": "servingLimitExceeded", "message": "Serving Limit Exceeded" } ], "code": 403, "message": "Serving Limit Exceeded" }
As you can see from the following screenshot I use maximum 37k of 50000k units available per day. I estimate one script run could take maximum 2k units. To make absolutely sure I'm not hitting 3k units/second/user limit I've added 1 second delay before every call to YouTube API, but I still see those errors occasionally.
What's going on? Does YouTube v3 API have some other limits, for example units per IP? (one IP at Parse could host thousands of installations)
I think this is a issue on YouTube's end and doesn't seem related to the actual quota used.
I have been getting quite a few of these errors sporadically, and seemingly for no reason, over the last few weeks - see YouTube Data API - “Serving Limit Exceeded”
I've just filed a bug report with YouTube. Please "star" that issue to draw attention to it and track updates, if you'd like.
Version 3 of the YouTube Data API has concrete quota numbers listed in the Google API Console where you register for your API Key. You can use 30,000 units/second/user and 50,000,000 per day.
You can read about what a unit is here.
A simple read operation that only retrieves the ID of each returned resource has a cost of approximately 1 unit.
A write operation has a cost of approximately 50 units.
A video upload has a cost of approximately 1600 units.
If you hit the limits, Google will stop returning results until your quota is reset. You can apply for more than 50M requests per day, but you will have to pay for those extra requests.
Also, you can read about why Google has deferred support to StackOverflow on their YouTube blog here.
There are a number of active members on the YouTube Developer Relations team here including Jeff Posnick, Jarek Wilkiewicz, and Ibrahim Ulukaya who all have knowledge of YouTube internals...
Related
At least I think thats what this error message means:
HttpError: <HttpError 429 when requesting https://youtube.googleapis.com/youtube/v3/playlists?part=snippet%2Cstatus&alt=json returned "Resource has been exhausted (e.g. check quota).". Details: "Resource has been exhausted (e.g. check quota).">
But my YouTube Data API v3 quotas have not been fully used up:
I have used 31k from my 112k limit and as I understand it the Queries per minute quota is higher than my daily limit, so that should also be no problem.
Am I understanding it right that this error happened because of Quota Limits or is this from another source?
My request basically was creating a new playlist.
I am getting
"code": 403,
"message": "User Rate Limit Exceeded"
while using Google Drive API in my web app
Although the quota is 10,000 requests per 100 seconds and my average is less than 2:
How can I resolve this error? How to implement exponential backoff as the documents say?
There are sevrail types of quotas with Google apis.
Project based quotas which effect your project itself. These quotas can be extended. If for example you your project can make 10000 requests pre 100 seconds. you could request that this be extended.
Then there is the user based quotas. these quotas limit how much each user can send.
User Rate Limit Exceeded
Means that you are hitting a user rate quota. User rate quotas are flood protection they ensure that a single user of your application can not make to many requests at once.
These quotas can not be extended.
if you are getting a user rate limiting quota then you need to slow down your application and implement exponential backoff.
How you implement exponential backoff is up to you and the language you are using but it basically involves just retrying the same request again only adding wait times each time it fails
the graph
the graph in the google cloud console is an guestimate and it is not by any means accurate. If you are getting the error message you should go by that and not by what the graph says.
After hours of searching and thinking, I found out that,
'User Rate Limit Exceeded' has a spam protection which allow max 10 requests per second.
Thus I found out a lazy trick to do so by delaying the calls using:
usleep(rand(1000000,2000000);
It simply delays the call by a random duration between 1 and two seconds.
Can someone please explain me in simple language how these quotas work?
I know where is a similar question, but I want an explanation related to the screenshot below.
First, I opened the quotas page in Google Dev Console for YouTube API.
But I don't understand what these lines are and how they work, why there are several lines?
For example, I was trying to make a simple request like this
https://www.googleapis.com/youtube/v3/search?part=snippet&q=welcome&type=playlist&key=[MY_API-KEY]
Which returns me a json response:
{
"error": {
"code": 403,
"message": "The request cannot be completed because you have exceeded your \u003ca href=\"/youtube/v3/getting-started#quota\"\u003equota\u003c/a\u003e.",
"errors": [
{
"message": "The request cannot be completed because you have exceeded your \u003ca href=\"/youtube/v3/getting-started#quota\"\u003equota\u003c/a\u003e.",
"domain": "youtube.quota",
"reason": "quotaExceeded"
}
]
}
}
So, I assume it gives me an error because somewhere there is a quota = zero, because I only tried to make this request once.
What should I do to get rid of this error and be able to use the API?
Thank you.
project based quotas
The YouTube data api is a cost based quota as opposed to a request based quota.
With request based quotas you are given a quota of say 10000 requests that you can make, each request you make removes one from your quota.
The YouTube data api is a cost based quota. This means you are given a quota of say 10000 points which you can spend on requests. Each requests has a different cost.
uploading videos costs around 1600 points against your request so you can upload a limited number of videos yet list only costs 50 so you could do more lists then uploads before running out of quota.
I recommend having a look at the quota calculator which will help you understand the cost of each request against your quota allotment.
This video may also help you understand cost based quotas YouTube API and cost based quota demystified
As far as the error you are getting from the following request
https://www.googleapis.com/youtube/v3/search
As search.list method costs 100 quota points each time you request it, this and the error message would suggest that you have exceeded your quota. You need to either apply for an extension or make fewer reqeusts.
How to check your current quota allotment:
Go to https://console.cloud.google.com/ -> library -> search for youtube data api -> quota
user based quotas.
Besides that there are also user based quotas which are the number of requests a user can make per second these are flood protection quotas.
We used Live stream API to create live broadcast events, but it hit userRequestsExceedRateLimit when create live broadcast (POST /liveBroadcasts/insert). And Docs didn't specify any exactly rate limit number. Can help us to find max rate limit ，5 requests/sec or 8 requests/sec?
Here is error message (request rate 10/sec)
{
code:403,
errors:[{
domain:"youtube.liveBroadcast",
reason:"userRequestsExceedRateLimit",
message:"User requests exceed the rate limit."
}]
}
The official docs on rateLimitExceeded say:
The request was sent too quickly after the previous request. This error occurs when API requests to retrieve messages are being sent more frequently than YouTube's refresh rates, which unnecessarily wastes bandwidth and quota.
Every request to the YouTube API has a cost and rate limit, all of which add towards your quota. You can use a tool such as the YouTube Bulk Reports API to track your requests to see which ones in particular are causing you to go over your quota. All Live Streaming API calls (write operations) cost about 50 units. You can check the quota available to your application in the Developers Console.
In the Developers Console under YouTube Data API v3, make sure your "Per-user limit" under "Quotas" is set to the maximum value of 3,000 requests/second/user. If you're going over that, you will need to contact Google to increase your quota.
I have developed a google places application to get info about places. I have verified my identity with google and as per the limits, I should be allowed up to 100 000 requests per day. However, after under 300 requests (different numbers go through each day), I get the message back: OVER_QUERY_LIMIT. Any similar experiences or ideas how to enable the requests to go through?
Thank you.
D Lax
You can track your requests at
https://code.google.com/apis/console/?noredirect#:stats
I ran into this issue as well. I was able to find 3 throttle limits for the places api, given by google.
10 api calls per every 1 second
100 api calls per every 100 seconds
50,000 api calls per 1 day
If I were to go over any of these limits, I would receive the OVER_QUERY_LIMIT error and it would return no results for that given address.
I found a way to have my program sleep for 11 seconds after calling the places api with a dataset of 10 addresses. Then the program would call the places api with a new dataset of 10 address. This solution gets around the 10calls/second and the 100calls/100seconds throttle limits. However, I did run into the OVER_QUERY_LIMIT error once I tried my 25th dataset of 10 address (after 240 api calls). So it is clear that there are other throttles not published to help protect the google maps platform.
But, I did see that the limits mentioned above may be changed if you get in contact with the google api help team and sort it out with them.