After 40 calls to users.settings.filters.create I start to receive User-rate limit errors. All future filters.create calls then fail for the next (approx) 24hrs. Specific error message below.
HTTP 429
"User-rate limit exceeded. Retry after 2021-05-19T07:24:15.104Z
(Forwarding rules)] Location[ - ] Reason[rateLimitExceeded]
Domain[global]"
I have a 5-second delay between each call, so well under the published daily usage and per-user rate limits. I calculate the API allows 250 / 5 = 50 calls per second.
https://developers.google.com/gmail/api/reference/quota
We are using Google Workspace Legacy edition, the project OAuth consent is set for Internal use and the project is not verified (not a requirement for Internal).
Is there an obvious reason that 40 consecutive filters.create calls spread over 200 seconds would trigger a User-rate limit in these circumstances?
Limitations for unverified apps are across the board. The fact that your app is for internal use only doesn't matter you must still abide by the unverified app restrictions.
If you want to be able to send more then that you will need to apply for verification.
Because your app is for workspace account and internal use it should be easer for you to verify it.
Related
No matter what we try, all YouTube API requests we make are failing.
As we first thought this was a propagation issue, we waited out 10 minutes, then 30 minutes, 2 hours and now over 24 hours, to no avail.
We have found this thread, which covers a similar issue with an iOS app, but does not correspond to our use case.
Here is a run-down of what the problem is:
Activating the "Youtube Data API v3" API for our account shows as successful, and the API shows as enabled.
A POST to https://www.googleapis.com/upload/youtube/v3/videos (videos insert) consistently fails with the following error, despite the fact that we have waited hours for the API enabling to propagate:
Access Not Configured. YouTube Data API has not been used in project XXXXXXXXXXXX before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=928939889952 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
Although the error does not directly point to this, our "Queries per day" quota for the YouTube Data API is showing as "0". We are not able to understand why this is showing as zero, and unfortunately, all attempts to edit it to something higher have failed, and disabling and then re-enabling the API has not solved the problem. In a completely separate project/account, this shows as "10,000" when enabling the YouTube Data API, and indeed video insert API calls work under that project.
This is a significant roadblock for us, as it prevents us from deploying our application: any help would be appreciated.
No access configured
Actually means that you dont have permission to access the api. It basically means you have enabled the api but dont have the quota to use it. Its different then the you have run out of quota error message.
After a strange change sometime last year by default the quota for the Youtube api is now 0. You need to request a quota extension it can take anywhere between a week to several months to get permission to use it.
It took me three months. No i dont have any idea how they expect anyone to develop a new application without any quota and to know ahead of time that they need to apply for quota in order to start developing their application. Its quite frustrating.
Since today (november 20 2018) we get error responses from Google's RuntimeConfig API:
Our systems have detected unusual traffic from your computer network. This page checks to see if it's really you sending the requests, and not a robot...
(check this link for complete HTML error)
We retrieve variables from Google's RuntimeConfig using the API in our code. We do quite a few request, but not more than before:
A developer starts his server locally, which retrieves all the needed variables (+- 30 everytime you start).
Requesting RuntimeConfig variables via GCloud results in the same HTML error:
gcloud beta runtime-config configs variables get-value databaseHost --config-name database --project=your-test-environment
Other gcloud api requests work (projects describe, gsutil, etc).
How can I verify if I violated any terms? I can only find a usage limit in GCloud Console of 6000 calls per minute.
You can find the quotas for Runtime Configurator and how much of those you are using in the Cloud Console under IAM & Admin. In the Quotas section you can filter on Service = Cloud Runtime Configuration API and you should see all the quotas and how close to those you are for this API. There are 4 quotas that may affect you (docs here):
1200 Queries Per Minute (QPM) for delete, create, and update requests
600 QPM for watch requests
6000 QPM for get and list requests.
4MB of data per project, which consists of all data written to the Runtime Configurator service and accompanying metadata.
We had the exact same issue on November 20th when a large amount of our preemptibles were reallocated at the same time.
Our startup-scripts make use of the gcloud beta runtime-config...-commands and they all responded with 503.
These commands responded correctly again after a few hours.
We have had a support-ticket with Google and there was a problem with their internal quota mechanisms at the time which since is fixed so the issue is resolved.
We have an web application which was working fine till yesterday. But since yesterday afternoon , one of our projects in google api console , all the keys started giving OVER_QUERY_LIMIT error.
And we cross checked that the quotas for that project and api are still not full. Can anybody help me to understand what may have caused this.
And after a days use also the API keys are still giving the same error.
Just to give more information we are using Geocoding API and Distance Matrix API in our application.
If you exceed the usage limits you will get an OVER_QUERY_LIMIT status code as a response. This means that the web service will stop providing normal responses and switch to returning only status code OVER_QUERY_LIMIT until more usage is allowed again. This can happen:
Within a few seconds, if the error was received because your application sent too many requests per second.
Within the next 24 hours, if the error was received because your application sent too many requests per day. The daily quotas are reset at midnight, Pacific Time.
This screencast provides a step-by-step explanation of proper request throttling and error handling, which is applicable to all web services.
Upon receiving a response with status code OVER_QUERY_LIMIT, your application should determine which usage limit has been exceeded. This can be done by pausing for 2 seconds and resending the same request. If status code is still OVER_QUERY_LIMIT, your application is sending too many requests per day. Otherwise, your application is sending too many requests per second.
Note: It is also possible to get the OVER_QUERY_LIMIT error:
From the Google Maps Elevation API when more than 512 points per request are provided.
From the Google Maps Distance Matrix API when more than 625 elements per request are provided.
Applications should ensure these limits are not reached before sending requests.
Documentation usage limits
Note: This question is about something that I do not understand in the documentation here:
https://developers.google.com/gmail/api/v1/reference/quota#concurrent_requests
Concurrent Requests
The Gmail API enforces a per-user concurrent request limit (in
addition to the per-user rate limit). This limit is shared by all
Gmail API clients accessing a given user and ensures that no API
client is overloading a Gmail user mailbox or their backend server.
enforces a per-user concurrent request limit (in addition to the per-user rate limit).
I do not find what is the 'per-user concurrent request' anywhere in their documentation. Whereas the per-user rate limit is found at the top in the same page.
https://developers.google.com/gmail/api/v1/reference/quota#top_of_page
The Gmail API enforces a per-user concurrent request limit (in addition to the per-user rate limit). This limit is shared by all Gmail API clients accessing a given user and ensures that no API client is overloading a Gmail user mailbox or their backend server.
The confusion here is the difference between per-user concurrent request limit and per-user rate limit
Lets say I make an app that lets users read from their Gmail account. I am going to be limited by the number of request each user can make though MY app the limit is per-user rate limit
Now lets say the user installs your app which also allows them to access their Gmail account. You are also limited to how fast the user can access the api via the per-user rate limit.
However both of our APPs and the gmail and inbox apps are are all running with the same per-user concurrent request limit concurrent meaning across all of the apps the user is using.
the per-user concurrent request limit is probably there to ensure that a developer doesn't create a number of different projects and rip data using all of them.
To my knowledge per-user concurrent request limit is not documented its a stealth limit and i have never seen anyone who has been able to nail down exactly what the numbers are for the concurrent limits in Google APIs. With the exception of the Google Analytics API which is 10000 not including the google analytics website and the official mobile apps
Example:
per-user rate limit
User number one logs in and lists all of his emails he has 10 user gets no error.
User number two logs in, he has 1000 emails and your application tries
to select them all out in 1 second. You are going to get a rate limit error for this user. You are flooding google.
However apps by other developers will be able to access the user inbox via the API
per-user concurrent
user number one is only running your application. logs in and lists all of his emails he has 10 user gets no error.
User number two is running your application and 20 other applications by other developers. Tries lists all of his emails assuming he has done this on all the applications at the same time he may end up getting an error.
These errors are user based due to the name Per-user in the name of the limit.
Example 2:
Lets look at the Google analytics API because i know the hard numbers for this api.
A user using your app can max make 100 requests over 90 seconds. (User-app based)
An application can make max 50000 requests a day. (App Based)
An all Applications can max make 10000 requests a day against a view (concurrent app based)
The API quota for my project was successfully increased to 4M through the application process. This is confirmed in the details for the quota via the Google Developer console quota page for the enabled API.
Videos are, however, being rejected with this message after just the standard 50 uploads with this message:
Task 310: Upload failed. Exception: HttpError 400 "The user has exceeded the number of videos they may upload."
Here is an example of a successfully published video.
Thousands of videos are waiting in the queue from an event that finished days ago. Several people from Google have been trying to help but this morning I was asked to post on Stack Overflow.
badRequest (400) uploadLimitExceeded The user has exceeded the number of videos they may upload.
This isn't a project based quota which would have been upgraded. This is a user based quota which can not be upgraded. It means that the user in question has exceed their upload limit for the day. The YouTube API has a slightly different quota system than other Google APIs. The ones listed in developer console are just the general ones. You should check calculating quota usage
200,000 read operations that each retrieve two resource parts.
10,000 write operations and 90,000 additional read operations that each retrieve two resource parts.
400 video uploads, 1500 write operations, and 50,000 read operations that each retrieve two resource parts.
This quota should reset at midnight West Coast USA time. It sounds strange to me that you are seeing this after only 50 videos. I am not all that sure what denotes a user; it could be IP address or the authenticated user.
Videos: insert
Quota impact: A call to this method has a quota cost of 1600 units in addition to the costs of the specified resource parts.
So Google support got back with a note saying that we'll just have to wait for the quota refresh... I can't even see any difference from the 'increased upload' capacity and the limit does seem to be 50 (roughly at least). I've asked them how long the anti-spam upload freeze actually is but I'll carry on breathing whilst I wait for that puppy.
Special thanks for help from DalmTo and halfer though.
Message from Google Support:
I have enabled "Increased upload" capacity for your channel. Please
re-login to your account and check if all works fine.
We don't have any 50 videos per day limit anymore tell the truth, but
we have antispam algorithms which may temporary suspend your Upload
function.
This algorithm is confidential, so I can not discuss this feature in
details. If you can not upload videos you should just wait I am
afraid. But I hope "Increased upload" will help.