I have an application that needs to stay in sync with google drive. To that end, I'm using the Changes feed that is described on this page.
I know the idea is to poll the changes feed so that I don't have to request a list of files and do a comparison. Right now I have it set to query every 30 seconds, and initiate a sync operation when the latest change number is updated. But, to make the application feel a little more responsive, I would like to query API more frequently (but still initiate a sync only when necessary)
Given that, I was wondering if requests against the Changes feed count toward the API quota? I don't want to query more frequently if it's going to double my quota consumption rate.
It looks like requests to the changes API do count toward the quota. I found the Reports section in the cloud console. It gives a detailed breakdown of requests by user, location, method, and more. Looking through the methods, I found that drive.changes.list accounts for the majority of my usage.
It's unfortunate, but better than burning through the quota with multiple calls to get the status of every file.
Related
I am using google smtp to send a lot of mails, but if I send too much mails it will return quota error.
So I want to check my quota anytimes.
But I didn't find any about this.
Is there any way like API to do this?
There is no way to check your quota via an api. You can check google developer console for your project manually but this is an estimate at the very best.
My advice is to keep a running total of the requests you are sending and try to space them out so that you don't hit any of the quotas.
Once you hit a quota error makes sure you stop and take a break for a little while, possibly implementing exponential backoff, but this depends upon which quota you are hitting.
I requested quota increase on the YouTube API though the google developer console from 10k to 100k.
I received a successful email, that says my increase has been approved
However if i check the UI in Google developer console it still states my quota is 10,000 per day.
When will my quota increase take effect?
The first thing i would do would be check that you are looking at the correct project. As this quota will be only valid for the project you applied for.
Then you need to edit the quota and increase it yourself. You should have that option to do that now. If you don't have that option I would wait a couple of days, to see if it doesn't kick in.
You might want to start sending requests and see if the quota increase is actually there its not not appearing in the UI yet.
I'm attempting to consume the Paypal API transaction endpoint.
I want to grab ALL transactions for a given account. This number could potentially be in the 10's of millions of transactions. For each of these transactions, I need to store it in the database for processing by a queued job. I've been trying to figure out the best way to pull this many records with Laravel. Paypal has a max request items limit of 20 per page.
I initially started off with the idea of creating a job when a user gives me their API credentials that gets the first 20 items and processes them, then dispatches a job from the first job that contains the starting index to use. This would loop forever until it errored out. This doesn't seem to be working well though as it causes a gateway timeout on saving those API credentials and the request to the API eventually times out (before getting all transactions). I should also mention that the total number of transactions is unknown, so chaining doesn't seem to be the answer as there is no way to know how many jobs to dispatch...
Thoughts? Is getting API data best suited for a job?
Yes job is way to go . I’m not familiar with paypal api but it’s seems requests are rate limited paypal rate limiting.. you might want to delay your api requests a bit.. also you can make a class to monitor your api requests consumption by tracking the latest requests you made and in the job you can determine when to fire the next request and record it in the database...
My humble advise
please don’t pull all the data your database will get bloated quickly and you’ll need to scale each time you have a new account it’s not easy task.
You could dispatch the same job at the end of the first job which queries your current database to find the starting index of the transactions for that job.
So even if your job errors out, you could dispatch it again, then it will resume from where it was ended previously
May be you will need link your app with another data engine like AWS, anyway I think the best idea is creating an APi, pull only the most important data, indexed, and keep the all big data in another endpoint, where you can reach them if you need
We have a self-hosted content management system which offers clients the ability to sync data from their GA profile with the content they maintain within the application. So, we're dealing with an arbitrary number of clients who can have an arbitrary number of profiles. Each client is required to simply enter their profile ID and request a token.
The issue we are running into is our Google Developer project is hitting its 50k/day quota pretty quickly, but Google has been unresponsive with our requests to increase this quota. Does anyone have experience with requesting an increase? If so, about how long did it take?
Aside from a quota increase, does anyone have suggestions on how we can avoid this situation?
It can take several weeks to get the quota set up. That is why it is recommended that you keep a check on your quota and request when you hit 80% of max. Only way to avoid it is to tune your requests don't request data you already have again store it in your system. if its an existing customer you should only be requesting the last few days worth of data. Everything else is static and shouldn't change.
How long ago did you request it and how critical is your problem?
Use a StackOverflow Q&A thread as an example - when you vote up, vote down, or favorite a question, you can see the UI quickly respond to that action with changes in the # of up-votes on the side.
How can we achieve that effect? If send every of such action to back-end for processing and use the returned response to update UI, you will see a slow update and feel the glitches. But if put some of the logic on the front-end, you will also need to take care of the fraud/abuse etc before reflecting the action on UI, i.e - before changing the # of up-votes, don't you need to make sure that's a valid click by an valid user first?
You make sure that a valid user is using the app before a user clicks on anything. This is done through authentication, and it must include various protection mechanisms against malicious users.
When a user clicks, a call is made to a server. In a properly architected app this call is lightweight, and the server responds very quickly. I don't know why you believe that "you will see a slow update and feel the glitches". Adding an upvote to the database should take a few hundred milliseconds at most (including the roundtrip from the client), especially if the commit is asynchronous or a memcache is used.
If a database update results in a need to do some complex operations, typically these operations are not done right away. For example, a cron job may run periodically to compute new rankings, etc., precisely because you do not want every user to wait. Alternatively, a task is created and put in a task queue to be executed when resources are available - again to make sure that a user does not wait.
In some apps a UI is updated immediately after the call to the server is made, before any response from a server arrives. You can do it when the consequences of a failed call are negligible. For example, if an upvote fails to be saved in the database, it's not a disaster, especially if it happens once in a million tries. Again, in a properly architected app calls fail extremely rarely.
This is a decision that an app developer needs to make. I would not update a UI before a server response if such an update may lead a user to believe that some other action is now possible. For example, if a user uploads a new photo, I would not show icons to edit or share this photo until I know that the photo is safely saved.