I am going to develop an Android application that enables tracking (and monitoring on map interface) of multiple users by a specific user. For this reason, I want to study on a mBaaS, Parse. However I cannot figure out how much requests per second performed by such an app considering the count of users. To exemplify, if I choose free option for the monthly cost, the limit will be 30 requests per second. I have some doubts about whether this number is sufficient for this app.
In other words, there will be periodic API requests (let's say every 30 seconds) for all users that are tracking. I think it is highly possible to exceed the limit of 30 requests per second with a very few active users. Even if 5 different users track 10 different users at the same time, the probability of catching 30 requests per second is very high.
Considering all these, what kind of strategy you advise? How can I manage periodic geolocation requests in this system? Is Parse the right choice? If not, any better alternative?
The approach used in Traccar GPS tracking system is to return all user's objects in one request. So, say if you want one user to track 100 other users, you still need only one request to get all 100 locations.
You can optimize it further by not sending location if it hasn't changed. So, if only 10 users from 100 changed their location since last request, you can return only 10 location items in response.
Related
I'm building a pretty large app for a client that is going to aggregate feeds from various sources. My client estimates around 900 follow-able users will be in this system to start out, with more being added over time. He wants to update the feed data every 15 minutes, so we would need to update one user feed per second, assuming 900 feeds and a 15 minute TTL. As the requests take a few seconds to complete, we would then need to load balance across a few threads to tackle the queue asynchronously.
Should I be worried about quota errors or hitting any kind of limitations? If so, what are our options?
I've already read their help pages and documentation, but it's very vague; I need concrete numbers. It's not feasible to load test their API to figure out the limitation.
Version 3 of the YouTube Data API has concrete quota numbers listed in the Google API Console where you register for your API Key. You can use 10,000 units per day. Projects that had enabled the YouTube Data API before April 20, 2016, have a default quota of 50,000,000 per day.
You can read about what a unit is here:
https://developers.google.com/youtube/v3/getting-started#quota
A simple read operation that only retrieves the ID of each returned resource has a cost of approximately 1 unit.
A write operation has a cost of approximately 50 units.
A video upload has a cost of approximately 1600 units.
If you hit the limits, Google will stop returning results until your quota is reset. You can apply for more than 1,000,000 requests per day, but you will have to pay for those extra requests.
There is a calculator provided by YouTube to check your usage. It is a good tool to estimate your usage.
https://developers.google.com/youtube/v3/determine_quota_cost
If you need to make more requests than allotted, you can request a higher quota here: https://support.google.com/youtube/contact/yt_api_form
Parse now allows us to send 30 requests/second, but it is not straightforward to me.
Quoting some info gathered:
Here they say
At 30 requests/sec, an app can send us 77.76 million API requests in
a month before needing to pay a dime.
So I suppose he meant
send up to 77.76 million
Here, they suggest the rate of requests/second is calculated in a small window, generally a minute. This was answered about 2 years ago.
On their pricing faq page they give an example:
if an app is set to 30 requests/second, your app will hit its request
limit once it makes more than 1,800 requests over a 60 second period.
Suggesting that the window is one minute, even though they didn't clearly state it.
What intrigues me is that they say:
Pricing is pro-rated by the hour.
What does it mean? (sorry if it's obvious, English is not my first language)
Has anyone actually used parse and kept track of those request/second and burst/limits?
The only resource I found was a guy saying he had a web application with 10,000 users/day staying in the website around 4min, and he had under 12r/s.
Moreover, if my app logs users' activities, would that be a good practice to cache this info then send it in random times like between from 3am-7am?
Any help is very appreciated. My company is deciding whether go forward with Parse or not, so it's very important.
They could have worded it better but it basically means the same as "We'll charge you for a minimum of 1 hour based on the request limit you have set".
Here's an example. Assume you are using a 40 rps settings ($100/month which is $100/720 hours). If you keep this for 1 minute, you'll be charged for 1 hour, roughly $1.40.
You can change the request limit as often as you want. So if your app/site receives peak traffic for only 12 hours/day, you can increase the limit just for those 12 hours and end up paying just for those 12 hours.
Check the third question (How frequently can I increase/decrease my request limit?) on the FAQ page at https://parse.com/plans/faq
How frequently can I increase/decrease my request limit?
You can increase/decrease your request/limit as frequently as you would like
within a given month. We will prorate your charges on an hourly basis.
Not really clear what the pro rating means as I understand the setting to be an explicit limit that you pay for. If your limit is exceeded then the requests fail. I don't think there's an option to allow for payment on demand when the limit is exceeded but pro rating would do that.
The one minute is accurate and that is the current limit management.
The point of the pricing model is that your service should be making money before you reach any of the limits. If you have enough users to hit the limits and you aren't making money then you need to reconsider your business plan. As such you shouldn't need to upload at random times of day as your users should naturally spread out a bit.
Here is something that can help you understand Parse's users request per second.
Parse estimates that the average app's active user will issue 10 requests. Thus, if you had a million users on a particular day, and their traffic was evenly spread throughout the day, you could estimate your app would need about 10,000,000 total API requests, or about 120 requests per second. Every app is different, therefore Parse encourages you to measure how many requests your users send.
You can read more this question answered directly from Parse staff here on Parse/F&A link
Hope this helps
We are building a new application in parse and are trying to estimate our requests/second and optimize the application to limit it and keep it below the 30/second. Our app, still in development, makes various calls to parse. Some only use 1 requests, and a few as many as 5 requests. We have tested and verified this in the analytics/events/api requests tab.
However, when I go to the analytics/performance/total requests section, the requests/second rarely go above .2 and are often much lower. I assume this is because this is an average over a minute or more. So I have two questions:
1) does anyone know what the # represents on this total requests/second screen. Is it an average over a certain time period. If so, how much?
2) when parse denies the request due to rate limit, does it deny based on the actual per second, or is it based on an average over a certain time period?
Thanks!
I supposed you have your answer by now but just in case:
You're allowed 30reqs/sec on a free plan, but Parse actually counts it on a per minute basis, or 1800 requests per minute.
I'm building a pretty large app for a client that is going to aggregate feeds from various sources. My client estimates around 900 follow-able users will be in this system to start out, with more being added over time. He wants to update the feed data every 15 minutes, so we would need to update one user feed per second, assuming 900 feeds and a 15 minute TTL. As the requests take a few seconds to complete, we would then need to load balance across a few threads to tackle the queue asynchronously.
Should I be worried about quota errors or hitting any kind of limitations? If so, what are our options?
I've already read their help pages and documentation, but it's very vague; I need concrete numbers. It's not feasible to load test their API to figure out the limitation.
Version 3 of the YouTube Data API has concrete quota numbers listed in the Google API Console where you register for your API Key. You can use 10,000 units per day. Projects that had enabled the YouTube Data API before April 20, 2016, have a default quota of 50,000,000 per day.
You can read about what a unit is here:
https://developers.google.com/youtube/v3/getting-started#quota
A simple read operation that only retrieves the ID of each returned resource has a cost of approximately 1 unit.
A write operation has a cost of approximately 50 units.
A video upload has a cost of approximately 1600 units.
If you hit the limits, Google will stop returning results until your quota is reset. You can apply for more than 1,000,000 requests per day, but you will have to pay for those extra requests.
There is a calculator provided by YouTube to check your usage. It is a good tool to estimate your usage.
https://developers.google.com/youtube/v3/determine_quota_cost
If you need to make more requests than allotted, you can request a higher quota here: https://support.google.com/youtube/contact/yt_api_form
Can someone please explain the correlation between requests per second and response time? Which are you trying to improve at first? If your competitor offers less 'requests per second' on his most used functionality then you, is your application performing better in terms of end-user performance?
Can someone please explain the correlation between requests per second and response time?
Think of this situation as if it were a gas station. Cars arrive at various intervals and occupy a pump; they spend some time filling up, and then they leave.
Each car that arrives and occupies a pump is a request.
The time it takes to fill up is your response time.
You can improve things in two ways:
If you add more pumps, you can service additional cars at once because there will be more capacity.
If you make all your pumps faster, you can service more cars over time with the same number of pumps, because each car will finish sooner.
Which are you trying to improve at first?
That depends. Do you want to serve people faster (improving their experience while making some others wait) and thus more people overall, or do you want to serve more people at once (at the possible expense of request time)? Ideally, get both metrics as good as possible.
It all depends on what sort of load your system will be under.
If you have millions of users then you need to handle more requests per second possibly at the expense of response time otherwise users may not be able to connect when they want to.
However, if you are only going to have 30 users then it's more important to them that your system responds quickly than it being able to handle a thousand requests a second.
Requests per second may be high while offering an awful user experience. You might have a lot of users buying thousands of concert tickets per second but the response time for each user is over 30 seconds.
For a high performing, enjoyable web site, you need to have a high number of requests per second and a maximum response time. As a user, I like 5 seconds or less.
If your competitor offers less 'requests per second' on his most used functionality then you, is your application performing better in terms of end-user performance?
I wouldn't agree with that. Look at Google. They make thousands of requests a second - hell, I think it's something like 100 million per day and 3 billion per month.
To answer your question, I think response time is more important than requests per second. Sure you can optimize/minimize the number of requests made, but if your product scales to handle unlimited requests (just by throwing more hardware at the problem) then I think that is more valuable.