As a contrived example, if the twitter datastream example from:
https://www.pubnub.com/developers/realtime-data-streams/twitter-stream/
was implemented in a website.
Is each new visitor using that website counted in the daily active devices quota? PubNub's free tier gives only 100 daily active devices and the next upgrade to 1000 is $150.
Using the free tier, will I only be able to have 100 unique visits on a simple twitter stream before my quota runs out?
Or once a visitor closes the tab, I can give the next new visitor the same UUID, effectively reducing the quota to mean 100 visitors simultaneously.
Maybe since every visitor subscribes to the same channel, I can give each visitor the same UUID, making the limiting quota messages sent. What will happen if multiple user's are initialized with the same UUID?
Perhaps this isn't really a good use case for PubNub and this example should probably be implemented using a custom solution.
Does the same apply for a website implementing a TIC-TAC-TOE game, only 100 unique visitors a day? or only 50 games (100 players) simultaneously?
PubNub Daily Active Devices and Pricing
The DADs (daily active devices) are counted as each unique IP address that connects to PubNub, so yes, you would likely go over the Free tier limits very quickly, but we do have custom pricing plans for drive-by web traffic that makes it more fairly priced.
UUIDs are just that – unique ID per client (subscriber) and is used for our Presence service but also helps for identifying unique devices/users.
For details on custom pricing, you can contact PubNub Support - which you already have :)
Related
I've an application with 10M users. The application has access to the user's Google Health data. I want to periodically read/refresh users' data using Google APIs.
The challenge that I'm facing is the memory-intensive task. Since Google does not provide any callback for new data, I'll be doing background sync (every 30 mins). All users would be picked and added to a queue, which would then be picked sequentially (depending upon the number of worker nodes).
Now for 10M users being refreshed every 30 mins, I need a lot of worker nodes.
Each user request takes around 1 sec including network calls.
In 30 mins, I can process = 1800 users
To process 10M users, I need 10M/1800 nodes = 5.5K nodes
Quite expensive. Both monetary and operationally.
Then thought of using lambdas. However, lambda requires a NAT with an internet gateway to access the public internet. Relatively, it very cheap.
Want to understand if there's any other possible solution wrt the scale?
Without knowing more about your architecture and the google APIs it is difficult to make a recommendation.
Firstly I would see if google offer a bulk export functionality, then batch up the user requests. So instead of making 1 request per user you can make say 1 request for 100k users. This would reduce the overhead associated with connecting and processing/parsing of the message metadata.
Secondly i'd look to see if i could reduce the processing time, for example an interpreted language like python is in a lot of cases much slower than a compiled language like C# or GO. Or maybe a library or algorithm can be replaced with something more optimal.
Without more details of your specific setup its hard to offer more specific advice.
I'm currently integrate Google+ API to my service.
I'd like to know what's the limitation for this kind of api:
https://www.googleapis.com/plus/v1/people/{user_id}/activities/public
in google develop console, I found this:
Quota summary
Free quota 10,000 requests/day
Remaining 9,998 requests/day
99.98% of total
Per-user limit
5 requests/second/user
I think there will be two kind of limitation:
Application level
For example, how many requests can an app send per day(sum of the number for all users), and what's the max qps?
User level
For example, how many requests can an app send per day for a special user, and what's the max qps?
But I can't find the exactly info, does anyone know?
Can't say particularly about this API, but when I used Google Places api the quota was linked with IP address.So if it expired, we need to use different IP for hit.No user / application quota.
you already have those answers in your question:
10,000 requests per day in total, using that developer console key, thus your total "app" calls.
per user there is no limit, there is a rate quota. a single user could at most make 5*(seconds in a day) requests per day.
in this specific api case this is much bigger than 10,000 thus that rate quota is not that useful (except it prevents users from quickly depeting the 10,000 daily quota).
you can edit that rate quota so its lower or higher, and is used so a single user cant consume all the app quota (maliciously or otherwise)
According to the parse.com pricing page, push notifications are free up to 1 million unique recipients.
API calls are free up to 30 requests / second.
I want to make sure there is no catch here.
An example will clarify: I have 100K subscribed users. I will send weekly push notifications to them. In a month, that will be 4 push "blasts" with 100K recipients each. Is this covered by the free tier? Would this count as 4 API calls, 400K API calls, or some other amount?
100k users is 1/10 the advertised unique recipient limit, so that should be okay.
Remember that there's a 10sec timeout, too. So the only way to blast 100k pushes within the free-tier resource limits is to create a scheduled job that spends about 2 hours (that's a safe rate of 15 req/sec) doing pushes and writing state so you can pick up later where you left off.
Assuming there's no hidden gotcha (you'll probably need to discover those empirically), I think the only gotcha in plain sight is the fact that the free tier allows only one (1) scheduled job. Any other long-running processing -- and there are bound to be some on 100k users -- are going to have to share the job, making the what-should-this-single-job-work-on-now logic pretty complex.
You should take a look at the FAQ for Parse.com:
https://www.parse.com/plans/faq
What is considered an API request?
Anytime you make a network call to
Parse on behalf of your app using one of the Parse SDKs or REST API,
it counts as an API request. This does include things like queries,
saves, logins, amongst other kinds of requests. It also includes
requests to send push notifications, although this is seen as a single
request regardless of how many recipients are targeted. Serving Parse
files counts as an API request, including static assets served from
Parse Hosting. Analytics requests do have a special exemption. You can
send us your analytics events any time without being limited by your
app's request limit.
I'm working on my enterprise SaaS application and some of my users would like to be charged on a per-seat approach.
I was wondering how to make sure that the access will indeed be limited. I can see right now that people with the same login/password are logging from different IP addresses, different user agents at the same time, even though the company have paid only for one seat.
What would be the best way to implement the limitation from business and technical perspectives? I don't want to be too strict, at the same time I want companies to pay for the number of seats they actually need.
Don't filter by user agent, I think that would be too strict, some people may have a variety of browsers installed. Filtering by IP could also be tricky, some users may have a dynamic IP that changes at regular intervals.
One idea I have would be to force users to install a browser extension, the browser extension could generate a unique ID from something on the operating system. Maybe use a HDD volume number or Windows serial key, anything that will be unique to that computer.
Once you have this unique ID, use it in the back end tracking every time the user logs in. If the user exceeds a certain number of seats you can either block the user account or contact them first.
It would also be a good idea to allow the user a certain amount of freedom, so that if they have one seat then the back end will allow maybe 2 seats for a month (rolling period), this would be in-case the user buys a new pc or installs a new HDD.
Using IP seems a bad idea - people in a LAN will (usually) all have the same IP.
Assuming PHP has no API to access inspect sessions different than the current one - one idea is:
if you're not already doing it, for each login cookie you issue, also issue an unique id (an UUID will serve your purpose)
store those ids somewhere (in java you could store them in the application context, maybe in php you'll need a db table for that - I'm no php expert), together with two timestamps: "session start" and "last activity"
at each request, record the current timestamp in "last activity"
Then, when you get a new request, count how many other active have
last activity >= current session start
last activity >= now - session TTL (only needed if can't prune expired sessions in a timely fashion)
That should give you the number of occupied seats.
Note you will not notice a violation of the number of seats until after the surplus user has logged in and precisely until you see activity in some other session – I don't see any way around this issue since you don't know exactly when a seat becomes vacant (well, you might say it's vacant only when its session has expired, but that seems unfair).
I'm coding a new website that will need users to enter their mobile phone number, the problem I'm facing is that I need to make sure that the user is in fact the owner of (or in this case, has access to) the mobile number.
The solution I've come up with is, upon number submission I send them a SMS with a token and ask the user to enter the token on my website, much like Google Calendar does. However I'm on a short budget and I need to make sure user A doesn't submit 100,000 mobile numbers, if that happens I'll be out of business in no time, since each SMS sent costs me about 0.10 USD.
So far, I've come up with the following solutions:
use a CAPTCHA (keeps some users away and it is still vulnerable to manual registrations)
limit the number of tokens a given IP address request (dynamic IPs, proxies, etc)
limit the number of tokens sent for a given mobile number (a user can request tokens for all the available numbers and when the real user tries to request a legitimate token, his number will be already blocked)
None of these solutions are perfect, how do you suggest I approach this problem?
In a recent project, we were associating SMS numbers with a user account. Each account needed a CAPTCHA and email activation. The user could activate SMS via token, like you are using.
You could rate limit IP addresses (not a total limit). No more than 10 requests from an IP within 5 minutes, or something like that.
And/or you could limit outstanding SMS requests. After an IP address requests a token for SMS, it must be submitted before that IP can request for another SMS number. Or no more than 10 outstanding SMS tokens per IP per day.
Also, like #Alan said, we put a cap on our SMS messages per month.
I would use a combination of CATPCHA and Limit the requests of a Given Mobile Number.
In addition you should be able to specify with your SMS aggregator a preset limit per month. After you reach that limit, service is shutoff. That way if you are a victim of an attack, you will only be liable for a limited amount of money.
Instead of SMS, you can make use of an automated service that calls a phone number speaks out a One Time Password (via Text 2 speech). These services are similar in pricing to SMS, and less likely to get spam abused, as there is more overhead.
Twilio cost $0.03 a minute, or in this case, $0.03 a call.
You could do what Twitter does, which is have the user text you the token (rather than you texting it to them).
This will require you to find a provider that let's you receive texts for free (or close to it), but that might be easier.
Why is SMS costing you a dime? Utilize the EMAIL address that is associated with every SMS system (at least here in the U.S).
http://www.sms411.net/2006/07/how-to-send-email-to-phone.html
If someone tries their best to abuse a system, they will more than likely find a way to do it.
Using a combination of the techniques you've already come up with is likely the best way to thwart most malicious users.
Limit what people can do (no more than 10 requests from 1 ip in 10 minutes, one phone number can only recieve 3 texts a week, captcha before number entry), but more importantly, if people have no control over the content of the message there's no real reason to exploit it.