Let's say you have an app where your user will authenticate with Picasa and Facebook in order for you to get all of the photos they have posted. To simply get all of a user's photos, both FB and Picasa require the same approach:
Get a list of albums for the user
Get a list of pictures for each album
So for any given provider with this approach, you are required to make N + 1 (N being number of albums) requests to the 3rd party. If you are doing a couple of these operations at once this seems like it would get preventably slow.
There seem to be a few alternatives to this approach:
Facebook:
Get all photos related to user
Parse these to find which were actually posted by the user
This will also give you other users' photos tagged with your user, so it may potentially end up performing worse than the original method due to sheer size of data as well as the number of request for paging involved.
Picasa:
There's a potential workaround here:
Get all photos from Picasa by person
That would probably work but seems hacky, i.e. what is a very high value that satisfies the allowable range but can still be guaranteed to be larger than the number of photos for the user.
I know this is not going to be fast no matter which route I go, but does anyone have suggestions on what I should do here? There's also always the possibility that I'm looking at it completely wrong too.
I suggest you use FQL->
http://developers.facebook.com/docs/reference/fql/photo/
and
http://developers.facebook.com/docs/reference/fql/photo_tag/
It allows you to make one big query and facebook process it on their end, you can tweak it so it returns to you a list of pictures where user is tagged in for example.
I'm sorry I can't help with Picasa though, I never worked with it.
Related
I was wondering if there is a way to limit the output of an encrypted or hashed value.
The case I have is, that I provide links for customers, having the id to an entry in my db, which contains relevant information for the receiver. Sooo... to avoid accessing to entries which are for someone else I am encrypting the id and append its outcome to the link. Now I am faced with the problem, that those "final links" are extremely long and ugly as f%!# (I actually got a lot responses, that they look highly suspicious and some of the customers didn't clicked on it, because they were afraid of being redirected to a phishing site).
However, this made me think about the option of limiting or individually setting the length of the encryptions outcome, like force it to contain 8 to 16 characters instead of about 250(? not sure how long they actually are). I also want to avoid using something like a redirecting page and a "self-made" URL shortener, because of the extra step I don't need.
Currently I've spent more than two hours of googling and reading several discussions regarding this topic and yeah... I am not satisfied with the results. Most of them started about two years up to five years ago.
What else I did?
I looked into Laravels api, especially into Illuminate\Encryption\Encrypter, but still no solution found.
Sooooo... I hope anyone can help me out with a solution based on laravel. I don't want to use anything else like php_mbcrypt itself than laravels encrypt or Hash::make.
Thanks in advance!
I believe I have a found a very good and fast solution for efficiently counting page views:
Working example in go playground here: https://play.golang.org/p/q_mYEYLa1h
My idea is to push this to the database every X minutes, and after pushing a key then delete it from the page map.
My question now is, what would be the optimal way to ensure that this isn't abused? Ideally, I would only want to increase page count from the same person if there was a time interval of 2 hours since last visiting the page.
As far as I know, it would be ideal to store and compare both IP and user agent (I don't want to rely on cookie/localstorage), but I'm not quite sure how to efficiently store and compare this information.
I'd likely get both the IP (req.Header.Get("x-forwarded-for")) and UserAgent (req.UserAgent()) from http.Request.
I was thinking making a visitor struct similar to my page struct that would look like this:
type visitor struct {
mutex sync.Mutex
urlIPUAAndTime map[string]time
}
This way should make it possible to do something similar to before. However, imagine if the website had so many requests that there would be hundreds of millions of unique visitor maps being stored, and each of these could only be deleted after 2 (or more) hours. I therefore think this is not a good solution.
I guess it would be ideal/necessary to write to and read from some file, but not sure how this should be done efficiently. Help would be greatly appreciated
One of optimization ways is to add a Bloom filter before this map. Bloom filter is a probabilistic structure which can say one of these:
this user is definitely new
and this user possibly was here
This is a way to cut off computation on early stage. If many of your users are new then you save requests to database to check all of them.
What if structure says "user is possibly non-unique"? Then you go the database and check it.
Here's one more optimization: if you do not need very accurate information and can agree with mistake about several percent, you may use the sole bloom filter. I guess many large sites use this technique for estimation.
I have an RSS feed that outputs around 100 articles per day. I wish to filter it to include only the more popular links, perhaps filter it to 50 or less. Back in the day, I believe you could use "postrank" to do this, but is now defunct after Google acquisition.
Anyone know how I can filter a specific RSS feed to include only the more popular outputs?
Thank you!
Your question is quite open, but one way to measure "popularity" of a post would be to use social networks.
Each post is probably associated with a certain URL, which can referred to on social networks. On facebook, for instance, one can share or like it. You could now get the number of shares for each post, order your list based on them and pop the top 50 ones.
This SO question asks for such share stats and contains answers for several networks.
Caution: Be aware that you'd need to do at least one API call per article. On the one hand, this may take some time, on the other hand, you'd have be careful to not exceed quotas. You're probably best off by locally caching fetched posts and only refresh every few hours or once a day.
I'm looking to add a feature in my web app that will determine whether a person has left a review on a specific Google Places listing. I want to verify that (1) user X left a review on the listing and (2) that the person I'm dealing with is user X.
I'm looking at the Google Places API, found here, but the most information I can find is on a simple rating.
Any leads on which API to use to find reviews left by users on Google Places?
As it turns out, there isn't a way to do this through an API, since there is so much risk of user reviews solicited by incentives. Google is one of the least picky review-collecting organizations, but they still try to deter biased reviews.
In other words, don't do this....
Okay, this question isnt exactly very clear, because i cant write it as a single question.
I have a game that i am designing using javascript, and it is basically a multiplayer game.
So say there is two players, player darklord and angel
angel shoots darklord
so darklord loses 1 life.
Now what happens is that i use ajax to submit the number of life that darklord loses.
And the request is GET /shootout.php?shooter=darklord&life=-1
so this allows me to store the new life of darklord.
Now the problem is say angel knows about computer, and he starts requesting /shootout.php?shooter=darklord&life=-3
Thus darklord loses more life then he should have. So angel cheated in the game.
No i want to prevent this kind of requests, and i am trying to get a way so that my requests can be hidden. I mean i know i can encrypt the url. So say i encrypted it such that the request should be GET /enc.php?e=934ufj30jf for darklord to lose a life, and different values of e for angel to lose a life, or gain a point. However for this to work i will need to send the data to the client, as in tell the javascript to request this url.
Now the user can easily go around reading the source of the file in order to find out what are the new requests for doing things,
I have found and thought of many other ways, but they all limit the amount of cheating or effect the game-play etc.. None of them eliminate this security completely.
So now my question is how do i make sure that users dont send data that is not real. How do i stop them from cheating?
I have thought of the best way being that i use server side scripts to actually calculate the possibility of someone shooting someone else and then matching it with the client input, but that will effect execution time by a LOT, so i am trying to find other ways, some public key encryptions?? (problem is the user can put the data as they want and then encrypt it) tokens? (problem is the user can put the data as they want and then put the current token)
so any other ideas anyone??
This isn't about hiding requests, it's about implementing proper access controls. Your example is referred to as an insecure direct object reference in that manipulating values in the querystring relating to direct DB objects causes an unintended outcome (have a look at OWASP Top 10 for .NET developers part 4: Insecure direct object reference).
There are a couple of things you can do but the most important is implementing proper access controls. You must authenticate the caller of the service and authorise them to perform the requested activity (and this all has to happen on the server). In this case, angle should not be able to perform an action on behalf of darklord.
The other thing you can do is use an indirect object reference map (refer to the link above), which obfuscates the IDs of the player with cryptographically strong, user-specific alternatives. You probably don't need this in addition to the access controls but it does give you more unpredictability.
Finally, think about the flip-side as well - if darklord is able to pass the amount of damage as a parameter, what's to stop him from re-issuing the request manually with "life=-100"? It will depend on the specifics of how the attack action is performed, but you're going to want to avoid people gaming this action too.
You have to assume that the user is completely in control of the client JavaScript. The only way to make this secure is to do the check on the server side.
You should not send result of action. You should send action.
i.e angel shoot darkangel from point (7,15) with angle 36 degree
than server checks is it correct shoot and decrease lifes of darklord
There was an excellent answer given on this subject a couple of years ago. It actually refers to Flash rather than JavaScript, but the security concerns and techniques are going to be applicable to this situation too.
What is the best way to stop people hacking the PHP-based highscore table of a Flash game
You should never have the client tell the server what changes to make in player state (eg. remove X amount of health) because the client could always be cheating. Instead only have the client tell the server what input the player has made and then the server determines what happens as a result of that input.
Although this doesn't remove the possibility of cheating by writing a bot that plays the game automatically (and is better at the game than any human player) you at least remove overt cheating of the "I did 10,000 damage, trust me" variety.
Detecting bots is best done by tracking behavioral data and doing data mining to find cheaters. And if there is no behavioral difference between bots and human players, then who cares about the bots.