I am working on an app that has some social network elements: users can create posts with images and they can share these publicly or with friends.
I am now considering the security aspect of this. These images should only be available to the person that uploaded them and the people they select to view them.
From the posts I have seen it seems that one of the recommended ways is to expose an API endpoint through my backend service to control access through it (this way I can check a user's permissions) and then return the requested image but I feel that serving images this way would be quite expensive.
Are there any other approaches that do not sacrifice security but achieve a good performance?
In case it matters, I am using Spring Boot for my backend, Expo + React Native for my app and I am planning to store the images on AWS S3
It turns out S3 on AWS allows access to files through signed URLs, which means only people with the given signed URL can access the file. This signed URL can be further restricted by specifying the duration for which the signed URL will be valid.
Generating these URLs can be done by the back-end service without reaching out to AWS, so that does not create a big performance hit.
Related
I'm working on a Laravel site where users could connect their Google account and manage their Google calendars directly from within the site. I found https://github.com/spatie/laravel-google-calendar but I'm not sure that it really meets my needs.
That package doesn't seem to follow the authentication flow (OAuth 2) I'm used to with other APIs. It uses service accounts and stores credentials in JSON files where I usually save access and refresh tokens in my users table.
So am I missing something or is that package not made for that kind of site ?
I've been searching for a solution that makes sense for the past few days, so forgive me if I overlooked something or am ignorant to the correct path.
I have an existing Website built with asp.net framework MVC where users register for Individual Accounts and are stored in a MSSQL Database. I'm creating a Xamarin.Forms app where we want users to login with their website credentials.
I've been looking at Microsoft docs to try and accomplish this. In the past I created an Apache Cordova app (2014) that communicated in a similar fashion successfully (potentially in an insecure way) that called the /Token endpoint and Authenticated the user with 'grant_password' flow and returned the user data to the app.
So I landed on this documentation: https://learn.microsoft.com/en-us/azure/active-directory/develop/v2-oauth-ropc
However, from what I can gather it basically says this should be avoided moving forward and a MSAL approach should be used if possible.
After reading through hours and hours of documentation trying to make sense of it all, I can't seem to grasp what my options are. It seems to me that in any MSAL flow, users will have to login with Microsoft accounts or other social accounts (facebook, google).
Am I missing something?
Am I going to have to go against Microsoft's advice and end up storing the client_id and client_secret within the Xamarin app source code? Which from everything I've read is a big security concern, especially with Android apps.
References: 1. Restrict API requests to only my own mobile app 2. How to secure an API REST for mobile app? (if sniffing requests gives you the "key")
Any help or direction would be really appreciated. Thank you
Your focus should be on requirements + understanding preferred designs rather than jumping to a technology.
DIRECTION
Standard modern systems look like this:
Mobile apps use OpenID Connect to sign users in
Authorization Server issues access tokens to the mobile app
APIs authorize requests via JWTs containing scopes and claims
Aim to make iterative steps towards this type of architecture.
PATTERNS
Plug in an Authorization Server, which uses your existing database as a credential store
Mobile app uses AppAuth Libraries to sign users in and receives access tokens
Back end can handle JWTs without data security risks
DIFFICULT AREAS
Your existing back end may have no support for mobile clients, and be too web focused - eg requiring cookies to access data, so may need to be split in be into 2 entry points.
Choosing an Authorization Server (while you are learning) is difficult, because you may not know what you want yet.
The mobile app will spin up the system browser and present a login page from the Authorization Server, so the login UX could be unexpected.
STEP 1
Ensure that you can authenticate from the mobile app, then make API calls with JWTs and ensure that requests for data are properly authorized. This could use ropc and involve a temporary API. But the deliverable should be that your back end now supports calls from mobile apps.
STEP 2
Integrate AppAuth into the mobile app, which is tricky but there are resources online such as Curity Mobile Guides. Update the mobile app to use the Code Flow and integrate an Authorization Server, then deal with connecting to credential stores.
SUMMARY
This stuff is hard and reflects the cost of modernising architectures. It requires people agreement as well as the technical stuff. Happy to answer follow up questions if it helps.
I'm building an application that has a core hub, say it's called musictickets.com
We'll provide a subdomain (bandname1.musictickets.com) to bands on which only their content will display, which they can mask using a CNAME record to be part of their domain - so tickets.bandname1.com
There would be multiple bands using the platform so you'll end up with pages at
tickets.bandname1.com
tickets.bandname2.com
etc.
I'd like a user who registers at tickets.bandname1.com to be automatically logged in on every site that uses the service, including the parent, musictickets.com . They should be able to register/login using OAuth or directly via form based authentication.
I'm looking at SAML (specifically https://github.com/aacotroneo/laravel-saml2) as one option, but want to throw this out to the wider community for comment.
I've also looked at using token based SSO as described here (single sign on (sso) laravel) and running an auth server (which I may do in any case). Alternatively, I've looked at using iframes to provide the functionality which feels quick but dirty.
As I understand it, I wouldn't be able to use cookies (for an API key for instance) because whilst all of the content will be displayed via a subdomain, the CNAME would make it a different domain.
Does anyone have any thoughts on the best strategy?
I have stacked over the problem where I cannot understand how organisation secure theirs public apis from any person who collects data.
I know we use passport and other ways of auth tokens in order to protect private info from unauthorised user.
But there are things like public Search Engine that doesnt need user to be authenticated in order to find on Facebook a person via search or access Profile info page. Which means there are open public api's that does not required authentication from user.
But going through couple of organisations I did not manage to get any public api request that I could access via Postman or simple via url.
So I am interested how do organisation protect their public api's from requests.
How the frontend send requests to those public (kinda private api), or even if it has some sort of default api key for all sort of public api calls, how they secure those from people, if in our modern browsers we can access Local Storage or Cookies where we can extract that public api_token
I am confused over MERN STACK and Laravel + SPA React application.
Developing public routes for api calls they are all accessible from browser url or postman unless if the route is private and requires auth_token from passport or jsonwebtoken that is already requiring user to be registered. But I am trying to achieve in my application for users to be able to search and access Product Details without authentication.
But obviously I would not like if any sort of Big Data engineer would easily steal all public data from my web application unless if he is not lazy and doing html parsing for each public Product Details page.
So how do I secure those above described public api routes in my backend application.
And how does it do big organisations like Facebook, Google, LinkedIn and etc.
The reason why I am asking this question, because it is so easy to find some MERN Stack course and they will teach you how to handle authentication for authorised users and so one. Or even the LAMP technologies. But no one explains how to protect this data without requiring any user to be signed in.
Thanks you very much in advance to any person who is going to answer this long and very much confusing question.
But there are things like public Search Engine that doesnt need user to be authenticated in order to find on Facebook a person via search or access Profile info page.
When I was coding in PHP and working with Prestashop e-commerce I have used a Crawler/Bot similar to the one in this gist, but this can be easily spoofed, because it's based on the HTTP_USER_AGENT. A better approach here is to use IP addresses to lookup the good know crawlers, aka the ones from searches engines like Google and Bing, but this will not work for keeping at bay the bad crawlers and bots. because they switch IP addresses very frequently.
But going through couple of organisations I did not manage to get any public api request that I could access via Postman or simple via url.
Companies like Facebook or even smaller ones, that have a good amount of resources at their disposal, use Artificial Intelligence(AI) to try to draw the line between Who is doing good and bad requests, and this type of software is known as User Behavior Analytics (UBA):
User behavior analytics (UBA) as defined by Gartner is a cybersecurity process about detection of insider threats, targeted attacks, and financial fraud. UBA solutions look at patterns of human behavior, and then apply algorithms and statistical analysis to detect meaningful anomalies from those patterns—anomalies that indicate potential threats. Instead of tracking devices or security events, UBA tracks a system's users. Big data platforms like Apache Hadoop are increasing UBA functionality by allowing them to analyze petabytes worth of data to detect insider threats and advanced persistent threats.
So this must be why you have such a difficulty in getting through the APIs of companies like Facebook, but that doesn't mean it's not possible, because Hackers have being do it, and the number of data breaches occurring every year in big companies are a proof of that.
The reason why I am asking this question, because it is so easy to find some MERN Stack course and they will teach you how to handle authentication for authorized users and so one. Or even the LAMP technologies. But no one explains how to protect this data without requiring any user to be signed in.
Well that may be because of a common misconception among developers that don't really understand the difference between Who vs What is doing the request to the API server.
The Difference Between WHO and WHAT is Accessing the API Server
I wrote a series of articles around API and Mobile security, and from the article Why Does Your Mobile App Need An Api Key? I will quote the following:
The what is the thing making the request to the API server. Is it really a genuine instance of your mobile app, or is it a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?
The who is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
Think about the Who as the user your API server will be able to Authenticate and Authorize access to the data, and think about the What as the software making that request in behalf of the user.
So, in my opinion a lot of Developers are not aware of this difference between Who and What in a request, therefore they concentrate in solutions for the Who.
Possible Solutions
So how do I secure those above described public api routes in my backend application. And how does it do big organisations like Facebook, Google, LinkedIn and etc.
This organizations are using very sophisticated UBA solutions that may not be in the reach of every organization in terms of cost or because they are proprietary solutions, but other solutions exists, and you can read the section Defending the API Server on this other reply I gave to the question secure api data from calls out of the app to understand how you can gradually improve the security of an API server for a web app.
If you also need to secure the API server for requests from a mobile app, then you can lock it down with a very high degree of confidence to your mobile app, by employing the Mobile App Attestation concept, and you can read more about in this reply I gave to the question How to secure an API REST for mobile app?.
Do you Want to Go the Extra Mile?
I cannot finish any response to a security question without give a reference to the excellent work from the OWASP foundation.
For Web Apps
OWASP Web Top 10 Risks
The OWASP Top 10 is a powerful awareness document for web application security. It represents a broad consensus about the most critical security risks to web applications. Project members include a variety of security experts from around the world who have shared their expertise to produce this list.
The Web Security Testing Guide:
The OWASP Web Security Testing Guide includes a "best practice" penetration testing framework which users can implement in their own organizations and a "low level" penetration testing guide that describes techniques for testing most common web application and web service security issues.
For Mobile Apps
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.
For APIS
OWASP API Security Top 10
The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.
A lot of the time I have seen APIs secured via Basic Auth or OAuth. When you use Basic Auth you send an Authorization header with a base64 encoded username and password. Headers are encrypted when you connect with HTTPS / SSL.
OAuth is a bit involved, but follows a similar idea. Your OAuth token is send via encrypted headers as well. You can read more about OAuth here: https://www.digitalocean.com/community/tutorials/an-introduction-to-oauth-2
I'm currently working on a rather interesting... project. I have a client who wants to allow form uploads (from a page presented on their server) specifically to their own google drive account. The platform being used is essentially LAMP.
Single (pre-authenticated) google drive account. Multiple otherwise anonymous upload sources (users).
They do not want users to be required to have their own google accounts (rules out simply using Picker on the user's own drive files).
They want some degree of backwards browser compatibility, such as IE8 (rules out XHR to form the post using HTML5's file API to read the filedata). They don't want to use flash/etc due to potential compatibility issues with certain mobile browsers.
What is working:
Authenticating (getting a refresh token, storing, using it to get access tokens as needed)
Uploading a file to the account without metadata
Result of file upload being sent to hidden iframe
Catching the iframe load event via jquery to at least know something has happened
Problems:
The REST API upload endpoint does not support CORS: there is no way to access the result iframe directly. (see: Authorization of Google Drive using JavaScript)
The return from a successful upload is only raw JSON, not JSONP.
There is seemingly no way to host anything with proper headers to open via browser on the googleapis.com domain, so easyXDM and similar multi-iframe with cross origin workaround communication javascript approaches are ruled out.
There is no way to embed a callback URL in the POST from the submit, the API does not allow for it.
The Picker displays errors on trying to upload if you pass it an Oauth2 token that is not for a user who is also authenticated in their browser (assumedly via cookie). Strangely enough you can show files from the Oauth2 token's matching account, but other than in a browser instance where the target Oauth2 token's account matches the already logged in user any file uploads fail with an ambiguous "Server rejected" message. This happens with all files and file types, including files working in an authenticated browser instance. I assume it's an authentication flow/scope issue of some sort. I haven't tried diving the Picker source.
All of the javascript Google Drive API upload examples seem to rely on using HTML 5 to get the file data, so anything of that nature seems to be ruled out.
While files are uploaded, there's no way other than guesstimating which file came from which user, since we can't get the file object ID from the result in our inaccessible iframe. At best we could make a very rough time based guess, but this is a terrible idea in case of concurrency issues.
We can't set the file name or any other identifier for the file (not even a unique folder) because the REST API relies on that metadata being sent via JSON in the post request body, not via form fields. So we end up with file objects in the drive with no names/etc.
We can't create the file with metadata populated server side (or via jquery/XHR, or the google javascript API client) and then update it with a form based upload because the update API endpoint exclusively works with PUT (tested).
We can't upload the files to our local server and then send them to google (proxy them) as the php ini is locked down to prevent larger file uploads (and back to restrictions imposed on using HTML5 or flash for why we can't chunk files/etc).
All of this has been both researched and to varying degrees tried.
At the moment this is going on hold (at least it was a useful way to learn the API and gain a sense of its limitations) and I'm just going to implement something similar on dropbox, but if anyone has any useful input it would be lovely!
e.g. is there any way to get this working with Drive? Have I overlooked something?
I also realize that this is probably essentially a less than intended use-case, so I'm not expecting miracles. I realize that the ideal flow would be to simply allow users to upload if necessary to their own google drives and then have them grant file access to our web app (via Picker or API+our own UI), but this becomes a problem when not all of our own users are necessarily already google account users. I know that google would OBVIOUSLY prefer we get even more people to sign up with them in order to have this happen, but making people sign up for a google account to use our app was ruled out (not out of any prejudice on our part, it was just too many added steps and potential user hurdles). Even simply having them sign in to google if they did have accounts was deemed unwanted for the basic LCD feature functionality, although it's likely to be added as an additional option on top of whatever becomes the base solution.
The biggest problem with the approach you described is you're introducing a big security issue. Allowing an anonymous user to directly upload to Drive from the client requires leaking a shared access token to anyone who comes by. Even with the limited drive.file scope, a malicious or even slightly curious user would be able to list, access (read/update/delete!) any file that was uploaded by that app.
Of course a public drop box feature is still useful, but you really need to proxy those requests to avoid revealing the access token. If your PHP environment is too restrictive, why not run the proxy elsewhere? You can host a simple proxy to handle the uploading just about anywhere -- app engine, heroku, etc. and support whatever features you need to ensure the metadata is set correctly for your app.