I'm currently building a RESTful API to our web service, which will be accessed by 3rd party web and mobile apps. We want to have certain level of control over API consumers (i.e. those web and mobile apps), so we can do API requests throttling and/or block certain malicious clients. For that purpose we want every developer who will be accessing our API to obtain an API key from us and use it to access our API endpoints. For some API calls that are not dealing with the specific user information, that's the only required level of authentication & authorization, which I call "app"-level A&A. However, some API calls deal with information belonging to the specific users, so we need a way to allow those users to login and authorize the app to access their data, which creates a second level (or "user"-level A&A).
It makes a lot of sense to use OAuth2 for the "user"-level A&A and I think I have a pretty good understanding of what I need to do here.
I also implemented OAuth1-like scheme, where app developers receive a pair of API key & secret, supply their API key with every call and use secret to sign their requests (again, it's very OAuth1 like and I should probably just use OAuth1 for that).
Now the problem that I have is how to marry those two different mechanisms. My current hypothesis is that I continue to use API key/secret pair to sign all requests to be able to access all API endpoints and for those calls that require access to user-specific information apps will need to go through OAuth2 flow and obtain access tokens and supply them.
So, my question to the community is - does it sounds like a good solution or there are some better ways to architect this.
I'd also appreciate any links to existing solutions that I could use, instead of re-inventing the wheel (our services is Ruby/Rails-based).
Your key/secret pair isn't really giving you any confidence in the authorship of mobile apps. The secret will be embedded in the executable, then given to users, and there's really nothing you can do to prevent the user from extracting the key.
In the Stack Exchange API, we just use OAuth 2.0 and accept that all we can do is cutoff abusive users (or IPs, in earlier revisions without OAuth). We do provide keys for tracking purposes, but they're not secret (and grant nothing of value, so there's no incentive to steal them).
In terms of preventing abuse, what we do is throttle based on IP in the absence of an auth token, but switch to a per-user throttle when there is one.
When dealing with purely malicious clients, we unleash the lawyers (malicious in our case is almost always violation of cc-wiki guidelines); technical solutions aren't sophisticated enough in our estimation. Note that the incidence of malicious clients is really really low (single digits in years of operation, with millions of daily API requests).
In short, I'd ditch OAuth 1.0 and switch your throttles to a hybrid of IP and user based.
Related
I've read a lot of articles on this subject and they all suggest completely different things that I can't yet structure in my head.
I have one backend app (spring-boot + kotlin). I have nginx and one android (kotlin) mobile app uses backend api and of course Postgres. By the way backend app and postgres are packages in docker containers via docker-compose.
My task is to make the API of my backend service can only be used by this mobile application and no one else. But I also want it to be able to use the API if I have a Web application in the future.
I would be fantastically grateful if you could describe, in a few words, modern technology that could be used to accomplish my task.
For example:
Spring-security: a huge thing that you don't know what to do with, most likely you can use it to solve your problems, but it's overkill. But if you decide to use spring-security, this will help you {...}
...
By the way, I'm not against spring-security, I just really think it's too much for my task. But I'd be happy to hear your opinion.
Your Problem
My task is to make the API of my backend service can only be used by this mobile application and no one else. But I also want it to be able to use the API if I have a Web application in the future.
You have in hands a very hard task to complete. While not impossible it's very hard to accomplish with code written on your own or by trying to leverage security features on your framework of choice.
To understand why it's so hard you first need to understand the difference between who is in the request versus what is doing the request.
The Difference Between WHO and WHAT is Accessing the API Server
I wrote a series of articles around API and Mobile security, and in the article Why Does Your Mobile App Need An Api Key? you can read in detail the difference between who and what is accessing your API server, but I will extract here the main takes from it:
The what is the thing making the request to the API server. Is it really a genuine instance of your mobile app, or is it a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?
The who is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
So think about the who as the user your API server will be able to Authenticate and Authorize access to the data, and think about the what as the software making that request in behalf of the user.
After you understand this idea and it's ingrained in your mindset, you will look into mobile API security with another perspective, and you will be able to see attack surfaces that you never though they could exist.
Possible Solution
I would be fantastically grateful if you could describe, in a few words, modern technology that could be used to accomplish my task.
I recommend you to read this answer I gave to the question How to secure an API REST for mobile app?, especially the sections Hardening and Shielding the Mobile App, Securing the API Server and A Possible Better Solution.
The best approach to solve your problem is to go with a Mobile App Attestation solution suggested in the answer I linked. A Mobile App Attestation needs to be able to work in tandem with your mobile app and backend in order for the backend to have a very high degree of confidence that what is making the request is indeed a genuine version of your mobile app, that hasn't been tampered with statically or at runtime, and it's not under a
MitM Attack
The Manipulator-in-the middle attack (MITM) intercepts a communication between two systems. For example, in an http transaction the target is the TCP connection between client and server. Using different techniques, the attacker splits the original TCP connection into 2 new connections, one between the client and the attacker and the other between the attacker and the server, as shown in figure 1. Once the TCP connection is intercepted, the attacker acts as a proxy, being able to read, insert and modify the data in the intercepted communication.
The MITM attack is very effective because of the nature of the http protocol and data transfer which are all ASCII based. In this way, it’s possible to view and interview within the http protocol and also in the data transferred. So, for example, it’s possible to capture a session cookie reading the http header, but it’s also possible to change an amount of money transaction inside the application context
Be aware that solutions to solve your problem that are specific to the backend or to the mobile app will not be able to achieve a very high degree of confidence in securing your API backend from serving requests not originated from your genuine mobile app, but it's better to have them then nothing.
Do You Want To Go The Extra Mile?
In any response to a security question I always like to reference the excellent work from the OWASP foundation.
For APIS
OWASP API Security Top 10
The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.
For Mobile Apps
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.
The easiest way probably is to define a shared secret on the phone and the backend service.
On the mobile phone, with each request, you send the secret, e.g., as an HTTP header.
On the backend, you need to implement a Filter (e.g., OncePerRequestFilter) that checks the request for the secret and compares it to the value stored in the backend.
Google APIs can have usage limits, both on a per-user and a per-application basis. For example, the GMail API free tier is limited to a billion daily quota units across all users of your application.
This works for well-designed server-side applications, which can centrally ensure they obey these usage limits. However, I’m not sure how this is supposed to work for client-side apps. As Google’s documentation says,
Installed apps are distributed to individual devices, and it is assumed that these apps cannot keep secrets.
These apps are still supposed to use a client_secret and credentials, but these are assumed to not be confidential despite the name. However, just saying they aren’t secret doesn’t prevent abuse; a user of the app can take the credentials file and use it for a different purpose, perhaps one that uses the APIs more. What can an application developer do to prevent people doing this from burning through all the available quota?
Edit for clarification:
The use case that prompted this is a purely desktop app that doesn’t connect to any service except GMail (see https://github.com/mbrt/gmailctl/issues/48). If it weren’t for a global quota for all users of the app, there would be no reason to worry about individual users at all; they don’t connect to any service except GMail itself.
You could write a server app (a Cloud Function would work) which holds the secrets. Clients call your endpoint with some form of identifier and you return an Access Token. If your users have a browser, they can auth each time; if not you would need to request a Refresh Token which you store and use that to generate an AT.
I am currently reading a lot about microservices but still, I don't understand some parts. I made the following draw:
Each microservice has 2 accesses:
REST: For http uses
gRPC: For intra/background communication/exchanges
If I want to login I can just send an Http Request to my Authentication service. But what about if I want to access the Stuff service that needs you to be already connected?
Let say that the user wants to display the stuff available in the database STUFF, the service Stuff will first check if the "token" of the connected user is right, by exchanging with the Authentication service, and then return the stuff or a "login requires request".
So the thing I don't understand is, if each services that needs a client already connected needs to exchange with Authentication, then it will create a huge internet traffic in order to check each user request.. So I though about make one Authentication service per service, but since I should have only one Database, then it's the database that will slow the traffic?
Also, if I understand, each micro service should be on separate servers, not the same one?
I hope I am clear, don't hesitate to ask for more details !
Thanks in advance :)
Max
Edit 1
Based on #notionquest's answer:
So it should more looks like that right?
Also, based on Peter's comment, each service can implement its own middleware (JWT as mentioned) so the API Gateway is only a "pass-through". However, I don't feel like it could be a nice for me since each service make a token check for each internal exchange, doesn't it?
For the stuff, it's easy since it checks only 1 time the token. Now, let's say that, after the user got the stuff, he choose one and wanna buy it. Then the "Buying service" will call the stuff service in order the verify the price of the item, but... It will have to check the user token since the stuff is a "on authenticated access", so it means that "Buying" service and "Stuff" service both check the token, which add an extra check.
I though about an internal guaranteed access between services but is it worth it?
Also, maybe you said to implement the middleware for each service since they have a REST access, but the API Gateway would just destroy the idea of having REST access
There are multiple solutions available for this problem. One of the solution is API Gateway pattern.
First request goes to API gateway
API Gateway authenticates & authroizes the request
Authentication is stored on cache database such as Redis, Memcache etc with expiry time on it
Saved access token is returned to client
Client can use the saved access token in the subsequent calls for the some time span (i.e. until the token is valid)
Once the token is expired, the API gateway will authenticate and share the new token to client
This solution will reduce the need to authenticate each request and improves the performance
API Gateway is the single entry point for all the services. So, you may not need separate cache for each service.
Refer the diagram in this page.
Apart from #notionquest answer, there is another approach which does not involve having an API gateway;
You can share a SESSION_SECRET among all your services, so the only task of your Authentication Service is to validate username and password against the database and then encrypt this information using SESSION_SECRET and return a jwt token. All other services won't need to interact with Authentication Service but simply check if the jwt token is valid (can be decrypted) with the SESSION_SECRET.
You then have two other options;
Store all user data you need in the token - this will increase the amount of data in transit from your client to the micro-services. This can be prohibitive depending on the size of this information
You can store only the userId, and request additional data as needed per each micro service, which depending on how often/how big your data is will generate a problem as you described.
Note that you will not always be able to use this approach but depending on your specific scenario and requirements having this architecture in mind can be useful.
Also keep in mind that rotating the SESSION_SECRET can be tricky (although necessary for security reasons). AWS has just released a service called AWS Secrets Manager, so one idea to make things simple would be to have your micro-services periodically query a service like this for the current valid SESSION_SECRET instead of having this values hardcoded or as environment variables.
I have this webservice at work. For that webservice our department have developed a client to consume the webservice.
What we want to prevent is, that they develop any other client to consume it.
Is there any algorithm, practice that we can improve in our client and webservice communication to validate that the consuming client is our application?
I have an idea, that we can develop an encrypting algorithm about the time (5 seconds of grace) that must match with the calculated on server.
But i want to be sure, there is no best practice for that... or if it still a good idea...
(sorry about my english)
I think part of the point of web services (SOAP or REST based, for example) is to publish an interface that will let your service potentially be used by a variety of client implementations (interoperability is one of the motivations for web services).
If you want to lock your service to be used by your client, the only benefit of having it as a "web service" is probably the tools and libraries with which you've implemented it. You may want to consider whether this was worthwhile (it's possible indeed).
If you distribute your client application, chances are that whatever protection mechanism that may ensure the requests come from that client will have to be embedded with this client. Therefore whatever secret mechanism you embed will probably be only obfuscated to a certain point, but breakable by more advanced users.
What you are looking for is known as "authentication".
You need application authentication. For Web based application and services try looking into 2-legged OAuth. In OAuth You give out an id and a secret for every client app which accesses your service and every message is signed for extra security.
The other minute I read an article on OAuth. It described especially the tokens being exchanged between client and service provider during a series of requests.
The article also mentioned that OAuth gains significant popularity in RESTful APIs as authorization layer. As I understood, REST should be kept completely stateless.
The question: Doesn't this repeated token exchange torpedo REST's "being stateless" principle? IMHO the tokens can be seen as a kind of session ID, can't they?
OAuth tokens are explicitly a session identifier, interaction is not stateless between requests in the OAuth token negotiation protocol as the requests must be performed in a specific sequence, and they do require per-client storage on the server as you need to track things like when they were issued. So yes, OAuth does violate the strict principles of a RESTful architecture.
Unfortunately there's the Real WorldTM to contend with where we need to do things like allow applications to authenticate on the behalf of individuals without requesting their password, which OAuth does fairly well. It would be impossible to implement a similarly secure authentication scheme without this kind of state. Indeed, one of the changes required by OAuth (1.0a) was to add more state to the token negotiation protocol to mitigate a security risk.
So, does it torpedo REST's stateless principle? Yes. Does that matter? Not unless you live in an ivory tower :-)
Authentication is a state that must be tracked somehow when dealing in web interactions. Ultimately if your app is restful or not, the server must be able to track each users "authenticated state" and unfortunately that requires some kind of circumvention of the underlying stateless nature of HTTP and any additional transports/techniques (like REST) on top of it.
Hence to develop any kind of authenticated app, a principle of state must be shoe horned in somewhere, and if that so happens to be OAuth on top of REST, thats how it must be!