We have a Azure APIM gateway where we are integrating 100s of different API for other teams. We have two clients mobile and web and with user token we will be calling Azure APIs and before calling 100s of API we generate on behalf of user token. We would like improve performance by caching all 100s of on behalf of user token per user in external redis cache . Would like to know is there any better way to do this?
In this advanced token cache sample, we cover a similar scenario of sharing the same external token cache with multiple applications, including background services that don't have user interactions but still can use the users token cached.
Related
I plan to set up a set of microservices with an API gateway, I am new to microservices architecture but the services I plan to add more services and keep this application highly extensible. The API gateway should manage the users and their permissions and should delegate the incoming requests to the underlying microservices. But my problem is, how can I create a relationship between the user at the gateway and an entity in a microservice.
Like in the picture above i need to figure out what is the best practice to deal with user relations in the underlying services. I want to implement all the services with laravel the gateway should use laravel\passport.
My thought was that the API gateway is responsible for authenticating the users and forwarding requests to the services behind the gateway. If the user is authenticated, he has access to the services through the gateway. But how can I provide the service with the information about the user, for example, if the user edits an item in service A, how can I store which user edited the item. What would be the approach to establish this relationship?
There are many aspects to consider when selecting an approach, so basically answering your question will mostly be giving you pointers that you can research deeper on.
Here are some approaches you should review that will greatly depend on your service:
Authentication/Authorization method for the platform as a whole
How each individual service talks to each other (sync REST calls, messaging, GraphQL, GRPC, ...)
How are individual service's secured (each service is public and does auth, every service is behind a secured network and only the gateway is public, service mesh takes care of auth, ...)
The most common auth method in REST based microservices is OAuth, with JWT tokens. I recommend that you look deeper into that.
(Now just digressing a bit to demonstrate how much this varies depending on the use case and architecture)
Taking OAuth and looking at your question, you still have different flows in OAuth that you will use according to the use case. For example, generating tokens for users will be different than for services.
Then you still need to decide which token to use in each service: will the services behind the gateway accept user tokens, or only service-to-service tokens? This has implications to the architecture that you need to evaluate.
When using user tokens you can encode the user ID in the token, and extract it from there. But if you use user tokens everywhere, then it assumes services only talk to each other as part of a user flow, and you are enforcing that through the use of a user token.
If you go with service-to-service tokens (a more common approach, I'd say) you need to pass the user ID some other way (again, this depends your chosen architecture). Thinking of REST, you can use the Headers, Request Params, Request Path, Request Body. You need to evaluate the trade-offs for each depending on the business domain of each service, which influences the API design.
If you don't use tokens at all because all your services are inside a secured network, then you still have to use some aspect of your protocol to pass the user ID (headers, parameters, etc...)
Many code examples suggest to use token cache provided by msal to cache the access token and to take advantage of the nice feature that msal can silently acquiring token. But how does msal protect the access token in cache? Is it safe or can it be easily hacked?
Thanks you in advance.
The security of the cache depends on which token cache you use, generally dictated by your application's scenario. I'll focus on MSAL.NET from here but similar concepts apply to the libraries in other languages and environments.
In .NET web apps and APIs, there are several options available including in-memory and distributed caches such as Redis, SQL Server, Cosmos DB, or a custom solution. In-memory caching is only accessible from the same process. For distributed caches, it is the responsibility of the developer to protect these data sources from unauthorized access and instrument them to detect when a security breach has occurred.
In .NET desktop and mobile apps, token caching uses OS-specific tools for encrypting and storing tokens. How access is granted to tokens stored in these are OS-specific.
In all cases (except mobile), you can write your own, custom token serialization and caching mechanism. I don't recommend this for all but the most advanced scenarios.
We are currently deploying to AppSync using the serverless-appsync plugin and the serverless-framework (naturally).
Our API is 100% public and unauthenticated. All queries and mutations are basically public, since we have at this point no need for users (via a Cognito pool for example).
We have added a first layer of security using api keys but this is undoubtedly not much, as the api key is included as-is in the frontend sources. We would like to add extra roadblocks to make it harder for malicious users to abuse that frontend API.
A few things come to mind:
rate limiting (not currently supported by AppSync but I've read it's apparently in the works). I do not want to do this using a custom made solution in a Lambda for example.
making sure only traffic from the website is allowed to use our API, in addition to our own devs... I could probably do this with pipeline resolvers but I am not too keen with doing that in that obscure and unwieldy VTL language.
cors, ...
I'm considering switching to Apollo Server since this solution seems more open and configurable...
I would recommend using AppSync's IAM auth option and then use Amazon Cognito Identity Pools to vend temporary AWS credentials to your client applications. Identity pools (as opposed to user pools) vend temporary AWS credentials that assume the access of a role of your choosing. When configuring the identity pool, you can define a role with full access to the AppSync API or you may selectively provide access. From the client applications, you use the temporary credentials to sign the requests to AppSync using SigV4 and AppSync will only allow requests with a valid signature to be executed. The temporary credentials also provide an extra layer of security as even if they are compromised, each credential will only provide access up to the max allowed time.
I'll also add that you can use multiple auth modes at once with an AppSync API. This allows you to protect all fields as mentioned above and then selectively mark specific other fields such that they are authorized via some other mechanism like user pools or OIDC.
Google APIs can have usage limits, both on a per-user and a per-application basis. For example, the GMail API free tier is limited to a billion daily quota units across all users of your application.
This works for well-designed server-side applications, which can centrally ensure they obey these usage limits. However, I’m not sure how this is supposed to work for client-side apps. As Google’s documentation says,
Installed apps are distributed to individual devices, and it is assumed that these apps cannot keep secrets.
These apps are still supposed to use a client_secret and credentials, but these are assumed to not be confidential despite the name. However, just saying they aren’t secret doesn’t prevent abuse; a user of the app can take the credentials file and use it for a different purpose, perhaps one that uses the APIs more. What can an application developer do to prevent people doing this from burning through all the available quota?
Edit for clarification:
The use case that prompted this is a purely desktop app that doesn’t connect to any service except GMail (see https://github.com/mbrt/gmailctl/issues/48). If it weren’t for a global quota for all users of the app, there would be no reason to worry about individual users at all; they don’t connect to any service except GMail itself.
You could write a server app (a Cloud Function would work) which holds the secrets. Clients call your endpoint with some form of identifier and you return an Access Token. If your users have a browser, they can auth each time; if not you would need to request a Refresh Token which you store and use that to generate an AT.
I am currently reading a lot about microservices but still, I don't understand some parts. I made the following draw:
Each microservice has 2 accesses:
REST: For http uses
gRPC: For intra/background communication/exchanges
If I want to login I can just send an Http Request to my Authentication service. But what about if I want to access the Stuff service that needs you to be already connected?
Let say that the user wants to display the stuff available in the database STUFF, the service Stuff will first check if the "token" of the connected user is right, by exchanging with the Authentication service, and then return the stuff or a "login requires request".
So the thing I don't understand is, if each services that needs a client already connected needs to exchange with Authentication, then it will create a huge internet traffic in order to check each user request.. So I though about make one Authentication service per service, but since I should have only one Database, then it's the database that will slow the traffic?
Also, if I understand, each micro service should be on separate servers, not the same one?
I hope I am clear, don't hesitate to ask for more details !
Thanks in advance :)
Max
Edit 1
Based on #notionquest's answer:
So it should more looks like that right?
Also, based on Peter's comment, each service can implement its own middleware (JWT as mentioned) so the API Gateway is only a "pass-through". However, I don't feel like it could be a nice for me since each service make a token check for each internal exchange, doesn't it?
For the stuff, it's easy since it checks only 1 time the token. Now, let's say that, after the user got the stuff, he choose one and wanna buy it. Then the "Buying service" will call the stuff service in order the verify the price of the item, but... It will have to check the user token since the stuff is a "on authenticated access", so it means that "Buying" service and "Stuff" service both check the token, which add an extra check.
I though about an internal guaranteed access between services but is it worth it?
Also, maybe you said to implement the middleware for each service since they have a REST access, but the API Gateway would just destroy the idea of having REST access
There are multiple solutions available for this problem. One of the solution is API Gateway pattern.
First request goes to API gateway
API Gateway authenticates & authroizes the request
Authentication is stored on cache database such as Redis, Memcache etc with expiry time on it
Saved access token is returned to client
Client can use the saved access token in the subsequent calls for the some time span (i.e. until the token is valid)
Once the token is expired, the API gateway will authenticate and share the new token to client
This solution will reduce the need to authenticate each request and improves the performance
API Gateway is the single entry point for all the services. So, you may not need separate cache for each service.
Refer the diagram in this page.
Apart from #notionquest answer, there is another approach which does not involve having an API gateway;
You can share a SESSION_SECRET among all your services, so the only task of your Authentication Service is to validate username and password against the database and then encrypt this information using SESSION_SECRET and return a jwt token. All other services won't need to interact with Authentication Service but simply check if the jwt token is valid (can be decrypted) with the SESSION_SECRET.
You then have two other options;
Store all user data you need in the token - this will increase the amount of data in transit from your client to the micro-services. This can be prohibitive depending on the size of this information
You can store only the userId, and request additional data as needed per each micro service, which depending on how often/how big your data is will generate a problem as you described.
Note that you will not always be able to use this approach but depending on your specific scenario and requirements having this architecture in mind can be useful.
Also keep in mind that rotating the SESSION_SECRET can be tricky (although necessary for security reasons). AWS has just released a service called AWS Secrets Manager, so one idea to make things simple would be to have your micro-services periodically query a service like this for the current valid SESSION_SECRET instead of having this values hardcoded or as environment variables.