is this use case applicable to be implemented using NIFI
I want to develop a connector between two saas applications, to transfer data from system to system B. each application is multi-tenant. this connector works as the following
user insert in a form the authorization information for both systems
once authenticated the data will move on a scheduled basis to the other systems
not all the users in the saas will use this .. only group of them
data belonging for each user should not be overlapped with other users
Regards,
Related
As part of my project, I'd like to use microservices. The application is a store website where the admin can add products and the user can order and buy them.
I envision implementing four services: admin service, user service, product service, and order service.
I had trouble with handling data between multi services but it's solved by duplicating some necessary data using message brokers.
I can do this solution between product and user and order service because I need some of the data not all of them
Now, my question is about handling admin service because in this service I need to access all of the data, for example, the admin should have a list of users and the ability to add new products or update them.
how can I handle data between these services and the admin service?
should I duplicate all data inside the admin service?
should I use Rest API?
no thats wrong. it seems you want run away from the fact. in general duplication is an anti-pattern mostly in case you describe.
the way you thinking about admin-service is wrong.
because in this service I need to access all of the data
i dont think you need to have such a service. accessing the data based on users must be handled by Identity server(oidc Oauth) which is the separated service and handle the accessing end points .
for example the product-service provides 1-return product list 2-return individual product data 3-create data. the first two can access by both user and admin but the 3rd must be accessed by admin. one of identity server duty is to identify user in case of user interaction(login) with services.
ADMIN Scenario
user-client request create product endpoint(services eg:product.service).
client-app(front end app) is configed with identity server and realize there is no require identity tokens and redirect to identity server login.
NOTE: there is also identifying the client-app itself i skipped.
user-client login and get require token that based on his claims and roles and etc.
user-client request create product endpoint with tokens included in request header
endpoint (product service) receives the request and check the header (the services also configured base on identity server and user claims)
get the user claims info.
the create-product requires admin role if its there then there we go otherwise no access.
the image uses identity server 4 . there are also several kinds and also you can implement by your self using 0AUTH and oidc protocol libraries.
so the admin just request to the certain service not getting data through the separate service for this goal.
Communication between Service:
the most struggling part of microservices is the wiring it up. the wiring is directly the consequence of your design.(recommand deep study on Domain Driven Design).
asynchronous communication :
to avoid coupling between services mostly use asynchronous communication which you pass event eg:brokers like rabbitmq and kafka..etc , redis etc. in this communication the source service who send event does not care about response and not wait for it.just it always ready to listen for any result event. for example
the inventory service creates item
123|shoe-x22|22units
and this service fire event with data 123|shoe-x22(duplicate maybe or maybe not just id) to product service to create but it does not wait for response from product service that is it created successfully or not.
as you see this scenario is unreliable in case of fault and you need handle that so in this case you have to study CAP theory,SAGA,Circuit-breaker.
synchronous communication :
in this case the service insist to have response back immediately. this push service to become more coupling. if you need performance then you can use gRPC communication other wise simple api call to the certain service. in case of gRPC i recommand using libraries like MassTransit
which also can be used for implementingf gRPC with minimum coupling.
Some of Requests need data from multiple services
if you are in such situation you have two options.
mostly microservices architecture using APIGATE WAY (EG:nginx,OCELOT,etc)
which provide reverse-proxy,load balancing,ssl terminations etc. one of its ability is to merge the multiple responses from a request.but it just merge them not changing the data structure of response.
in case of returns desire response data structure you may create an Aggregator service which itself calls other two, gathers data and wrap it in desire format and return it.
so in the end still the Domain Driven Design is the key and i think i talked tooooo much. hope help you out there.
I started reading microservices architecture and got confused with one below point.
Each service should have a separate database.
Does each service mean a single web(rest) api having its own database?
For example if I take motor insurance claim operation as a business scenario where I modelled business domain services in 3 part Insurance claim services , partner (automobile service providers) services and customer services.
Insurance claim settlement operation in claim api will require other information like incident , survey done by an inspector, policy detail , documents etc.
Now I can create 5 web(rest) api in Insurance claim services and will store its data in common db and other services like partner and customer service will have their own web apis and db
What is correct ?
All web api (claimAPI, PolicyAPI, IncidentAPI, SurveyAPI and DocumentAPI) in claim insurance services should have their own db or they can keep data in single data base ?
Thanks.
To follow microservice best practice, it is correct that they should each have their own database and be exposed solely by APIs. This is because every service in your architecture should be independent and de-coupled from other services. If 2+ services share a database, then there may arise problems in operation or upgrade.
One big issue with a shared database is each service would need to trust that another service doesn't modify it's information. But since they all have access to the same database, one of the others could in fact modify the underlying data and make things unstable or insecure.
Further, with 2+ services relying on a shared database, then you're forced to use the exact same database/version with all. You lose the freedom to independently use MySQL for one and MongoDB for another. Even if the same tool is used for all, when you do maintenance or migration on one you're forced to do it for the rest. All this adds up to some coupled services that make them harder to maintain and scale.
you can have common database for all microservices, it is one of the microservices patterns:
https://microservices.io/patterns/data/shared-database.html
https://docs.aws.amazon.com/prescriptive-guidance/latest/modernization-data-persistence/shared-database.html
check those links to see advantages and disadvantages of this approach.
Let's assume below scenario.
There can be multiple tenants. Each tenant offers different services based on their business to clients. Depending on the size of the client, they may have their own database as well (ex:- Large clients prefer their data to be isolated).
Arranging tenants to fulfill above scenario is possible with the AspNetBoilerplate out of the box.
Then comes the customers who consumes various services provided by multiple businesses (tenants). There is a single mobile app for these customers. Once they login, they should be able to get services from any tenant unless a tenant has blocked a user.
Note that the point here is these customers are NOT registering for each tenant separately.
What are my options to get this setup done properly using AspNetBoilerplate? My initial thoughts are;
1) Have a separate table - MyCustomers - All mobile users will get authenticated against this table. For this should i create a separate auth pipeline ?
2) Current AspNetBoilerplate demo; you choose the tenant and login. But in my case; they log in and they will choose a service provider (tenant). At this point;
Should I create a new user automatically IF the selected tenant is in a different database (For large businesses having their own database)?
If the selected service provider is a small customer, who are in the same database differentiated with a TenantId I will have to additionally have a --> (many) relationship maintained so when a new service is selected a new record goes to this table?
At the end of the day, MyCustomers will only be using a single mobile app to get any service from any business after they login. Once they login, they should have access to all tenants (unless they are blocked).
Is there a better way to do this using AspNetBoilerplate existing architecture?
We are working on new requirement where we need to enable features based on contract with the client. It`s a Saas based software and support multi tenancy. The software by default have basic features enabled.
Basic features
Customer management
Basic billing/invoicing
Notification via email
Payments
Advance feature
Notification over SMS
Workflow
Etc
All the client serviced by same software (Obviously same deployment ) but data will be stored on different database schema.
How to handle basic, advanced, add on, and pro feature in this case?
We are using spring and hibernate.
AFAIK, you should be handling these internally in your application so that the features like notification, workflow are marked as application features and for each tenant, there will be a mapping between the tenant id and the feature so that the tenant has access only to the features that he is subscribed to.
So, when a tenant or his user logs in to the system, we identify the features and the contents that he can access and then show them to the user. This is called as a tenant licensing system.
Also, whenever a feature is consumed, your code should be recording the tenant that consumed that feature and by how much and when and there can be cost associated with units of usage measurements.
For more detailed discussion, please refer here
Though these are all explained in C#, there should be no hinderance in the conceptual understanding.
starting a proof of concept for this model it appears not to be possible to configure the membership provider per request to the respective clients database. It appears that the web application instantiates the membership providers on application start and doesn't allow modification to this during subsequent requests.
I have tried using reflection and modifying the connection string in the Application_BeginRequest (in the global.aspx) but it appears that the membership provider is designed and implemented to only initialise once per application rather than per request.
This implies for the separate database approach, the default asp.net membership provider wont work with the single application/multiple database (multitenant seperate database) model.
what authentication/authorization model do people use in this architecture? do they use a custom implementation of the Microsoft membership providers or do they use a different package or library?
Cheers
Tim
In the case of a multi-tenant application development scenario, the best preferred is a SSO or to go with a custom profile management system.
In order for you to enable tenant based authentication using the databases, you can have a connection string manager that identifies your tenant based on some of the criteria like the company code or the URL and then use the user login data from that database for that tenant's users.
The usage will be like you identify / infer the tenant code from the login credentials and then choose the database that is available for that tenant and then authenticate the user against this database. Hence the tenants can have their own databases for their user's authentication.
This is feasible and can also comply with the regulatory requirements that may be for different tenants based on their geography.