So, I just started to learn about the microservices architecture so forgive me if this is an obvios question..
Let's say I have created 2 microservices User and Posts, each one has it's own DB and its decouple from one another trying to follow good practices, the architecture looks something like this:
Now, suddenly, I start to get a surge of calls to my Posts microservice and its too much for it to handle the requests so I want to scale it and create a new instance of it.
Now I have 1 instance of User
And I have 2 intances of Posts
Are these 2 instances of Posts sharing the same DB or
each instance has it's own DB and I have to find a way to sync both of them to maintain data consistency?
Are these 2 instances of Posts sharing the same DB
Yes they are sharing the same Database.
Regardless of the fact that you micro-service is deployed to multiple instances or one instance the deployment instances of your micro-service are accessing and using your one database which belongs to that micro-service.
All your instances of your micro-service should access the same database. This is the standard and most common use case. There are exceptions to this.
Here is an similar question where I explained it in detailed way.
All your Microservice (Posts) instances will be sharing single DB.
Generally it's a single Database against all of your particular micro service (Post in this case) instances. However in some cases we can have master copy of DB accompanied by read-only copy to divide the read and write traffic to different respective DBs. We now even have multi master configuration available where we could have even more than one master however that architecture require more thought and generally suitable for very high traffic sites.
Related
I know there are many questions like this and almost all answers are No. And the reason is a single microservice should be independent of another one. And if there is a change in a table, all microservices using that table need to be changed.
But my question is, if my database structure is fixed (hardly there will be any change in the table structure) will it be a good idea of creating multiple microservices pointing to same database.
Okay... here is my project.
We are going to a migrate struts 1.3/EJB 2.0 project to Angular/microservices. This project has 5 different modules and each module is a huge one. And this project is there in production since past 13 years. So there is very little chance of changing the table structures.
The reason I want to make different microservices is, since each modules are huge and complicated, and we still get requirements to add/change the business logics. So in that case, I can deploy only one microservice.
Any suggestions please.
I suggest creating a new service that access that database and all other services communicate with this service instead of directly to the database.
If you don't want to create a new service, at least access the DB using some database
abstraction layer.
For example, in SQL server use views and store procedures instead of directly access the tables.
I understand that microservice architecture suggests that each service should have its own private database. But when such a service is scaled, then is it one db per service instance or one db shared by all service instances?
Your first statement may be misleading to some: "each service should have its own private database."
Your architecture should be careful about sharing a single set of tables across multiple services-- that sharing frequently leads to a shared schema dependency, which creates a tight coupling that makes it difficult to update the schema without updating many of the services that share that schema at the same time.
However, sharing a single database instance (or database cluster) doesn't mean your services are accessing the same tables or even the same schema within the database. And if they aren't accessing the same tables, they aren't coupled. (Relying on the same database instance isn't coupling any more than relying on the same network. Don't confuse coupling with shared infrastructure.)
Frequently, multiple instances of the same service share the same database. In my opinion, there is nothing inherently wrong with this, but there are some things to be aware of. If you go this route, you need to be very careful when making changes to the data schema. Because multiple versions of that service may be accessing the data at the same time during updates, any schema changes need to compatible to at least any two adjacent versions. If you add a column or table, that's fine. The older version won't attempt to use it, so there will be no problem. (Note too, that the older version won't populate it either.) Removing a column or table is another problem entirely and to make that kind of breaking change, you will likely need to do it in several smaller steps to ensure that the older version of the service isn't broken. It can be done, it's just tougher.
A general rule of microservice development is that each microservice
should manage its own data. In an ideal world, the data managed by
each service would be completely independent. There would be no need
to propagate data changes made in one service to other services.
In the real world, however, complete data independence is impossible.
There will always be overlaps between the data used in different
services, Consequently, as an architect, you need to think carefully about
sharing data and managing data consistency. You need to think about
the microservices as an interacting system rather than as individual
units.
This means:
You should isolate data within each system service with as little
data sharing as possible.
If data sharing is mavoidable, you should design microservices so
that most sharing is read-only, with a minimal number of
services responsible for data updates.
If services are replicated in your system, you must include a
mechanism that can keep the database copies used by replica
services consistent.
Good question indeed. I would answer it like: "at least a database per microservice (not instance)"
A concern is the scalability of the databse itself, i.e. can service instances outscale the database?
If so, you could opt for e.g. an in-memory database or a sidecar for your microservice. The database would be ephemeral and you would need to populate it after the pod/container (re)starts. So the state not really lives in the database.
Apache Kafka is a tool that fits this spot, as it would allow you to populate the database after the service comes up and also provides the tooling to synchronize state for all currently running and future instances. But successfully implementing a Event-Sourcing with Kafka is not a trivial task, but you could come the conclusion that you don't need databases at all.
So the question remains, can service instances really outscale the database?
The answer would be "no" more often than not.
So by having a database instance per microservice (physically or logically) already gives you a lot in terms of "loose coupling and cohesive behaviour" as you don't share databases.
Another concern are breaking changes to the database between versions of the microservice. If things go wrong you could find yourself being unable to rollback. An ephemeral database could sync itself up in a compatible way.
Some say they change database technologies throughout the lifetime of a microservice, I never had the neccessity to do so, but an in-memory/sidecar approach would fit here very well.
I presume you share one database with all instances of one microservice. So that one update is available for every instance of the same microservice immediately. You may use one database instance per microservice instance to avoid the database as a single point of failure. But you would have to keep in sync every database which, it seems like an unnecesary overload for the database and application. I assume the database is able to keep a group of db instances in sync (every insert,update, delete is properly propagated).
Is it considered as a good practice to connect to two different databases in on microservice API Or I need to implement another microservice for working with the second database and call the new microservice API inside the first one?
The main thing is that you have only one microservice per database, but it is ok to have multiple databases per microservice if the business case requires it.
Your microservice can abstract multiple data sources, connect them, etc. and then just give consistent api to whoever is using it. And who's using it, doesn't care how many data sources there actually is.
It becomes an issue, if you have same database abstracted by multiple microservices. Then your microservice is no longer isolated and can break, because the data source you are using was changed by another team who's using the same data source.
I am now trying to design database for my micro service-oriented application in a distributed way. My application is related with management of universities. I have different universities say A, B, C. Each university have separate users for using their business data. Now I am planning to design separate databases for separate universities for storing their user data. So each university has their own database for their users and additional one database for managing their application tables. If I have 2 universities, Then I have 2 user details DB and other 2 DB for application tables.
Here my confusion is that, when I am searching for database design, I only see the approach of keeping one common database for storing all users (Here for one DB for all users of all universities). So every user is mixed within one database.
If I am following separate database for each university, Is possible to support distributed DB architecture pattern and micro service oriented standard? Or Do I need to keep one DB for all users?
How can I find out which method is appropriate for microservice / Distributed database design pattern?
Actually there could be multiple solutions and not one solution is best, the best solution is the one which is appropriate for your product's requirements.
I think it would be a better idea to go with separate databases for each of your client (university) to keep the data always isolated even if somethings wrong happens. Also with time, the database could go so huge that it could cause problems to configure/manage separate backups, cleanups for individual clients etc.
Now with separate databases there comes a challenge for managing distributed transactions across databases as you don't know which part is going to fail among many. To manage that, you may have to implement message/event driven mechanism across all your micro-services and ensure consistency.
Regarding message/event mechanism, here is a simple use case scenario, suppose there are two services "A" (user-registration) and "B" (email-service)
"A" registers a user temporarily and publishes an event of sending confirmation email.
The message goes to message broker
The message is received by "B".
The confirmation email is sent to the user.
The user confirms the email to "B"
The "B" publishes event of user confirmation to the broker
"A" receives the event of confirmation and the process is completed.
The above is the best case scenario, problems still can happen in between even with broker itself.
You have to go deep into it if you think you need this.
Some links that may help.
http://how-to-implement-a-microservice-event-driven-architecture-with-spring-cloud-stre
A Guide to Transactions Across Microservices
I don't think that this is a valid design, using a database per client which is a Multi-tenant architecture practice, and database per microservice is a microservice architecture practice. you are mixing things up.
if you will use microservice architecture you better design it as Bounded contexts and each Context has its own database to achieve microservices main rule Autonomy
I'm a beginner in microservice architecture and I have read in a lot of blog that in a microservice architecture, it is mandatory that each micro service has its own database. In my case it may cost very expensive.
My question is, is it possible to make the persistence layer as micro service in itself ? Which would have the function of allowing other microservices to have read/write access to the database.
Thanks
To answer your question first of all lets understand :
it is mandatory that each micro service has its own database. In my
case it may cost very expensive.
Yes it is said that every microservice should have its own database.
What they mean is tables/collection of each microservice should be separate (you could use a single scalable database instance) and one microservice should only access the data of other microservices only through API calls
Benefits of having a separate model are:
Model will be clean. Eg: In E-Commerce Customer have diff. meaning for Shipping Microservice, Order Microservice, Customer Management Microservice and so on. If we put all data required by multiple microserives Customer Object will become very big
Microservices could evolve independently. In this case if we have a single Customer object and one microservice lets say Order one want to add something to the schema, all microservices needs to change
If we have a single Database Schema we will be getting into a big mess.
In my case it may cost very expensive.
If expensive means read model actually require data from multiple microservices. then its better to listen to events from multiple microservices and create a single read model , little duplication of data is ok.
If anything else, ask more specific question.
Having all Microservices accessing the same database will result in Loose Cohesion and Strong Coupling
Try to see if you can define separate Schema for each of the Microservices, so that you can ensure Microservices doesn't refer to the tables of other MicroServices.
This way in future, you can seamlessly move to separate Database for each service when your infrastructure cost concern goes off.
Micro services follows database per service model