I am currently trying to decouple a monolith into microservices and am running into performance issues. Currently, I have a ServiceX which requires a list of parameters to run, call them p1, p2, and p3. However, the last 2 parameters are data structures which are loaded from other services such as ServiceY and ServiceZ. Therefore, I have to call 2 services to gather p2 and p3 before being able to call ServiceX. In addition, ServiceY and ServiceZ may have their own dependencies from other services. My initial strategy was to instead have my ServiceX hit the DB directly in order grab the data I need for p2 and p3 internally. Therefore, my ServiceX will no longer be reliant on ServiceY and ServiceZ.
However, now, ServiceX takes a large performance hit by making these extra DB calls. In addition, I now have duplicate code to load p2 in ServiceX and ServiceY and duplicate code to load p3 in ServiceX and ServiceZ.
Is there a better pattern I should be using to get around these issues? The data provided from ServiceX, ServiceY, and ServiceZ is different and belongs in different services.
So one of the fundamental tennets of microservices is that each individual service owns its own data completely; and your solution of reading directly from another services's data violates that premise directly. It is not what you need. Your better solution here is to introduce a caching layer between the services where the performance is a problem.
Another solution to the problem is going to be greater parallelism for the services and better optimization for the data layer (is the persistence technology being used the correct one for the problem, for example.)
It is extremely difficult to give you a reasonable answer beyond extremely generic architectural pointers; but maybe this will help.
Related
Is it a cardinal rule of microservices that a single database table should only be represented by a single microservice? I was asked that in an interview. My first reaction was that it should only be 1 to 1. But then I think I was overthinking it, thinking that maybe there are some edge case scenarios where that may be acceptable.
So is it a cardinal rule of microservices that a single database table should always be represented by a single microservice? Or are there some edge case scenarios where that may be acceptable? If it's a cardinal rule then is there any type of standard acronym that includes that principal? For example, relational databases have the ACID principals.
It is not a cardinal rule. But, it is the most effective way to manage data. Design patterns are not set in stone. You may choose to handle things differently.
But, each microservice should be independent. This is why we use the microservices architecture. But, say you update a table using multiple microservices, then they (the services) become interdependent. Loose coupling no longer exists. The services will impact each other any time a change takes place.
This is why, you may want to follow the following paradigms:
Private-tables-per-service – each service owns a set of tables that
must only be accessed by that service.
Schema-per-service – each service has a database schema that’s
private to that service
Database-server-per-service – each service has it’s own database
server.
Refer to the data management section here for more: https://microservices.io/patterns/
It is not just a separate database for individual microservices, there are other factors that need to consider while developing microservices like codebase, config, log etc.
Please refer to below link which explains in detail.
https://12factor.net/
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 days ago.
Improve this question
Which are the conditions based on which a system should be split in micro-services and how "small" should a micro service be?
We where implementing micro-services architecture in multiple projects and I
will try to share my experience with them and how we where doing it.
Let me first try to explain how we split our Domain into micro-services. During this the
criteria for how small or how big the micro-service should be will be explained as well. In order to
understand that we need to see the whole approach:
Conditions based on which we split our system in micro-services:
We where splitting the micro-services based on 2 set of environments:
Based on Production/Staging setup.
This is in general how would the system run on a Production environment to be used
by our customers.
Based on Development(Developers development machine) setup.
This the setup which each developer has to have on their machine in order
to run/debug/develop the full system or parts of it.
Considering only the Production/Staging setup:
Based on DDD(Domain Driven Design) Bounded Context:
Where Bounded Context = 1 micro-service.
This was the biggest ms that we ended up having. Most of the time the
Bounded Context was also split into multiple micro-services.
Why?
The reason is that our Domain was very big so keeping the whole Bounded Context
as one micro-service was very inefficient. By inefficient I mean mostly scaling reasons, but
also development scaling(having smaller teams taking care of one micro-service) and some other reasons
as well.
CQRS(Command Query Segregation Principle):
After splitting to micros-service per Bounded Context or multiple micro-services per 1 Bounded Context
for some of those micro-services we had them split in 2 or more microservices instead of 1. One Command/Write micro-service and the second was the Read/Query micro-service.
For example
lets say you have a "Users micro-service" and "Users-Read micro-service". The "Users micro-service" was responsible for
creating, updating, deleting and general management of Users. On the other hand the "Users-Read micro-service" was just responsible
for retrieving Users(it was read-only). We where following the CQRS pattern.
The Write/Domain micro-service in some extreme cases had multiple Read micro-services. Sometimes these Read-micro-service where so small that they where just having one De-Normalized View-like
representation mostly using some kind of No-SQL db for fast access. In some cases it was so small that
from code prospective it would just have a couple of C#/Java classes in it and 1,2 Tables or JSON Collections
in their Database.
Service which provides Domain agnostic or Static work or services
Example 1 of this was a very small micro-service which was responsible for generating
specific reports in form of pdf from an html template.
Example 2 was a micro-service which was just sending simple text messages to some specific users
based on their permissions.
Considering the Development Setup:
In addition to the ones for Production/Staging setup for local purpose of developing/running we needed
special micro-services which would do some work in order to make the local setup work.
The local setup was done using docker(docker-compose).
Examples of small micro-services there was:
Database, Cache, Identity, Api Gateway and File Storage.
For all this things in the Production/Staging setup we where using a Cloud provider which
provides services for these things so we did not have a need to put them in micro-services.
But in order to have a whole system running for development/testing purposes we needed to
create small micro-services in form of Docker containers to replace these Cloud services.
Adding test/seeding data into the system.
In order to feed the local development system of micro-services with data we needed to have a
small micro-service which sole purpose was to call some API's exposed by the micro-services and
post some data into them.
This way we could setup a working development environment with predefined data in order to test
some simple business scenarios.
The good thing about this is that you can use this test data setup in combination with the local
development setup for creating integration and end-to-end tests.
How small a micro-service should be?
In one our cases the smallest micro-services where a couple of
View/Read-only mircro-services which had only one De-Normalized View(1 Table or 1 JSON Collection) which from code prospective
had a couple of C#/Java classes in it. So when it comes to code I don't think that much smaller then this would
be a reasonable option. Again this is subjective view.
This size can be considered as "too small" based on some suggestions around micro-service
which you can read about online. We did that because it helped us solve performance issues.
The demand for this data was so big that we isolated it so that we can scale it independently.
This gave us the possibility to scale this micro-service/view individualy based on its needs and independently
from the rest of that Domain.
Second case of a small micro-service is the html-to-pdf service which was just creating pdf documents based
on specific formatted html template. You can imagine how small this subset of functionality was.
Suggestion:
My suggestion for everyone designing micro-services would be to ask the right questions:
How big should micro-service be so that we don't have situation with monolith split into multiple monoliths?
This means that the size of the created micro-services would be too big and hard to manage as this was the problem
for monoliths. On top of this you get the drawbacks of distributed systems.
Is that size of a micro-service going to affect your performance?
For you customers the key is that the system performance is good, so considering this as
criteria for micro-services system architecture could be a very valid point.
Should or can we extract some critical part of the functionality/logic in order to isolate it?
Which critical logic is so important that you can not afford to have it broken or service-downtime
for it?
This way you can protect your most critical part of the system.
Can I organize my team or teams with this kind of micro-services architecture split?
Considering I have the micro-services architecture done and I have "n" micro-services
how will I manage them? This means support them, orchestrate deployment, scale based on needs,
monitor and so on?
If this architecture that you came up turns out to be challenging and not manageable for your
organisation or teams then reconsider them. Nobody needs an unmanageable system.
There are many more which could could lead you to the right directions but these where the ones we where following.
Answers to those questions will automatically lead you to the smallest/biggest possible micro-service
for your Domain/Business. My example about micro-services size from above might now work for your case
and the rules that we where using as well, but answering these questions will bring you closer to your
own approach/rule to decide that.
Conclusion
The micro-services should be as small as you need it to fit your needs. The name "micro" indicate that they can be very small.
Be careful to not make this a rule for all your system.
The smallest micro-services are rather an exception to solve some specific problem like scaling or
critical logic isolation rather then a rule for designing/splitting the whole system in micro-services of that size.
If you have to many very small micro-services just for the sake of having them and them being small you will have hard time
in managing them with no real benefit. Be careful how you split it.
Folks, I am evaluating options/ pattern and practices around key challenge of maintaining db atomicity (across multiple tables) that we are facing in distributed (microservices) architecture.
Atomicity, reliability and scale all are critical for business(it might have been common across businesses, just putting it out there).
I read few articals about achieving but it all comes at a significant cost and not without certain trade offs, which I am not ready to make.
Read couple of SO questions, and one concept SAGA seems interesting, but I don’t think our legacy database is meant to handle it.
So here I am asking experts of their personal opinion, guidance and past experience so I can save time and effort without try and learn bunch of options.
Appreciate your time and effort.
CAP theorem
CAP theorem is the key when it comes to distributed systems. Start with this to know if you want availability vs consistency.
Distributed transactions
You are right, trade offs involved and there is no right single answer. when it comes to distributed transaction it's no different. In microservices architecture Atomicity is not easy to achieve. Normally we design the microservices by keeping eventual consistency in mind. Strong consistency is very hard and not a simple solution.
SAGA vs 2PC
2PC it's very easy to achieve atomicity using 2 phase commit , but that option is not for microservices. your system can't scale system since if any of the microservice goes down your transaction will hang into abnormal state and locks are very common with this approach.
SAGA is most acceptable and scaleable approach . You commit local transaction (atomically) once done you need to publish the event , and all the interested services will have to consume the event and update their own local database. If there is exception or particular microservices can't accept the event data , it would raise compensation transaction , which mean you have to reverse and undo the actions taken by all microservices against that event. This is widely accepted pattern and is scaleable.
I don't get legacy db part. What makes you think legacy DB will have problem ? SAGA has nothing to do with legacy system . It simply mean if you have to accept the event or not. If yes then save it into database. If not then raise compensated transaction so all other service can undo.
What's the right approach ?
Well it really depends on you eventually. There are many pattern around when it comes to save the transaction . Have a look at CQRS and event sourcing pattern which is used to save all the domain events. Since disturbed transactions can be complex . CQRS solve many problems e.g. eventual consistency etc.
Hope that helps! shoot me questions if you have.
One possible option is Command Query Responsibility Segregation (CQRS) - maintain one or more materialized views that contain data from multiple services. The views are kept by services that subscribe to events that each services publishes when it updates its data. For example, the online store could implement a query that finds customers in a particular region and their recent orders by maintaining a view that joins customers and orders. The view is updated by a service that subscribes to customer and order events.
In my current project, the business logic is implemented in stored procedures (a 1000+ of them) and now they want to scale it up as the business is growing. Architects have decided to move the business logic to application layer (.net) to boost performance and scalability. But they are not redesigning/rewriting anything. In short the same SQL queries which are fired from an SP will be fired from a .net function using ADO.Net. How can this yield any performance?
To the best of my understanding, we need to move business logic to application layer when we need DB independence or there is some business logic that can be better implemented in a OOP language than an RDBMS engine (like traversing a hierarchy or some image processing, etc..). In rest of the cases, if there is no complicated business logic to implement, I believe that it is better to keep the business logic in DB itself, at least the network delays between application layer and DB can be avoided this way.
Please let me know your views. I am a developer looking at some architecture decisions with a little hesitation, pardon my ignorance in the subject.
If your business logic is still in SQL statements, the database will be doing as much work as before, and you will not get better performance. (may be more work if it is not able to cache query plans as effectivily as when stored procedures were used)
To get better performance you need to move some work to the application layer, can you for example cache data on the application server, and do a lookup or a validation check without hitting the database?
Architectural arguments such as these often need to consider many trades-off, considering performance in isolation, or ideed considering only one aspect of performance such as response time tends to miss the larger picture.
There clearly some trade off between executing logic in the database layer and shipping the data back to the applciation layer and processing it there. Data-ship costs versus processing costs. As you indicate the cost and complexity of the business logic will be a significant factor, the size of the data to be shipped would be another.
It is conceivable, if the DB layer is getting busy, that offloading processing to another layer may allow greater overall throughput even if the individual responses time are increased. We could then scale the App tier in order to deal with some extra load. Would you now say that performance has been improved (greater overall throughput) or worsened (soem increase in response time).
Now consider whether the app tier might implement interesting caching strategies. Perhaps we get a very large performance win - no load on the DB at all for some requests!
I think those decisions should not be justified using architectural dogma. Data would make a great deal more sense.
Statements like "All business logic belongs in stored procedures" or "Everything should be on the middle tier" tend to be made by people whose knowledge is restricted to databases or objects, respectively. Better to combine both when you judge, and do it on the basis of measurements.
For example, if one of your procedures is crunching a lot of data and returning a handful of results, there's an argument that says it should remain on the database. There's little sense in bringing millions of rows into memory on the middle tier, crunching them, and then updating the database with another round trip.
Another consideration is whether or not the database is shared between apps. If so, the logic should stay in the database so all can use it.
Middle tiers tend to come and go, but data remains forever.
I'm an object guy myself, but I would tread lightly.
It's a complicated problem. I don't think that black and white statements will work in every case.
Well as others have already said, it depends on many factors. But from you question it seems the architects are proposing moving the stored procedures from inside DB to dynamic SQL inside the application. That sounds very dubious to me.
SQL is a set oriented language and business logic that requires massaging of large amount of data records would be better in SQL. Think complicated search and reporting type function. On the other hand line item edits with corresponding business rule validation is much better being done in a programming language. Caching of slow changing data in app tier is another advantage. This is even better if you have dedicated middle tier service that acts as a gateway to all the data. If data is shared directly among disparate applications then stored proc may be a good idea.
You also have to factor the availability/experience of SQL talent vs programming talent in the organisation.
There is realy no general answer to this question.
In general, what kinds of design decisions help an application scale well?
(Note: Having just learned about Big O Notation, I'm looking to gather more principles of programming here. I've attempted to explain Big O Notation by answering my own question below, but I want the community to improve both this question and the answers.)
Responses so far
1) Define scaling. Do you need to scale for lots of users, traffic, objects in a virtual environment?
2) Look at your algorithms. Will the amount of work they do scale linearly with the actual amount of work - i.e. number of items to loop through, number of users, etc?
3) Look at your hardware. Is your application designed such that you can run it on multiple machines if one can't keep up?
Secondary thoughts
1) Don't optimize too much too soon - test first. Maybe bottlenecks will happen in unforseen places.
2) Maybe the need to scale will not outpace Moore's Law, and maybe upgrading hardware will be cheaper than refactoring.
The only thing I would say is write your application so that it can be deployed on a cluster from the very start. Anything above that is a premature optimisation. Your first job should be getting enough users to have a scaling problem.
Build the code as simple as you can first, then profile the system second and optimise only when there is an obvious performance problem.
Often the figures from profiling your code are counter-intuitive; the bottle-necks tend to reside in modules you didn't think would be slow. Data is king when it comes to optimisation. If you optimise the parts you think will be slow, you will often optimise the wrong things.
Ok, so you've hit on a key point in using the "big O notation". That's one dimension that can certainly bite you in the rear if you're not paying attention. There are also other dimensions at play that some folks don't see through the "big O" glasses (but if you look closer they really are).
A simple example of that dimension is a database join. There are "best practices" in constructing, say, a left inner join which will help to make the sql execute more efficiently. If you break down the relational calculus or even look at an explain plan (Oracle) you can easily see which indexes are being used in which order and if any table scans or nested operations are occurring.
The concept of profiling is also key. You have to be instrumented thoroughly and at the right granularity across all the moving parts of the architecture in order to identify and fix any inefficiencies. Say for example you're building a 3-tier, multi-threaded, MVC2 web-based application with liberal use of AJAX and client side processing along with an OR Mapper between your app and the DB. A simplistic linear single request/response flow looks like:
browser -> web server -> app server -> DB -> app server -> XSLT -> web server -> browser JS engine execution & rendering
You should have some method for measuring performance (response times, throughput measured in "stuff per unit time", etc.) in each of those distinct areas, not only at the box and OS level (CPU, memory, disk i/o, etc.), but specific to each tier's service. So on the web server you'll need to know all the counters for the web server your're using. In the app tier, you'll need that plus visibility into whatever virtual machine you're using (jvm, clr, whatever). Most OR mappers manifest inside the virtual machine, so make sure you're paying attention to all the specifics if they're visible to you at that layer. Inside the DB, you'll need to know everything that's being executed and all the specific tuning parameters for your flavor of DB. If you have big bucks, BMC Patrol is a pretty good bet for most of it (with appropriate knowledge modules (KMs)). At the cheap end, you can certainly roll your own but your mileage will vary based on your depth of expertise.
Presuming everything is synchronous (no queue-based things going on that you need to wait for), there are tons of opportunities for performance and/or scalability issues. But since your post is about scalability, let's ignore the browser except for any remote XHR calls that will invoke another request/response from the web server.
So given this problem domain, what decisions could you make to help with scalability?
Connection handling. This is also bound to session management and authentication. That has to be as clean and lightweight as possible without compromising security. The metric is maximum connections per unit time.
Session failover at each tier. Necessary or not? We assume that each tier will be a cluster of boxes horizontally under some load balancing mechanism. Load balancing is typically very lightweight, but some implementations of session failover can be heavier than desired. Also whether you're running with sticky sessions can impact your options deeper in the architecture. You also have to decide whether to tie a web server to a specific app server or not. In the .NET remoting world, it's probably easier to tether them together. If you use the Microsoft stack, it may be more scalable to do 2-tier (skip the remoting), but you have to make a substantial security tradeoff. On the java side, I've always seen it at least 3-tier. No reason to do it otherwise.
Object hierarchy. Inside the app, you need the cleanest possible, lightest weight object structure possible. Only bring the data you need when you need it. Viciously excise any unnecessary or superfluous getting of data.
OR mapper inefficiencies. There is an impedance mismatch between object design and relational design. The many-to-many construct in an RDBMS is in direct conflict with object hierarchies (person.address vs. location.resident). The more complex your data structures, the less efficient your OR mapper will be. At some point you may have to cut bait in a one-off situation and do a more...uh...primitive data access approach (Stored Procedure + Data Access Layer) in order to squeeze more performance or scalability out of a particularly ugly module. Understand the cost involved and make it a conscious decision.
XSL transforms. XML is a wonderful, normalized mechanism for data transport, but man can it be a huge performance dog! Depending on how much data you're carrying around with you and which parser you choose and how complex your structure is, you could easily paint yourself into a very dark corner with XSLT. Yes, academically it's a brilliantly clean way of doing a presentation layer, but in the real world there can be catastrophic performance issues if you don't pay particular attention to this. I've seen a system consume over 30% of transaction time just in XSLT. Not pretty if you're trying to ramp up 4x the user base without buying additional boxes.
Can you buy your way out of a scalability jam? Absolutely. I've watched it happen more times than I'd like to admit. Moore's Law (as you already mentioned) is still valid today. Have some extra cash handy just in case.
Caching is a great tool to reduce the strain on the engine (increasing speed and throughput is a handy side-effect). It comes at a cost though in terms of memory footprint and complexity in invalidating the cache when it's stale. My decision would be to start completely clean and slowly add caching only where you decide it's useful to you. Too many times the complexities are underestimated and what started out as a way to fix performance problems turns out to cause functional problems. Also, back to the data usage comment. If you're creating gigabytes worth of objects every minute, it doesn't matter if you cache or not. You'll quickly max out your memory footprint and garbage collection will ruin your day. So I guess the takeaway is to make sure you understand exactly what's going on inside your virtual machine (object creation, destruction, GCs, etc.) so that you can make the best possible decisions.
Sorry for the verbosity. Just got rolling and forgot to look up. Hope some of this touches on the spirit of your inquiry and isn't too rudimentary a conversation.
Well there's this blog called High Scalibility that contains a lot of information on this topic. Some useful stuff.
Often the most effective way to do this is by a well thought through design where scaling is a part of it.
Decide what scaling actually means for your project. Is infinite amount of users, is it being able to handle a slashdotting on a website is it development-cycles?
Use this to focus your development efforts
Jeff and Joel discuss scaling in the Stack Overflow Podcast #19.
FWIW, most systems will scale most effectively by ignoring this until it's a problem- Moore's law is still holding, and unless your traffic is growing faster than Moore's law does, it's usually cheaper to just buy a bigger box (at $2 or $3K a pop) than to pay developers.
That said, the most important place to focus is your data tier; that is the hardest part of your application to scale out, as it usually needs to be authoritative, and clustered commercial databases are very expensive- the open source variations are usually very tricky to get right.
If you think there is a high likelihood that your application will need to scale, it may be intelligent to look into systems like memcached or map reduce relatively early in your development.
One good idea is to determine how much work each additional task creates. This can depend on how the algorithm is structured.
For example, imagine you have some virtual cars in a city. At any moment, you want each car to have a map showing where all the cars are.
One way to approach this would be:
for each car {
determine my position;
for each car {
add my position to this car's map;
}
}
This seems straightforward: look at the first car's position, add it to the map of every other car. Then look at the second car's position, add it to the map of every other car. Etc.
But there is a scalability problem. When there are 2 cars, this strategy takes 4 "add my position" steps; when there are 3 cars, it takes 9 steps. For each "position update," you have to cycle through the whole list of cars - and every car needs its position updated.
Ignoring how many other things must be done to each car (for example, it may take a fixed number of steps to calculate the position of an individual car), for N cars, it takes N2 "visits to cars" to run this algorithm. This is no problem when you've got 5 cars and 25 steps. But as you add cars, you will see the system bog down. 100 cars will take 10,000 steps, and 101 cars will take 10,201 steps!
A better approach would be to undo the nesting of the for loops.
for each car {
add my position to a list;
}
for each car {
give me an updated copy of the master list;
}
With this strategy, the number of steps is a multiple of N, not of N2. So 100 cars will take 100 times the work of 1 car - NOT 10,000 times the work.
This concept is sometimes expressed in "big O notation" - the number of steps needed are "big O of N" or "big O of N2."
Note that this concept is only concerned with scalability - not optimizing the number of steps for each car. Here we don't care if it takes 5 steps or 50 steps per car - the main thing is that N cars take (X * N) steps, not (X * N2).