I am preparing Microservice design for one of the application where Microservice communicates to UI and that part is very straight forward and can achieve. Where other part is, one of microservice communicates to third party system using SOAP and it is SYNC communication. I want to avoid SYNC communication and build some components. That component should be responsible to fetch data from third party system and stores in local database.
So far I considered this could separate module and microservice communicates to internal module.
Now problem is
Is there any possible solution where application achieve ASYNC communication. Although these are real time request
Does it possible to store millions of transaction data in our database and then use database communication.
Related
If my company sole purpose is processing a specific payload, but there is lot of orchestration for it. Should the orchestration, be in a separate domain. Lets say, payment is what the company does, but there is a workflow service, for that payment payload? If that is in a seperate domain, how should the workflow service domain talk to payment service domain?
It's better to use Event Driven Design which powered by message services like RabitMQ (or Kafka, MSMQ, or ..). It's not recommended to speech microservice each other directly via APIs. On the other hand to aggregate, some information from multiple services you can use 2 techniques, first using a BFF (back end for frontend layer), Second use a materialized view to gather information from many services.
Planning to migrate my PHP APIs to Graphql using Apollo Federation. After a bit of research, I see it is done using the following way:
My questions are:
Is there any better way to create the federated services so it is not a separate layer (1 for each REST API)? Maybe something close to the previous schema stitching approach where all can sit in one place and be stitched together at the end (instead of a specific federated layer for each service).
If this is the recommended way, how do I deploy this infrastructure? From the diagram, does it mean I have 5 instances running to cover all of the services?
Is it recommended to run Gateway and Federated services all inside one instance (from diagram - 3 servers running in one instance)?
Let me know if it helps.
Federated services are great when you want to break the monolithic structure of the non federated implementation of apollo server. It can be designed by incorporating the micro-service best practices. Instead of blindly having one federated service per rest endpoint, you can have federated services based on the functionality the service is suppose to take care. One service can call multiple rest endpoit. This would provide you better control on scaling, securing and managing services at infrastructure level. An example can be as simple as amazon where item browsing hits will be way more than buying transactions. In this case you can have one federated service which provides browsing data where as another one can for managing transactions. Then you can scale one to multiple instance to handle user load and have additional security in place for the one hadnling transactions.
2 & 3. Yes you would need to have deploy all the components separately. I would recomend to have all the services in the same VPC cluster so that you don't have to worry about network layer security. If the services are deployed across multiple clusters, it will be adding handling firewall and https/tls for every request, which would cause unnecssaery delay becuase of network call. Although it would be in milliseconds but can be easily avoided.
I am well aware of the fact that east/west, or service to service synchronous communication between services is not the gold standard, and should only be used sparingly in a microservice architecture. However, in every real world implementation of a microservice architecture, I have seen some use-cases which require it. For example, the user service is often needs to be communicated with by other services to get up the millisecond details on the user (I'm aware that event based sharing of that data is also a possibility, but in some cases that isn't always the right approach).
My question is, what is the best way to do function to function, service to service communication in a Lambda + API Gateway style architecture?
My guess is that making an http request back out on the domain name is not ideal, since it will require going back out over the internet to resolve DNS.
Is it using the SDK to do an invoke on the downstream function directly? Will this cause issue if the downstream function depends on an API Gateway Proxy Event structure?
I'm managing a very large enterprise application in that I've implemented microservice architecture. Standalone microservices have been created based on business entities & operations.
For example,
User Operations Service
Product Operations Service
Finance Operations Service
Please note that each service implemented using an n-tier architecture with WCF. i.e have separate tiers(which is independently deployable to separate server) for business and data access.
There is a centralized database which is accessed by all the microservices. There are a couple of common entities like 'user' accessed by all the services, so we have redundant database calls in multiple services. More efforts required due to database access from many places(i.e a column rename requires deployment of all the apps)
To reduce & optimize code, I'm planning to create separate microservice and move all the database operations into it. i.e services can call "Database Operations Service" for any database operations like add/update/select.
I want to know if there are any hidden challenges that I'm not aware of. Whether should I go with this thought? What can I consider as improvements in this concept?
I'm planning to create separate microservice and move all the database operations into it
That's how you will lose all benefits from microservice architecture. One service is down — the whole application is down. Unless you have replication on several nodes.
If your app does not work if one service went down(not implying that it's that service that connects to database), then it's still bad architecture and you are not using benefits of microservice architecture.
Correct for of communication would be if service would have their own databases. Or at least that every service that wants, for example, entity User, will not fetch it from DB, but will fetch it from appropriate service. And that appropriate service could fetch it from common DB at the beginning.
Next step (improvement) in the process of accommodation to microservice architecture would be creation of separate databases for each service. And by “separate” I mean that temporal fault of one service or temporal fault of one database will allow the rest of the app to be alive and functioning.
Generally, there are no hidden challenges in your approach. It just does not give any benefits, as an intermediate form between monolith application and microservice-based.
We are evaluating a move to microservices. Each microservice would be its own project developed in isolation. During planning, we have determined that some of the microservices will communicate with other via REST calls, pub/sub, messaging (ie. a order service needs product information from product service).
If a microservice depends on retrieving data from another microservice, how can it be run in isolation during development? For example, what happens when your order service requests product details, but there is nothing to answer that request?
What you probably need is an stub rest service. Create a webapp that takes the expected output using a path that is not part of the public api. When you invoke the public api it sends what it just received
If a microservice depends on retrieving data from another microservice, how can it be run in isolation during development?
It should be always temporally isolated from other services during development and production as well.
For example, what happens when your order service requests product details, but there is nothing to answer that request?
This is a place where design flaw reveals itself: order service should not request product details from another service. Product details should be stored in the message (event) that order service will be subscribed to. Order service should be getting this message in an asynchronous manner using publish-subscribe pattern and saving it in its own database. Data about the product will be stored in 2 places as the result of that.
Please consider reading this series of articles about microservices for more details. But in a nutshell: your services should be temporally decoupled, so when your product service is down - order service can continue its operations without interruptions. This is the key thing to understand about good distributed systems design in general.