I want to reduce the load of one of the services with horizontal scaling. In service-oriented architecture, do I have to copy or move(whatever you say) all services to the server I will add, since each of the services is a separate project?
Can I carry only the service I want?
In fact, ability to scale services independently is one of major advantages with micro-services. Typically, as time goes, a service would get more and more customers (other services in this case). So the service should be scalable to support increased load.
More specific: a service A is called by two other services, B and C. And A is scaled properly to handle that load. At some point, there is a new service D to start calling A. In that case, the team owning service A will do capacity calculations and will scale out their service. The interesting aspect is, that neither service B nor C will even know that service A has scaled out.
In your question you have this statement: "reduce the load of one of the services with horizontal scaling". Technically, if you horizontally scale out a service, the total load will stay the same, but each individual instance of the service (each server/container/etc) will need to support less load. This is a common case to balance throughput and latency in a service.
As for "do I have to copy or move(whatever you say) all services to the server I will add, since each of the services is a separate project" - no, you definitely should scale out only services on as needed basis.
To clarify what is a service (one of different definitions): a service is a logical component of your architecture, service executes specific functions.
The implementations of service very largely: e.g. some services is just a single service with a singe process, some a set of boxes/containers behind a load balancer, and some services a sets of load balancers and servers to support regionalization.
It is not uncommon to have several services/processes/apps being deployed to the same physical service. And then these services are moved around as needed.
Related
Can micro-service interact with downstream service through localhost origin, since my all service is running in same server is that is correct approach ? I found that interacting a downstream service with domain name takes much time when compared to localhost. i was curious to know whether we can do like this ?
You're right, you can communicate with other services running in the same host with localhost. It's completely fine and when thinking about network round trips, it's beneficial.
But,
what if you want to scale the services?
What if you want to move any of the services to a different host?
While considering at least these scenarios, binding to a specific host is not worth. And this is applicable if you are using the IP of the host.
*I found that interacting a downstream service with domain name takes much time when compared to localhost.*.
I see what you're saying.
Microservices architecture is not a silver bullet for software development design and always come with tradeoffs
And about your deployment strategy Multiple Service Instances per Host pattern.
How you are going to handle if your services have different resource requirements?
say what if one of your services is utilizing all the host resource?
What if you need to scale out one independent service?
How you are going to ensure the availabilities of your services?
..
..
So there are so many questions you must consider before going with a pattern in microservices. It all depends on your requirements.
If your services are on the same server you should using a message broker or mechanism like grcp to talk between your services so doesn't matter if your orgin is. If you are using HTTP to communicate between your micro services then it totally not gain any advantages of micro services architecture and your architecture is flawed.
Microservice is a concept, it does not force you to where you deploy your application and how they might call each other. You may deploy your microservices on different virtual machines that are hosted on the same physical server. The whole point is you need to have a reason for everything that you decide to do with your architecture.
The first question is why you have split your application into different microservices? for only carrying the word of microservice on your architecture or having better control on the business logic, scalability, and maintainability of the project?
These are important things you need to take care of them when you are designing an application. draw the big picture of your product, how it's going to be used. which service/component is mostly being used by the customers, does keeping it with other microservices on the same server makes performance issues or not? what if any issue happens to the server and whole applications would be unreachable.
I have a micro-service project with multiple services in .NET Core. When it comes to placing the controllers, there are 2 approaches:
Place the controllers in respective Micro Services, with Startup.cs in each micro-service.
Place all controllers in a separate project and have them call the individual services.
I think the 1st approach will involve less coding effort but the 2nd one separates controllers from actual services using interfaces etc.
Is there a difference in terms of how they are created and managed in Fabric using both approaches.
This is very broad topic and can raise points for discussion because it all depends on preferences, experiences and tech stacks. I will add my two cents, but do not consider it as a rule, just my view for both approaches.
First approach (APIs for each service isolated from each other):
the services will expose their public APIs themselves and you will need to put a service discovery approach in place to enable clients to call each microservice, a simple one is using the reverse proxy to forward the calls using the service name.
Each service and it's APIs scales independently
This approach is better to deploy individual updates without taking down other microservices.
This approach tends to have more code repetition to handle authorization, authentication, and other common aspects, from there you will end up doing shared libraries using on all services.
This approach increase the points of failures, it is good because failures will affect less services, if one API is failing, other services won't be impacted (if the failure does not affect the machine like memory leak or high CPU usage).
The second approach (Single API to forward the calls to right services):
You have a single endpoint and the service discovery will happen in the API, all work will be handled by each services.
The API must scale for everyone even though one service consumes much more resources than others. just the service will scale independently.
This approach, to add or modify api endpoints, you will likely update the API and the service, taking down the API will affect other services.
This approach reduces the code duplication and you can centralize many common aspects like Authorization, request throttling and so on.
This approach has less points of failures, if one microservices goes down, and a good amount of calls depend on this service, the API will handle more connection and pending requests, this will affect other services and performance. If it goes down, every services will be unavailable. Compared to the first approach, the first approach will offloaded the resilience to the proxy or to the client.
In summary,
both approaches will have a similar effort, the difference is that the effort will be split into different areas, you should evaluate both and consider which one to maintain. Don't consider just code in the comparison, because code has very little impact on the overall solution when compared with other aspects like release, monitoring, logging, security, performance.
In our current project we have a public facing API. We have several individual microservice projects for each domain. Being individual allows us to scale according to the resources each microservice use. For example we have an imaging service that consumes a lot of resources, so scaling this is easier. You also have the chance to deploy them individually and if any service fails it doesn't break the whole application.
In front of all the microservices we have an API Gateway that handles all the authentication, throttles, versioning, health checks, metrics, logging etc. We have interfaces for each microservice, and keep the Request and Response models seperately for each context. There is no business logic on this layer, and you also have the chance to aggregate responses where several services need to be called.
If you would like to ask anything about this structure please feel free to ask.
I'm designing a RESTful Service Oriented Architecture web application to make it scale as good as possible and put different kind of services on different machines (separating resource intensive operations from other services).
I also want users to be able to access their data to make their own applications.
I'm not sure if I have to design these services to be opened to the world, so it's just a matter of make them listen on a web domain (like AWS) or create another service to handle API requests.
It makes sense to me to have secure opened webservices, but it does add a lot of complexity to the architecture itself because each service becomes a client that has to be recognized (trust) by other services in the same suite, just as well as I have to recognize 3rd party applications trying to access their own data.
Is this a right SOA approach? What I want to be sure is that I'm not mixing wrong concepts designing a wrong service oriented architecture.
All services have crud interfaces so they could be queried using REST principles.
Depending on the nature of your system, it may be viable to have unsecured webservices, so they can all talk to each other without the security overheads. To make the services available to 3rd parties, you could then use a Service Perimeter Guard as the only mechanism for accessing the services externally and apply security at this layer. This has the benefit of providing consistent security across all of your services, however if the perimeter is compromised then access to all of the services is obtained.
This approach may not be viable for all services. For instance information that is considered "personal-in-confidence" (e.g., employee data such as home addresses, emergency contact details, health data, etc), will need to be secured so that unauthorised staff cannot access it.
Regarding your comment of putting different services on different machines, this will result in under-utilised resources on some machines and possibly over-utilised resources on others. To avoid this, deploy all services to all machines and use a load-balancer. This will provide more optimal resource usage and simplify deployments (e.g., using Chef or Puppet) as all of the nodes are the same. As the resource usage increases, you can then simply add more nodes. Similarly if the resource usage is low, you can remove nodes.
Regarding your last sentence, there is a whole lot more to REST than CRUD (such as HATEOAS).
I am writing a Java EE application which is supposed to consume SAP BAPIs/RFC using JCo and expose them as web-services to other downstream systems. The application needs to scale to huge volumes in scale of tens of thousands and thousands of simultaneous users.
I would like to have suggestions on how to design this application so that it can meet the required volume.
Its good that you are thinking of scalability right from the design phase. Martin Abbott and Michael Fisher (PayPal/eBay fame) layout a framework called AKF Scale for scaling web apps. The main principle is to scale your app in 3 axis.
X-axis: Cloning of services/ data such that work can be easily distributed across instances. For a web app, this implies ability to add more web servers (clustering).
Y-axis: separation of work responsibility, action or data. So for example in your case, you could have different API calls on different servers.
Z-Axis: separation of work by customer or requester. In your case you could say, requesters from region 1 will access Server 1, requesters from region 2 will access Server 2, etc.
Design your system so that you can follow all 3 above if you need to. But when you initially deploy, you may not need to use all three methods.
You can checkout the book "The Art of Scalability" by the above authors. http://amzn.to/oSQGHb
A final answer is not possible, but based on the information you provided this does not seem to be a problem as long as your application is stateless so that it only forwards requests to SAP and returns the responses. In this case it does not maintain any state at all. If it comes to e.g. asynchronous message handling, temporary database storage or session state management it becomes more complex. If this is true and there is no need to maintain state you can easily scale-out your application to dozens of application servers without changing your application architecture.
In my experience this is not necessarily the case when it comes to SAP integration, think of a shopping cart you want to fill based on products available in SAP. You may want to maintain this cart in your application and only submit the final cart to SAP. Otherwise you end up building an e-commerce application inside your backend.
Most important is that you reduce CPU utilization in your application to avoid a 'too-large' cluster and to reduce all kinds of I/O wherever possible, e.g. small SOAP messages to reduce network I/O.
Furthermore, I recommend to design a proper abstraction layer on top of JCo including the JCO.PoolManager for connection pooling. You may also need a well-thought-out authorization concept if you work with a connection pool managed by only one technical user.
Just some (not well structured) thoughts...
I'm porting a huge application to Windows Azure. It will have a web service frontend and a processing backend. So far I thought I would use web roles for servicing client requests and worker roles for backend processing.
Managing two kinds of roles seems problematic - I'll need to decide how to scale two kinds of roles and also I'll need several (at least two) instances of each to ensure reasonable fault tolerance and this will slightly increase operational costs. Also in my application client requests are rather lightweight and backend processing is heavyweight, so I'd expect that backend processing would consume far more processing power than servicing client requests.
This is why I'm thinking of using web roles for everything - just spawn threads and do both servicing requests and backend processing in each instance. This will make the role more complicated but will I guess simplify management. I'll have more instances of a uniform role and better fault tolerance.
Is it a good idea to reuse web roles for backend processing? What drawbacks should I expect?
Sounds like you already have a pretty good idea of what to think about when using multiple roles:
Cost for 2 instances to meet SLA (although some background tasks really don't need SLA if the end user doesn't see the impact)
Separate scale units
However: If you run everything in one role, then everything scales together. If, say, you have an administrative web site on port 8000, you might have difficulty reaching it if your user base is slamming the main site on port 80 with traffic.
I blogged about combining web and worker roles, here, which goes into a bit more detail along what we're discussing here. Also, as of some time in March, the restriction of 5 endpoints per role was lifted - see my blog post here for just how far you can push endpoints now. Having this less-restrictive endpoint model really opens up new possibilities for single-role deployments.
From what I understand your are asking if it makes sense to consolidate service layers so that you only have to deal with a single layer. At a high level, I think that makes sense. The simpler the better, as long as it's not so simple that you can't meet your primary objectives.
If your primary objective is performance, and the calls to your services are inline (meaning that the caller is waiting for an answer), then consolidating the layers may help you in achieving greater performance because you won't have to deal with the overhead of additional network latency of additional physical layers. You can use the Task Parallel Library (TPL) to implement your threading logic.
If your primary objective is scalability, and the calls to your services are out-of-band (meaning that the caller implements a fire-and-forget pattern), then using processing queues and worker roles may make more sense. One of the tenets of cloud computing is loosely coupled services. While you have more maintenance work, you also have more flexibility to grow your layers independendly. Your worker roles could also use the TPL mentioned above so that you can deploy your worker roles on larger VMs (say with 4CPUs, or 8), which would keep the number of instances deployed to a minimum.
My 2 cents. :)
I would suggest you to develop them as separated roles: a web role and a worker role, and then just combine them into a single web role.
this way, in the future you can easaly convert to real separated roles, if needed.
for more details:
http://www.31a2ba2a-b718-11dc-8314-0800200c9a66.com/2010/12/how-to-combine-worker-and-web-role-in.html
http://wely-lau.net/2011/02/25/combining-web-and-worker-role-by-utilizing-worker-role-concept/