In Our project we are using the Blue - Green deployment. We are also using the cache in our project. I want to maintain different regions to support Blue / Green deployment with Redis cache. The idea is to support Blue - Green deployment without impacting production active instance. As we deploy to a side that is not production, the caching region should use the region created or defined for that side.
Someone have an idea how to achieve it with redis.
I have attempted this numerous different ways but IMHO Hazelcast does it right. It's built in as part of a circuit breaker so the clients will automatically switch over to the desired cluster resources.
Related
I have a huge Laravel application, it contitutes of a dashboard where users have many different complex cruds that are all saved in a database with more than 100 tables, it also have an api for mobile app that can reach a peack of 300 thousand requests per minute. As the app scales I'm having issues with performance, as all is in one single aws hosted ec2 server, by all I mean all app images, company logos etc, all the resources for the dashboard and all the api for mobile app. I need a solution for this problem, should I separate all in different machines? If so, how?
All the app is currently running PHP 7.2 and Laravel 5.5 on a aws ec2 12xlarge instance.
You are asking us some basic concept of scalability in the Cloud.
I will try to give one direction you could follow.
The current design is very bad for couple of reasons:
As you said, it cannot scale because everything hold in one server;
Because everything hold in one server, I hope you have automated backup in case your instance fails
The only thing you can do in this configuration is to scale vertically, instead of horizontally (using more instances instead of a big one)
Files are on the same disk, so you cannot scale
In term of application (Laravel), you are running with a monolith: everything in one app. I don't need to tell you that it doesn't scale well but it can.
Lets dive into to main topic: How to scale this big fat instance?
First of all, you should use a shared space for your images. There are NFS (expensive), S3 (cheap) and shared EBS (cheaper than NFS, but can only be used by a limited number of instances at a time). I would use S3.
We can skip the part where you need to refactor your monolith application to a micro-service architecture, with smaller parts. It can be done if you have time and money, but I would say it is not the priority to your scaling issue.
I don't know if the database is also on the same EBS or not. If it is, use RDS: it is an almost no management managed database. You can have multi-AZ for very high availability, or Multi-AZ DB Cluster (new) which will spread the load for reads into 2 shadow instances.
To go further with your application, you can also run mobile and web on separated instances, to avoid one impacting the other.
And...That's all! Laravel has a transparent configuration mechanism for the storage to easily switch from one to another.
When I say "That's all", I mean in term of way to improve the scaling.
You will have to migrate the data from the EC2 database to RDS, perform the transfer of your images from the EBS to S3, create an autoscaling group, create an IAM Instance role for your EC2 Autoscaling group to access S3, know when the application has peaks so you can do a predictive scaling, etc.
I would recommand using IaC for this, like CloudFormation or Terraform.
This is the tip of the iceberg, but I hope you can start building a more robust system with these tips.
I'm working on a Spring Boot application, using AbstractRoutingDatasource in order to provide multitenancy feature. New tenants can be added dynamically, and each one has it's own datasource and pool configuration, and everything is working well, since we only have around 10 tenants right now.
But I'm wondering: since the application is running on a docker container, with limit resources, as the number of tenants grows, also more and more threads will be allocated for each connection (considering a pool from 1 to 30 threads for each tenant) and the container, at some point (with 50 tenants, for example), will be killed due to memory limit defined at container startup.
It appears to me that, this multitenancy solution (using AbstractRoutingDatasource) is not suitable to an application designed to be containerized since I can't simply scale it horizontally to deal with more tenants.
Am I missing something? Should I be worried about that?
The point made in the post is about the system resource exhaustion that might arise with the increasing volume of requests as a result of increased tenants in the system. I would like to address few points
The whole infrastructure can be managed efficiently using ECS & AWS Fargate so that when there is a huge load, there are automatically new containers spun up to take the load. In case of having separate servers, ELB might be of help. There will be no issues when spinning up new containers / servers as your services are stateless
Regarding the number of active connections to a database from your application, you should profile your app and understand the DAP data access patterns. Any master data or static information should NOT be taken always from the database (Except for the 1st time), instead they should be cached. There are many managed cache services that can help you scale better.
In regards to the database, it is understood that tenants have their own databases, in case of a very large tenant, try to scale out the databases as well.
Focus on building the entire suite of features using async features in JAVA or using RxJava so that the async nature will help managing the threads.
Since you have not mentioned what cloud your applications will be deployed, I have cited sample using AWS. However most of the features can be used across Azure , GCP or AWS.
There are lot of strategies to scale, the right understanding of the business needs and data usage patterns etc... could help us decide the right approach.
Hope this clarifies.
How can I use Kuma to run a multi-cloud service mesh that spans across a VM-based environment as well as a Kubernetes-based environment?
Specifically, how will service discovery work in such a way that VM-based workloads can discover K8s-based ones and vice-versa?
Kuma defines the so-called zone as a domain of control isolation, i.e. all workload connections are managed by a single control plane. Such a control plane is called remote. The overall view and policy management is done in a global control plane, which unifies all zones.
When one starts planning a distributed deployment, they have to enlist the following items:
Where the Global control plane will be deployed and its type. The latter can be either Universal (VM/BareMetal/Container) or Kubernetes(on-premise/cloud).
Number and type of zones to add. These can be changed over time.
Follow the instructions to install the global control plane following the steps specific for the chose type of deployment. Gather the relevant IP address/ports as described.
Installing remote control plane is fairly trivial. This process can be repeated as needed during the lifetime of the whole multi-zone deployment.
Cross-zone service consumption is described in brief here. In short, we do recommend using the following syntax to access a service echo-server, deployed in a Kubernetes namespace echo-example and exposed on port 1010:
<kuma-enabled-pod>$ curl http://echo-server_echo-example_svc_1010.mesh
Using this syntax, the service can be found and consumed even from a neighbouring Universal zone where the workload runs in a VM. Kuma leverages its own DNS service, that allows for this service discovery.
It is recommended that service declared in VMs follow the same service naming format so that if needed to have a service replica in a Kubernetes cluster, they can be easily interchanged without the need to reconfigure the whole infrastructure.
I have a 3 tier application. My question relates to the spring boot rest api middle tier and mongodb backend on aws.
I am thinking of running mongodb docker container in elastic beanstalk/single container option, for scaling the backend.
My rest api will run as docker container in a separate elastic beanstalk environment.
My understanding that elastic beanstalk will scale the dockerized mongodb service as needed.
High level architecture:
Frontend - Angular - s3 static website hosting
Middle tier - 3 Spring boot rest services - 3 separate environemnts with single container docker scaled with elastic bean stalk.
Backend - Mongodb - Single Docker container scaled with elastic bean stalk.
Questions:
Qn: Will this work? Will each tier scale? Will rest service be able to connect to database? How much will this cost? Will there be too much latency between the middle tier and backend?
Qn: Is this a foolhardy chase for any reason, that has some hurdle I am not seeing? My research on this approach has yielded almost nothing. Would someone discourage from even trying this? :)
Notes:
Elastic beanstalk appears to offer convenience at a slightly higher cost. I am willing to accept it, as I am just testing. Kubernetes/docker swarm appear too complex and time consuming as I need to focus on application function in the near term.
I should be able to map a volume to a physical location in aws. Guess elastic block storage or EFS. Any pros and cons or better alternatives?
I am aware that I can use thin jars for efficiency.
I have tested it with mongodb deployed on EC2. I should be able to set it up to work launch configurations and autoscaling groups. But I think it will be more expensive, and likely more work.
I am not sure what you mean by "Single Docker container scaled with elastic bean stalk" but if you intend to launch more containers running MongoDB, the reality is a little more complicated than that.
While MongoDB does scale horizontally, when a new node is launched in a replica set topology, it:
must be added to the replica set configuration
needs to have the data that the other nodes already have synced onto it
There are tools that handle both of these requirements but simply bringing up another container is not enough.
Sharded clusters are even more complex because a node needs to be assigned into a shard so there are two levels of management decisions made.
I need to focus on application function in the near term.
You may consider MongoDB Atlas which will handle all of this for you. There is a free tier available.
I am using Cloud Foundry and I deployed my Spring boot application on Cloud. Whenever there is some updates/upgrade happens on Cloud foundry, my application got restart and some request got failed to reach to application as restart of application takes more time to get up.
Is there any way in CF that some instances of application will be running while upgrade/restart of application to process requests.
Also I want to know, if CF provides services from different locations/regions, so consider my application will be deployed on 2 CF containers available on different region. Wherever there is some updates/upgrade available, proceed upgrade on one region for Cf so other CF service from another region will be available and some application instances will be running to serve requests and vice versa.
-Thank you.
What you're describing is the intended behavior of CF.
If you have two or more instance of your application, they should never both go down at the same time. i.e. one will be taken down, then after it's restarted successfully, then the other will be taken down and restarted.
If your operator has configured multiple availability zones for the foundation that you've targeted, then application instances will be distributed across those AZs to help facilitate HA and best possible availability.
If you're not seeing this behavior then you should take a look at the following as these items can affect uptime of your apps:
Do you have more than one application instance? If you only have one application instance, then you can expect to see some small windows of downtime when updates are applied to the foundation and under other scenarios. This happens because a times Diego will need to evict applications running on a Diego Cell. It makes an attempt to start your app on another Cell before stopping the current instance, but there are no guarantees provided around this. Thus you can end up with some downtime, if for example your app is slow to start or your app does not have a good health check configured (like it passes the health check before the app is really up).
Did your operator set up multiple AZs? As a developer, you cannot really tell. This is abstracted away, so you would need to ask your platform operations team and confirm if there are more than one and if so how many. For best possible uptime, have at least as many app instances as you have AZs.
The other thing often overlooked, does your application depend on any services? If so, it is also possible that you will see downtime when services are being updated. That all depends on the services you are using and if there will be associated downtime for management and upgrades of those services. You may be able to tell if this is the case by looking more closely at your application logs when it fails to see if there are connection failures or errors like that. You might also be able to tell by looking at the plan defined in the CF Marketplace. Often the description will say if there are stipulations regarding the plan, like it is or isn't clustered or HA.
UPDATE
One other thing which can cause downtime:
If your operator has the "max in flight" value too high for the number of Diego Cells this can also cause downtime. Essentially, "max in flight" dictates how many Diego Cells will be taken out of service during an upgrade. If this value is too high, you can run into a situation where there is not enough capacity in the remaining Cells to host all of your applications. This ends up resulting in downtime for app instances as they cannot be rescheduled on another Cell in a timely manner. As a developer, I don't think this is something you can troubleshoot, you would need to work with your platform operators to investigate further.
That is probably a theme here. If you are an app developer, you should be talking to your platform operations team to debug this.
Hope that helps!