How to handle large traffic sites?
Please correct me if I am wrong or is there some more methods that have to be taken care
Caching
Horizontal scaling
CDN for static links like images,css,js
Database insert should be queued
Load balancer
How to handle MySQL heavy traffic?
Related
Let's say i have this HTTP2 service, that has a list of users and this user hair color, in memory and database well.
Now i want to scale this up into multiple nodes - however i do not want the same user to be in two different servers memory - each server shall handle those specific users. This means i need to inform the load balancer where each user is being handled. In case of de-scaling, i need to inform this user is nowhere and can be routed to any server or by a given rule - IE server with less memory being used.
Would any1 know if ALB load balancer supports that ? One path i was thinking of using Query string parameter-based routing, so i could inform in the request itself something like destination_node = (int)user_id % 4 in case i had 4 nodes for instance - and this worked well in a proof of concept but that leads to a few issues:
The service itself would need to know how many instances there are to balance.
I could not guarantee even balancing, its basically a luck based balancing.
What would be the preferred approach for this, or what is a common way of solving this problem ? Does AWS ELB supports this out of the box ? I was trying to avoid having to write my own balancer, a middleware that keeps track of what services are handling what users, whose responsibility would be distributing the requests among those servers.
In AWS Application Load Balancer (ALB) it is possible to write Routing-Rules on
Host Header
HTTP Header
HTTP Request Method
Path Pattern
Query String
Source IP
But at the moment there is no way to route under dynamic conditions.
If it possible to group your data, i would prefere path pattern like
/users/blond/123
I am showing images for flipkart and snapdeal products. They are been downloaded every time a user sends request. On a decent speed internet(not very high speed like 3G/4G), the loading takes way too much time, hence bad experience for user. However, the images which are on my server,even though they are 10 times in size, they get loaded very fast. Is there a way I can optimize these external images or any other way to enhance user experience
Take a look if images are routed through your servers or directly from client, also it depends on number of request from client if it is due to request then you can load images on focus.
as of i now flipKart, snapdeal all are using cdn for faster static files with image sprite model so something at your code blocking the request or routed through your server.
I have spent a whole day understanding what stateless architecture is. I read many posts and answers like
Can My Web App Implement User Login and Remain Stateless?
Pros and Cons of Sticky Session / Session Affinity load blancing strategy?
http://www.quora.com/What-is-stateless-and-statefull-web-architecture
It seems that stateless is only transferring some user state to
somewhere else(database/memcache or client cookies).Is this right? If yes, the state is only stored somewhere else, so there must be something that is not stateless(client or server), though the load balancer now does not need to worry about which machine to route .
If above is right, if we choose transfer the user information to central place(transfser to client seems not always be the solution according to some answers) like database or memcache , we still need to find this session info for every requests. That means the place where holds the user state will have the same pressure on handling tens of millions requests at the same time. And probably, the way we find the the session information is just like sticky session(route the information request into a single node in the memcache). So why do we consider transferring the state is more scalable? The pressure is only transferred(And always, database has already had too much load)
Do I miss something or understand wrongly?
Thank you!
You are correct in that moving your state to a different layer means your application is stateful (there are very few truly stateless applications, mostly only ones doing pure math).
That doesn't mean individual layers can't be stateless, and those layers that are will scale differently than the stateful layers. The idea is that by making a particular part of the application stateless, you will be able to scale it horizontally, instead of vertically, thus able to respond to many more requests by simply buying more hardware.
You will still need to scale wherever you push that state to. So if you are are pushing it out to a database, you will need to be able to scale that database accordingly. This works well if you can push it out to a layer that can be scaled cheaply (like memcached).
It is often the goal to make your business and web layers stateless because they are generally much more expensive to scale than your data-store layers, but this isn't always true. If you have put a lot of load on your data store layer, and very little load on your application or web layers (like a data-driven vs an interaction-driven app, then you will overload your data layer.
So, like everything else, whether to make your application stateless comes down to "it depends". Generally, stateful business and web layers tend to get overloaded long before data layers do. Especially if you are doing significant OOP.
I'm working on a little search engine, where I'm trying to find out how to cache query results.
These results are simple JSON text, retrieved using an ajax request.
Storing results in memory is not an option, I can see two options remaining:
Use a nosql database to retrieve cached results.
Store results on a CDN and redirect the http request (307 - Temporary Redirect) in case the result was already cached.
However, I don't have much experience with CDN, and wonder if using it for a huge amount of temporary small text files is a good practice.
Is it a good practice to use redirection on an ajax request?
Is a CDN an appropriate solution to cache small text files?
Short answer: no.
Long: Usually, you use a CDN for large static files that you want the CDN to mirror all around the world so it's close to a user when she requests them. When you have data that changes a lot, it would always take a while to propagate the changes to all nodes of the CDN, in the meantime users get inconsistent results (this may or may not matter to you).
Also, to avoid higher latency I wouldn't use an HTTP redirect (where you tell the client to make a second request to somewhere else) but rather figure out whether to get the data from the cache or the engine on your end (e.g. using a caching proxy or a load balancer) and then serve it directly to the client.
Sorry i'm a beginner in load balancing.
In distributed environments we tend more and more to send the treatment (map/reduce) to the data so that the result gets computed locally and then aggregated.
What i'd like to do apply for partionned/distributed data, not replicated.
Following the same kind of principle, i'd like to be able to send an user request on the server where the user data is cached.
When using an embedded cache or datagrid to get low response time, when the dataset is large, we tend to avoid replication and use distributed/partitionned caches.
The partitionning algorithm are generally hash-based and permits to have replicas to handle server failures.
So finally, a user data is generally hosted on something like 3 servers (1 primary copy and 2 replicas)
On a local cache misses, the caches are generally able to search for the entry on other cache peers.
This works fine but needs a network access.
I'd like to have a load balancing strategy that avoid this useless network call.
What i'd like to know: is it possible to have a load balancer that is aware of the partitionning mecanism of the cache so that it always forwards to one of the webservers having a local copy if the data we need?
For exemple, i have a request www.mywebsite.com/user=387
The load balancer will check the 387 userId and know that this user is stored in servers 1, 6 and 12. And thus he can roundrobin to one of them or other strategy.
If there's no generic solution, are there opensource or commercial, software or hardware load balancers that permits to define custom routing strategies?
How much extracting data of a request will slow down the load balancer? What's the cost of extracting an url parameter (like in my exemple with user=387) and following some rules to go to the right webserver, compared to a roundrobin strategy for exemple?
Is there an abstraction library on top of cache vendors so that we can retrieve easily the partitionning data and make it available to the load balancer?
Thanks!
Interesting question. I don't think there is a readily available solution for your requirements, but it would be pretty easy to build if your hashing criteria is relatively simple and depends only on the request (a URL parameter as in your example).
If I were building this, I would use Varnish (http://varnish-cache.org), but you could do the same in other reverse proxies.