I read somewhere "A modern server has 144GB RAM memory", is that 144GB all used as cache?
When we talk about a server's cache, does that mean the server's memory?
It all depends on the caching method utilized by the applications that run on the sever. There are numerous caching methods, but two methods frequently used are persistent Caching and In Memory Caching.
With persistent cache, the application stores cache values somewhere intended to be “permanent”, such as the file system, database or otherwise.
Whereas, with In Memory Caching, the application uses the memory (AKA RAM, in your question 144GB) to store data. Using this method, the data is intended to be semi-permanent and not persist across reboots, application recycles, or otherwise.
If, when coding, you allocate a new object, dictionary, list or otherwise, these objects are stored in memory. Additionally, all of a servers memory is not available to the applications that run on said server. All operating systems and processes that are installed use the same RAM. Therefore, it’s common for a device that has 4GB RAM to only have 2GB reasonably usable, as the other 2GB is used by the operating system. Of course, these numbers depend on a lot of factors.
Related
I was reading about Redis and Apache ignite, both of them are in-memory cache and also act as distributed cache. I was wondering what is in-memory cache? Where is the data stored ? In the memory of local system on which an application is being used or in the memory of server where the application is hosted? How does in-memory caching works?
Example:
An application with ignite cache is running on x IP address and I am using the application on y IP address so cache will stored in memory of x IP address system or y IP address system?
Also What does it mean when we say distributed cache?
The in-memory cache can be thought of as a cache that has performance critical information/data of a database that is shared across requests in an application. There is direct access to data/memory rather than through other mechanisms that enables database related operations to operate with high efficiency inturn increasing the throughput, responsiveness of the system.
In general, in the case of distributed cache based on deployment model, the cache memory can be between database and application in a distributed manner. This cache memory can be distributed between the nodes and shall operate based on the distributed hash table and the type of data. The access of data from cache in respective nodes can in turn shall apply the in-memory cache logic to bring in performance optimization.
Here is an example of how it is achieved with Amazon Elastic Cache
As you can see, the Amazaon Elastic Cache solution has a Cache Engine running in each node which implements the caching protocol/algorithm and the Amazon Elastic Cache can support cache sizes from 6 to 67 GB in a particular node. A DNS name is assigned to each Cache Node while it is created and you need to configure the DNS names of the nodes into the client library that is being used. Once your application invokes the Put or Get requests to the cluster, the library shall algorithmically choose a particular node using the hash function that shall spread the data out across the nodes and also help in fetching the same from the nodes.
A distributed cache partitions/shards your data across multiple cluster nodes.This allows utilizing memory and CPU resources of the entire cluster, and load balance requests. A node is a process that can be running on your physical server, virtual machine, or just be a Kubernetes pod. This article might be helpful to understand the basics.
Usually, an application needs to know the IP address of at least one cluster node to open a connection. Once the connection is opened, you would work with the cluster in a way similar to relational databases - just issue your SQL requests, compute tasks and perform other operations.
Also, watch the In-Memory Computing Essentials for Software Engineers recording that covers most of your questions and introduces you to the essential capabilities of Ignite. There is free instructor-led training that is scheduled from time to time on this topic.
I'm using MemoryCache in a dotnet core C# project. I'm using it store a list of enums that I read out of a collection (I want to store it since it uses reflection to load the enums which takes time).
However, since my collection isn't going to change, I don't want it to expire at all. How do I do that? If I don't set any Expiration (SlidingExpiration or Absolute Expiration), will my cache never expire?
If you do not specify an absolute and/or sliding expiration, then the item will theoretically remain cached indefinitely. In practical terms, persistence is dependent on two factors:
Memory pressure. If the system is resource-constrained, and a running app needs additional memory, cached items are eligible to be removed from memory to free up RAM. You can however disable this by setting the cache priority for the entry to CacheItemPriority.NeverRemove.
Memory cache is process-bound. That means if you restart the server or your application restarts for whatever reason, anything stored in memory cache is gone. Additionally, this means that in web farm scenarios, each instance of your application will have it's own memory cache, since each is a separate process (even if you're simply running multiple instances on the same server).
If you care about only truly doing the operation once, and persisting the result past app shutdown and even across multiple instances of your app, you need to employ distributed caching with a backing store like SQL Server or Redis.
I have an api (Express.js driven) that doesn't do any disk operation. It only reads/writes to db. Would there be a difference if the machine runs an ssd type of disk or standard disk?
Does it influence the performance? Because I believe the require loads files only one time not every request.
One of my Railo web applications generates too many I/O requests.
Since it's hosted on an Amazon Ec2 instance, that directly affects my billing badly, because of EBS disk activity (hundreds of milions of operations).
How can I monitor I/O requests? The perfect tool would allow me to find which template/component makes intensive I/O.
I'm already using FusionReactor and that's great for profiling memory spaces and so on, but it doesn't have anything for I/O.
so you could start out by using the operating system monitoring tools to see if you have mainly reads or writes, next step is looking at memory issues despite it being an disk IO issue, maybe your servers are low on memory and thrashing the drives as they are swapping pages in and out of memory.
if you have not done so turn on the template cache this will stop railo checking the file system on every page request (provided you have the memory).
if you have plenty of memory (both for your OS and for the JVM) and you have template caching on start looking for your busy pages in fusion reactor, check for cffile, cfdirectory and other tags in these pages.... good luck.
also use of queries of queries is often a culprit in high disk io as internally a database is used which runs pages to disk on large resultsets if I remeber correctly.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
We are deploying a large scale web application that uses only redis as a data store. I notice the the benchmark of our redis master is around 8000 transactions per second on EC2, far less than the stated benchmarks on dedicated hardware.
I understand that there is a performance penalty for running Redis on a virtual machine like EC2, but I would love some pointers from people who have deployed Redis in production environments on EC2 on what EC2 setup you have found most effective for getting more out of redis.
Thanks.
EC2 is probably not the best environment to run Redis on virtualized hardware, but it is a popular one, and there are a number of points to know to get the best from Redis on this platform.
I'm one of the authors of http://redis.io/topics/benchmarks and http://redis.io/topics/latency which cover most of the topics I present below. This is just a summary of the main points.
Virtualization toll
It is not specific to EC2, but Redis is significantly slower when running on a VM (in term of maximum supported throughput). This is due to the fact for basic operations, Redis does not add much overhead to the epoll/read/write system calls required to handle client connections (like memcached, or other efficient key/value stores). System calls are typically more expensive on a VM, and they represent a significant part of Redis activity (especially in benchmarks). In that conditions, a 50% decrease in term of maximum throughput compared to bare metal is not uncommon.
Of course, it also depends on the quality of the hypervisor. For EC2, Xen is used.
Benchmarking in good conditions
Benchmarking can be tricky, especially on a platform like EC2. One point often forgotten is to ensure a proper configuration for both the benchmark client and server. For instance, do not run redis-benchmark on a CPU starved micro-instance (which will likely be throttled down by Amazon) while targeting your Redis server. Both machines are equally important to get a good maximum throughput.
Actually, to evaluate Redis performance, you need to:
run redis-benchmark locally (on the same machine than the server), assuming you have more than one vCPU core.
run redis-benchmark remotely (from a different VM), on a machine whose QoS configuration is equivalent to the server machine
So you can evaluate and compare performance of the machines and the network.
On EC2, you will have the best results with second generation M3 instances (or high-memory, or cluster compute instances) so you can benefit of HVM (hardware virtualization) instead of relying on slower para-virtualization.
The fork issue
This is not specific to EC2, but to Xen: forking a large process can be really slow on Xen (it looks better with kvm). For Redis this is a big problem if you plan to use persistence: both persistence options (RDB or AOF) require the main thread to fork and launch background save or rewrite processes.
In some cases, fork latency can freeze Redis event loop for several seconds. The more memory managed by the Redis instance, the more latency.
On EC2, be sure to use a HVM enabled instance (M3, high-memory, cluster), it will mitigate the issue.
Then, if you have large memory requirements, and your application can tolerate it, consider running several smaller Redis instances on the same machine, and shard your data. It can decrease the latency due to fork operations to an acceptable level.
Persistence configuration
This is a key point to get good performance from Redis (both on VM and bare metal). So please take the time to carefully read http://redis.io/topics/persistence
If you use RDB, keep in mind the memory copy-on-write mechanism will start duplicating pages once the save background process has been forked off. So you need to ensure there is enough memory for Redis itself, plus some margin to cope with the COW. the amount of extra memory depends on your workload. The more you write in the instance, the more extra memory you need.
Please note writing a file may also consume some memory (because of the filesystem cache), so during a Redis background save, you need to account for Redis memory, COW overhead, and size of the dump file.
The machine running the Redis server must never swap. If it does, the result will be catastrophic. Contrary to some other stores, Redis is not virtual memory friendly.
With Linux, be sure to set sensible system parameters: vm.overcommit_memory=1 and vm.swappiness=0 (or a very low value anyway). Do not use old kernel versions: they are quite bad at enforcing a low swappiness (resulting in swapping when a large file is written).
If you use AOF, review the fsync options. It is a tradeoff between raw performance and durability of the write operations. You need to make a choice and define a strategy.
You also need to get familiar with the EC2 storage options. On some VM, you have the choice between ephemeral storage and EBS. On some others, you only have EBS.
Ephemeral storage is generally faster, and you will probably get less issues than with EBS, but you can easily loose your data in case of disk failure or reboot of the host, etc ... You can imagine putting RDB snapshots on ephemeral storage, and then copying the resulting files to EBS directories, as a tradeoff between performance and robustness.
EBS is remote storage: it may eat the standard network bandwidth allocated to the VM, and impact the maximum throughput of Redis. If you plan to use EBS, consider selecting the "EBS-optimized" option to establish a QoS between the standard network and storage links.
Finally, a very common setup for performance demanding instances with EC2 is to deactivate persistence on the master, and only activate it on a slave instance. It is probably less safe for the data, but it may prevent a lot of potential latency issues on the master.