Do tools like Redis provide control over the hardware cache present in the computer or does it run on the computer RAM? If it is the latter, how can it give better performance than the existing hardware cache which is controlled by the operating system?
After a lot of scattered reading I think I have got a better idea about this. So answering the question in case someone else has this question too.
The cache in a computer is not controlled by the Operating System. It is a part of the micro architecture. No software access can 'alter' cache configuration. On a linux machine, typing vi /proc/cpuinfo will show the cache size and alignment as prescribed by the chip manufacturer.
Tools like Redis and memcached,'cache' data by persisting it in the physical memory (RAM) of a machine. It is still caused caching as this prevents the data from being written to disk and hence gives faster access.
Related
I currently have a problem, that some packages are getting dropped in my local network.
But just on one device.
Ping to local router
Here you can see a ping to my router. I only have this problem on my pc. Mobilephone and Laptops are completly fine.
I tried a network card and two Wlan usb sticks all with the same problem.
Does somebody a clue on what could cause these problems?
*OS: Windows 10 21H2
*CPU usage ideling around 4-10%
*RAM usage 40%
*Network usage 0-1%
Your question is a bit broad - there are so many things that can disturb a network connection, from physical issues (i.e. cable defects, WIFI interference), to driver problems, CPU bottlenecks, etc. That being said, I would tip at a CPU bottleneck (app using most or all of your CPU), but even that is by no means certain.
Take a look at you CPU usage with TaskManager or ProcessExplorer (from the Sysinternals package). They both also show network usage. If your machine shows excessive CPU (constantly over 30% with frequent peeks), then you might want to explorer the reasons for that, and there can be many.
Using those same tools you can also try to identify apps that are possibly using alot of network bandwidth.
Windows has much happening in the background and those processes require resources (CPU, RAM, Network, Harddisk, etc.). Should any of those resources be limited then, you can easily see issues as you describe as there is a certain interdependence between those resources, i.e. you have many apps running with limited RAM, that leads to paging and as the hard disk is slow, the CPU is then busy with data shoveling and can't keep up with the NIC requests.
However, I am theorizing here. Supply some hard data (machine config, OS Info, Network info/config, task list, CPU usage, etc.) and we can continue.
I need to optimize the app I'm working on and I can't get reliable profiling data on my development machine. The app should run on low end ARM hardware on QNX, but from logistic reasons I don't have access to the final hardware for profiling.
I've tried to do profiling on my development machine, but as you can imagine everything is so fast that I can't pin point the slow parts. I've created a Linux virtual machine with reduced memory and CPU cores count, but they are still too fast compared to the final hardware.
Is it possible to reduce the CPU clock speed/ram speed/disk speed in a virtual machine to simulate low performance hardware or is there any other way to get relevant profiling data on my development machine?
Considering the app is processing several gigabytes of data I assume disk access is a major bottleneck and limiting disk speed might help
I can use any (as in most open source and commercially available) tool/approach that runs on Windows/Linux/MacOS on real or virtual machine.
This URL describes how to limit disk bandwidth on VirtualBox images. You could run a Linux VM on Virtualbox and use this method to limit disk access speeds, turn off Disk Caching using suggestions from this answer and profile your application. Alternatively you can download QNX SDP, which comes with the option of a prebuilt x86_64 Virtual Machine image that can be run using VMWare/Virtualbox/qemu
My previous experiences with QNX on armv7 and x86_64 suggest that the devb-sdmmc driver is possibly a bottleneck when working with a lot of big files being read from flash storage. devb-sdmmc and io-blk often require fine tuning of the drivers with proper cache, block, read-ahead size and other parameters helps improve disk access performance.
I use this version on my dev environment : Redis-64 .
And I want to know if this version is suitable for the production environment?
If can use, then compared with under Linux, what need to be pay attention to?
Since version 3.0.3 the windows port developers abandoned the dlmalloc and began to use jemalloc as memory allocator. And the port was actually considered for production usage. The 3.0.500 build is approved for production by ms developers (see here).
And there is some kind of hell so how they bypassed the unix fork to save data to disk. Microsoft developers port call it point-in-time heap snapshot. And this is the most controversial part when used in production:
Redis under windows may need up to 3 times more memory than you need in linux version. This behavior is considered normal, because swap file in the windows can easily be up to 3 times larger than the actual amount of RAM.
I think this is acceptable only if the use Redis as LRU cache or not to save data to disk at all.
At least Redis under windows is absolutely susceptible if you Redis node use lot of memory. For example - we try to use Redis for windows (v2.8, v3.0.3, v3.0.5) on server with 512 gb of memory with 2 SSD drives (each 256 gb in raid 0) used as system disk. No any limits on windows swap file. Our test emulates our production - lots of writes and saves with RDB with utilization ~60-70% of memory. And here is was lots of hands up behaviours then this node try to save snapshots - memory consumption jumps, connection freeze during saving. Such behaviour never happens undex linux on same hardware.
How to get with IPMI enabled on your Dell iDRAC 5/6/7 the disk volumes and raid volumes?
At this moment we only get the status and fan speed.
We couldn't find any documentation about the disk part.
Kind regards
The Disk part and Raid Part are some thing done by OS.
IPMI is only used as a remote control for the BIOS. Essentially, Processor (Board), Display, General Peripherals and monitoring Hardware used in the server chassis.
Most of the Disk based components are either from external hardware vendors or custom OS/software based solution which is difficult to be integrated and used in IPMI/KVM's firmware.
Thus the best solution for getting raid and disk volumes is from querying into OS's information.
This could eventually become a future IPMI work.
I was wondering what could be the effect and possible advantage/disadvantage of replacing the Virtual VMWare with physical servers on performance of a web application
Unfortunately there's not nearly enough information to give any advice on this.
If you have one VMware ESX server on a high-end hardware box, converting it to a physical server will give you a minimal performance advantage.
But there are SO many variables here, your application could be going slower than it would on a physical machine for a number of VMware configuration reasons. Generally a properly configured VMware infrastructure in a production environment won't be much slower than the physical equivalent with the same allocated resources.
You need to look at where your performance is being hit - are you CPU, memory, drive, network bound? You need to understand what is slowing you down before you can even start to ask questions like this.
If you are CPU bound then can you move clients between you host boxes to even out load. If you are memory bound then can you increase the RAM assigned to the bound client. Drive bound - then you need to sort out your SAN throughput. If you are network bound then adding an additional network card to the host machine and adding an additional virtual network to spread the load can help.