Performance I/O degrades after changing mysql data directory? - performance

MYSQL server is installed on windows environment : Windows server 2012 R2
PHYSICAL installed RAM : 672 GB
C : drive 1 TB and D Drive : 15 TB
We had recently database crash because of C drive full as mysql data directory was located on c drive.
Before Database crash, I/O throughput was very high and it went 1500 rows/sec
After database crash ,it went to 800 rows /sec
Nothing has changed on table, we moved Data directory of mysql to D drive and base directory to C drive, will it matters ? If it not ,then what would be cause that brought low to I/O throughput?
My.ini file config:
1. INNODB_BUFFER_POOL_SIZE=400G
2. INNODB_BUFFER_POOL_INSTANCES=24
3. INNODB_READ_IO_THREADS=64
4. INNODB_WRITE_IO_THREADS=64
5. INNODB_LRU_SCAN_depth=2500
6. INNODB IO capacity =2000
7. INNODB_io_capacity_max=4000
8. MAX_ALLOWED_PACKET=1024M
Anyone help me on this? Its been quite 25 days I'm researching on this, feel i like data directory should be placed on C drive ,but not sure on this ?

Related

Elastic search on Windows - Error - Native memory allocation (mmap) failed

We are running Elastic Search 7.13.2 on a windows server. This was running but suddenly the the ES.bat file is returning an error as below. I have verified that there is no Memory overload in the server. The system has 16 GB RAM, which is 50 percent or less occupied. The free Disk space was > 25GB. Any idea why this error comes up?
I tried to reduce the Heap memory to 4 GB, i created a file named jvm.options inside the jvm.options.d folder and added the below lines, but this doesnt seem to take effect. Any clue on whats going on?
jvm.options file content as below
-Xmx4g
-Xms4g

C drive free space drops very fast with no obvious reason

My operating system is Windows 10 and I have a problem with the free space dropping for no reason.
A couple of days ago I ran a python code in jupyter notebook, and in the middle of the execution my C drive ran out of space (there was ~50 GB free space), and since then the C drive free space changes significantly (even shrinks to few MBs) without no obvious reason.
Since then I found some huge files in a pycharm temporary directory, and I freed 47GB of space, but after a short time, it runs out of space again ( I am not even running any code anymore)!
When I restart, the free space gradually starts to increase, and again after a some time, it shrinks to a few GB or even MBs.
PS. I installed WinDirStat to show me the stat of the disk space, and it shows 93 GB under this path: C:\ProgramData\Microsoft\Search\Data\Applications\Windows\Files\Windows.edb, but I can't open Data folder in the file explorer, and it shows 0 bytes when I open the folder properties.
Windows.edb is an index database of the Windows Search function. It provides data to speed up searching in the file system due to indexing of files. There are several guides in the internet about reducing it's size. The radical way would be deleting it but I do not recomment this. You had to turn Windows Search off to do so:
net stop "Windows Search"
del %PROGRAMDATA%\Microsoft\Search\Data\Applications\Windows\Windows.edb
net start "Windows Search"
You wrote in your question that the file suddenly grew while your program was running. Maybe files will be created there. These files should be set to not be indexed. You should do that for the folder where the files are created. If this all fails, you could finally turn indexing off which slows down Windows Search.

Windirstat/TreeSizeFree showing incorrect disk space compared to Windows

Windows says the disk is at 160 GB used.
WinDirStat and TreeSizeFree say 90 GB.
I have no idea where the discrepancy is from.
I've checked this thread: https://superuser.com/questions/917110/why-does-windirstat-report-less-space-used-than-windows-does?noredirect=1&lq=1&newreg=459190bfe2a3407f8010b3e5cc070fd4.
Running as admin doesn't change anything. Also I've checked my recovery/restore options, and I've only allotted 18 GB, of which only 5.5% is being used, and windirstat/treesizefree take this into account.
I ran space sniffer as admin, and it found the hidden folder windirstat could not.
40 GB of log files in c:\windows\system32\winevt\logs. Maybe I will just check archived event logs folder next time. Its huge, and its hidden.

Redis's huge files won't delete?

I'm using Redis-server for windows ( 2.8.4 - MSOpenTech) / windows 8 64bit.
It is working great , but even after I run :
I see this : (and here are my questions)
When Redis-server.exe is up , I see 3 large files :
When Redis-server.exe is down , I see 2 large files :
Question :
— Didn't I just tell it to erase all DB ? so why are those 2/3 huge files are still there ?
How can I completely erase those files? ( without re-generating)
NB
It seems that it is doing deletion of keys without freeing occupied space. if so , How can I free this unused space?
From https://github.com/MSOpenTech/redis/issues/83
"Redis uses the fork() UNIX system API to create a point-in-time snapshot of the data store for storage to disk. This impacts several features on Redis: AOF/RDB backup, master-slave synchronization, and clustering. Windows does not have a fork-like API available, so we have had to simulate this behavior by placing the Redis heap in a memory mapped file that can be shared with a child(quasi-forked) process. By default we set the size of this file to be equal to the size of physical memory. In order to control the size of this file we have added a maxheap flag. See the Redis.Windows.conf file in msvs\setups\documentation (also included with the NuGet and Chocolatey distributions) for details on the usage of this flag. "
I know this is an old thread, but I am facing the same issues with the file sizes.
In case you have problems with your C ssd drive (like me), you can make a directory junction:
1) Stop redis service
2) Move C:\Windows\ServiceProfiles\NetworkService\AppData\Local\Redis folder to another drive / location.
3) Open a command prompt in C:\Windows\ServiceProfiles\NetworkService\AppData\Local then execute:
mklink /J "C:\Windows\ServiceProfiles\NetworkService\AppData\Local\Redis" "[newpath]"
PD: [newpath] must be absolute, like "D:\directory junctions\Redis"
4) Start redis service. Now the files are in another drive.
Check http://ss64.com/nt/mklink.html if doubts regarding this command.
I faced this same issue on my development machine. It was resolved by stopping the redis service and I used WinDirStat (which is what I used to detect the issue originally) to permanently delete these files in appdata/local/redis.
I then started redis back up and things were working fine.
Before following this same procedure others may want to ensure that this data isn't needed. In my case it wasn't critical since this is my development workstation.
When you flush the DB you only flush the keys from memory. I'm not sure why you've got files of different names, it may be an artifact of the way the Windows port of Redis manages files, but Redis itself doesn't delete files when you remove keys. You will need to manage outdated files outside of Redis.

How to use debug.exe to load a flash drive [ pendrive ] and view its boot record?

The drive letter if F, and i want to read its boot record for this task : http://joelgompert.com/OS/lesson1.htm ... [ i am using xp ]
I cant get the l command to work properly...
when they ask us to type this is debug.exe :
-l 0 0 0 1
I replaced it with :
-l 0 5 0 1
Cause i do not have a floppy at drive A...
I get a prompt telling me :
C:\WINDOWS\system32\cmd.exe - debug.exe
An application has attempted to directly access the hard disk, which cannot be supported. This may cause the application to function incorrectly. Choose 'Close' to terminate the application.
And the option to choose are Close and Ignore.
I selected ignore and it gives me the following output in the CMD :
Disk error reading drive F
What am i doing wrong ? Isn't it supported to use a flash drive ? Any alternative programs for the task ?
Any help will be appreciated :)
In all NT-based systems (Windows NT, 2000, XP/2003, Vista/2008, 7/2008 R2) direct disk access to hard drives is prohibited for older software. There are special APIs for accessing hard drivers directly, but older software simply does not know them.
DEBUG under XP does allow for direct access to floppy disks, so if you want to learn DEBUG, an USB floppy drive is an option. If you want to learn about data structures on HDD and don't care about debug, look for more recent disk editor. I think you'll find something suitable in this article at Wikipedia.

Resources