Can't copy big DB files using "copy", "robocopy" or even "eseutil" - windows

I am having a problem copying large DB files (~100GB) in an automated script I am trying to write for a Windows Server. I have tried using "copy", "robocopy", and even "eseutil".
My script is running on a Windows 2008 Server (destination of the file) and is pulling from a Windows 2003 Server (source of the file).
I have already tried changing the IRPStackSize registry setting, as well as both of the ones in the HKLM/SYSTEM/CurrentControlSet/Control/SessionManager/MemoryManagement hive. This was all done on the 2008 server and rebooted with no effect. Does anyone have a good workaround?
Copy and Robocopy both give me this:
Not enough server storage is available to process this command.
Eseutil.exe gives me this:
H:\TempSQLBackups>eseutil /y \\SRC_SERVER\SQL_BACKUPS\BIG_DB.BAK /d H:\TempSQLBackups\BIG_DB.bak
Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 08.01
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating COPY FILE mode...
Source File: \\SRC_SERVER\SQL_BACKUPS\BIG_DB.BAK
Destination File: H:\TempSQLBackups\BIG_DB.bak
Copy Progress (% complete)
0 10 20 30 40 50 60 70 80 90 100
|----|----|----|----|----|----|----|----|----|----|
........FAILURE: ReadFile: The specified network name is no longer available.
Operation terminated unsuccessfully after 11336.16 seconds.
H:\TempSQLBackups>

I figured out how to fix this!
Use a LINUX machine to samba mount the source and destination directories/drives and copy them via the network. Personally, I use rsync since it will recreate the directory structure and only copy files that aren't there or are different. Thus, you can stop and/or restart at anytime without losing your progress.
I can't believe we're still paying Microsoft for this trash of an OS. I had similar problems and there seems to be no fix other than this one. Its a little slow but not nearly as slow as doing on natively since it will fail EVERY TIME.
At one point I thought robocopy would surely do it using the /IPG:xx option (InterPacketGap in milliseconds). Nope. It just PROLONGS the stack overflow and remote console lockout. I thought, maybe, Microsoft got it right with this OS. So much for Win2K8 being solid. Ugh! Windoze is for workstations. For servers you need a server OS not tinkertoy code.

Use the XCOPY with the /J option to avoid network failures of large files. This will ONLY works in 2008 R2 and Windows 7 though. This solved my timeout issue.
Please check.

Have you tried to copy the files with the old fasion way of drag and drop?
I would do this once, to make sure its not your network failing. Make sure that works, and then try look at other solutions.
1) Make sure your destination drive, is NTFS and NOT Fat32.
2) Check when its failing to copy, is it always at the same point? ( IE if it always failing after 2gb )

Have you tried xcopy? It works better for large files and recursive copy. doc
Also, from my own experience working with network drives and command line is a pain and buggy. It is also a good idea to map the network drive and use drive letter such as z:\
xcopy /K /R /E /I /S /C /H /G /X /Y s:\*.* t:\
/K Copies attributes. Normal Xcopy will reset read-only attributes.
/R Overwrites read-only files.
/E Copies directories and subdirectories, including empty ones.
/I If destination does not exist and copying more than one file, assumes that destination must be a directory.
/S Copies directories and subdirectories except empty ones.
/C Continues copying even if errors occur.
/H Copies hidden and system files also.
/Y Suppresses prompting to confirm you want to overwrite an existing destination file.
/G Allows the copying of encrypted files to destination that does not support encryption.
/X Copies file audit settings (implies /O).

The format for the second part of eseutil should be:
/d\\server\folder\filename
Notice there is no space after the /d

Related

How to sync 2 directory content on 2 different computer during file change only

Lets say I have PC A(with folder C:\PCA) and PC B (with folder C:\PCB)
first i do a WNetAddConnection2 to map PCB folder on PC A with S drive letter
followed by a copy command cmd.exe /C copy S:*.* C:\PCA\ /Y
but this will be just copy all files in PCB to PCA everytime i call it
I'm looking for a way to Sync PCA and PCB (One way is ok) and the process should copy only files with changes(to save bandwidth).
This type of action could be performed on Linux with a tool called rsync
Maybe try take a look at this answer:
https://superuser.com/questions/69514/windows-alternative-to-rsync
for some Windows alternatives.

Only with certain users: UNC paths are not supported Defaulting to windows directory

I have a batch file, whose purpose is to copy a directory from a network location and place it in the C: location of the user's system if it doesn't already exist. The problem is when the main user attempts this, the above message is displayed and the application subsequently errors out. BUT, when I run on my machine, to try and debug, it works just as it should with no problems.
I've seen a lot out there on this, but none where it works depending on the user. Also, most are about only working with network drives and not locals.
Here is the code. I am not the author of this .bat so let me know if something doesn't look quite right.
#echo off
echo Starting Application...
rem copy files over to the users local computer to prevent .dll problems when running App
C:
CD \
if not exist "C:\App" mkdir c:\App
CD App
robocopy "\\server\shared\fuller\Apps\ThisApp" c:\App /S robocopy.log
echo You may close this window or it will close by itself when the program is done.
ThisApp1.exe
I've tried using popd and pushd in various spots, but I'm not sure where I would put those, or even if it's applicable to this situation.
Put this line as the 2nd line (right after #echo off)
pushd %~dp0 & REM needed in case 'Run as Administrator' or executed from network drive
https://ss64.com/nt/pushd.html
Note that 'Run as Administrator' changes the current directory... this will put it back to where it was.

Maria DB change datadir on windows server

I installed MariaDB on my windows server and saw it defaults the datadir where the databases are stored to C:\Program Files\MariaDB 10.2\data
I wanted it moved the the E: disk that's dedicated to database stuff.
I thought copying the datadir and then modifying the my.ini in the C:\Program Files\MariaDB 10.2\data datadir would solve it, but when I tried to restart the service it wouldn't start.
I also tried setting the folder permissions to network service but that didn't seem to help.
What do I have to do to move my datadir on windows?
After a lot of searching I stumbled across http://baazgusht.blogspot.nl/2017/02/change-datadir-of-mariadb-in-window.html
It gave me the hint I needed to move the datadir on windows.
open cmd in administrator mode(start search cmd, rightlick execute as administrator)
type Net stop MariaDB (or whatever you named your service. for me it was net stop "MariaDB 10_2_7"
CD to C:\program files\mariadb 10.2\(or wherever your mariadb is installed)
Type: xcopy data E:\MariaDB /O /X /E /H /K (E:\MariaDb is my location, you may have another location. replace E:\MariaDB with whatever you want, just use the same value below for the datadir in my.ini)
Xcopy will ask when the destination doesn't exist if it should be a file or a folder. Choose the option that represents Folder.
Type: notepad data/my.ini
change the value of datadir=C:\Pro.... to datadir=E:\MariaDB
save (ctrl + s)
in CMD type net start mariadb (or whatever you named your service. for me it was net start "MariaDB 10_2_7"
I think that real problem is this one.
MariaDB use MySQL5.5 (or MariaDB10 for other version I use) as "group user" to set full permission on data folder. But this "group user" is not create under Windows "users and groups". So you cannot add this group on security tab and set permission to another folder.
Effectively, the workaround is to xcopy the original data folder to new location preserving the original permission.
But my problem is that I recover after a crash an older data folder. So it's not clear that workaround work (I wil try it tonight). I need to set the correct user group.
A true bug fix: fix intallation of MariaDB to effectivily create the "group user" in right place not just use it in folder permission.
Additional information to #Tschallacka's answer:
The copied my.ini file should be deleted from the target directory ("E:\MariaDb" in their example), since it is not honored by MariaDB. (It still contains the old path anyways.) And the contents of the source directory ("C:\Program Files\MariaDB 10.2\data" in their example) except my.ini should be deleted too, since those files are now found at the new path.
#Tschallacka's solution works because xcopy also copies permissions of the files. Windows Explorer does not.
Explanation of the options for xcopy:
/E Copies directories and subdirectories, including empty ones.
/H Copies hidden and system files also.
/K Copies attributes. Normal Xcopy will reset read-only attributes.
There are no hidden, system or read-only files in there right after install and there shouldn't after normal use and it shouldn't matter for the copied files, but it won't hurt.
/O Copies file ownership and ACL information.
The original directory has some additional permissions set. Without those, the MariaDB service seems to not be able to access the copied files properly.
/X Copies file audit settings (implies /O).
The service seems to start fine even without this option.
After some confusion with this error I can recommend that you:
Use slashes in Windows paths, for example C:/MariaDB/DB.
Experiment with adequate permissions. This seems to work:
cacls Path_to_MariaDB_DBData /E /T /P "Network Service":F

Script to remap current network drive?

We need to disconnect and re-map a network drive on Windows 7, using a set of scripts (or an app) that runs off the same network path.
That is, I need something that loads itself into RAM before it runs, so it continues to run after the drive is disconnected.
Any ideas?
Please note that 16-bit apps are NOT supported in 64 bit systems (this explains why the Novell utility failed).
You would need a vbs file running throughout a logon session to remap drives if it's disconnected by user. Need to make this script to run when domain user logs on - e.g. Logon Script in AD or GPO. There are many ways to do it.
You could even disable "Remove Network drives" feature from Explorer GUI via GPO or Reg key (net use command still works).
Or you can tweak solution by Julius for this SO question to fit your need. But consider performance impact of the vbs - only check every n minute(s) in an infinite loop.
We do something similar. We have a batch file on the network that maps the drives a user needs. We update the batch file from time to time, and users run it from a shortcut that we've placed on their desktop:
C:\WINDOWS\system32\cmd.exe /c (#echo off&if not exist \\172.x.x.x\Login (echo Unable to access server&pause) else (md c:\TMP > NUL 2>&1 &copy \\172.x.x.x\Login\MapDrives.bat C:\TMP /y > NUL 2>&1 &call C:\TMP\MapDrives.bat&del C:\TMP\MapDrives.bat&rd c:\TMP))
You can see that it checks to see if they can access the server, and if they can, it creates a folder C:\TMP, copies the MapDrives.bat file locally, then runs it. Since it is running locally, it can remap network drives without terminating it own execution. And we can update the batch file on the server without pushing it to each user's computer.
If you don't want to create a shortcut with the long command line above, it might work to create a second batch file on the server (e.g., RunMe.bat) that users run from the server. You could place all of the code from the shortcut in the RunMe.bat and accomplish the same thing. Of course, you'd want to add one more line of code to change to the local drive (so Windows doesn't hold open a handle to the network drive). Something like this:
#echo off
C:
if not exist \\172.x.x.x\Login\MapDrives.bat (
echo Unable to access server
pause
) else (
md c:\TMP > NUL 2>&1
copy \\172.x.x.x1\Login\MapDrives.bat C:\TMP /y > NUL 2>&1
C:\TMP\MapDrives.bat
)
I kept the if not exist ... because you might place the RunMe.bat in a different location than the MapDrives.bat, so it still makes sense to verify the user can access the file. Because I didn't use call C:\TMP\MapDrives.bat, it transfers control to the local batch file and any handles to the server should be closed so the drive can be remapped. This means however, that you cannot place more commands after the C:\TMP\MapDrives.bat command.

How to run batch file from network share without "UNC path are not supported" message?

I am trying to run a batch file from a network share, but I keep getting the following message: "UNC path are not supported. Defaulting to Windows directory." The batch file is located on \\Server\Soft\WPX5\install.bat. While logged in as administrator, from my Windows 7 Desktop, I navigate to \\Server\Soft\WP15\ and double click on install.bat, that's when I get the "UNC path are not supported." message. I found some suggestions online stating that mapping drive will not work, but using a symbolic link will solve this issue, but the symbolic link didn't work for me. Below is my batch file content, I would appreciate any assistance that can help me accomplish what I am trying to do. Basically, I want to be able to run the batch file from \\Server\Soft\WP15\install.bat.
Batch file content
mklink /d %userprofile%\Desktop\WP15 \\server\soft\WP15
\\server\soft\WP15\setup.exe
robocopy.exe "\\server\soft\WP15\Custom" /copyall "C:\Program Files (x86)\WP\Custom Templates"
Regedit.exe /s \\server\soft\WPX5\Custom\Migrate.reg
Also, how do I remove the symbolic link after the install is completed?
PUSHD and POPD should help in your case.
#echo off
:: Create a temporary drive letter mapped to your UNC root location
:: and effectively CD to that location
pushd \\server\soft
:: Do your work
WP15\setup.exe
robocopy.exe "WP15\Custom" /copyall "C:\Program Files (x86)\WP\Custom Templates"
Regedit.exe /s WPX5\Custom\Migrate.reg
:: Remove the temporary drive letter and return to your original location
popd
Type PUSHD /? from the command line for more information.
I feel cls is the best answer. It hides the UNC message before anyone can see it. I combined it with a #pushd %~dp0 right after so that it would seem like opening the script and map the location in one step, thus preventing further UNC issues.
cls
#pushd %~dp0
:::::::::::::::::::
:: your script code here
:::::::::::::::::::
#popd
Notes:
pushd will change your working directory to the scripts location in the new mapped drive.
popd at the end, to clean up the mapped drive.
There's a registry setting to avoid this security check (use it at your own risks, though):
Under the registry path
HKEY_CURRENT_USER
\Software
\Microsoft
\Command Processor
add the value DisableUNCCheck REG_DWORD and set the value to 0 x 1
(Hex).
Note:
On Windows 10 version 1803, the setting seems to be located under HKLM:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Command Processor
Basically, you can't run it from a UNC path without seeing that message.
What I usually do is just put a CLS at the top of the script so I don't have to see that message. Then, specify the full path to files in the network share that you need to use.
I needed to be able to just Windows Explorer browse through the server share, then double-click launch the batch file. #dbenham led me to an easier solution for my scenario (without the popd worries):
:: Capture UNC or mapped-drive path script was launched from
set NetPath=%~dp0
:: Assumes that setup.exe is in the same UNC path
%NetPath%setup.exe
:: Note that NetPath has a trailing backslash ("\")
robocopy.exe "%NetPath%Custom" /copyall "C:\Program Files (x86)\WP\Custom Templates"
Regedit.exe /s %NetPath%..\WPX5\Custom\Migrate.reg
:: I am not sure if WPX5 was typo, so use ".." for parent directory
set NetPath=
pause
Instead of launching the batch directly from explorer - create a shortcut to the batch and set the starting directory in the properties of the shortcut to a local path like %TEMP% or something.
To delete the symbolic link, use the rmdir command.
I ran into the same issue recently working with a batch file on a network share drive in Windows 7.
Another way that worked for me was to map the server to a drive through Windows Explorer: Tools -> Map network drive. Give it a drive letter and folder path to \yourserver. Since I work with the network share often mapping to it makes it more convenient, and it resolved the “UNC path are not supported” error.
My situation is just a little different. I'm running a batch file on startup to distribute the latest version of internal business applications.
In this situation I'm using the Windows Registry Run Key with the following string
cmd /c copy \\serverName\SharedFolder\startup7.bat %USERPROFILE% & %USERPROFILE%\startup7.bat
This runs two commands on startup in the correct sequence. First copying the batch file locally to a directory the user has permission to. Then executing the same batch file. I can create a local directory c:\InternalApps and copy all of the files from the network.
This is probably too late to solve the original poster's question but it may help someone else.
This is the RegKey I used:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Command Processor]
"DisableUNCCheck"=dword:00000001
My env windows10 2019 lts version and I add this two binray data ,fix this error
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Command Processor DisableUNCCheck value 1
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Command Processor
DisableUNCCheck value 1
This is a very old thread, but I still use Windows 7. :-)
There is one point that no one seems to have taken into account, which probably would help Windows 10 users also.
If Command Extensions are enabled, the PUSHD command accepts network paths in addition to the normal drive letter and path.
So the obvious - and simplest - answer might be to enable command extensions in the batch script, if you intend to use PUSHD. At the very least, this ought to reduce the problems you might have in using PUSHD wqith a network path.
I stumbled upon this question while searching for a solution to a specific problem. I needed to make a batch script that sits in a network folder (UNC path) with a Python script. The goal was to be able to double click on the batch script and have it run the Python script:
with the network folder containing the script as the working directory,
without modifications to the Python script (no command line parameters or hard-coded paths).
without creating another Python file.
The pushd and popd solutions were unsatisfactory. They work, but if the user were to get in the habit of forcefully terminating the script while it was running, they would end up with a bunch of mapped drives in My Computer since popd wasn't run.
I start by using cls to clear the UNC path error. I then assign the path containing the batch script to a variable. I slice the path to remove the trailing backslash (otherwise, Python throws a SyntaxError). Finally, I run a couple Python commands inside the batch file that change the working directory and execute the target script:
cls
#echo off
set pyfile=myscript.py
set batchdir=%~dp0
set wdir=%batchdir:~0,-1%
python -c "import os; import runpy; os.chdir(r""%wdir%""); runpy.run_path(r""%pyfile%"")"
pause
Editing Windows registries is not worth it and not safe, use Map network drive and load the network share as if it's loaded from one of your local drives.

Resources