powershell - robocopy/List option not kicking back folder access denies - windows

Edit: Also, the reason I'm not using logging via Robocopy is that it doesn't seem to want to work with /MT: either. All I get is a start and finish status using robocopy logging.
Below is my current script for finding specific files on a network via robocopy. It works perfect, especially with the multithreading. However, I am searching network shares, and occasionally, I come across a directory that I do not have access to. Robocopy seems to have some ability to kick back access denied messages, but I have not been able to get them to work with the /L option. /MT: also sometimes tends to do some weird things to the output. /V (verbose) has no effect on the output when combined with /L.
Without /MT: set, I am able to slowly get all directories and files listed. However, if I am denied access to a particular directory, the output will only show the parent directory where I am initially denied.
If I enable /MT: I am still able to find files and folders, however, the log then doesn't output the top parent directory in a path where I am denied access.
Robocopy is my only option due to longpath problems with DIR and Get-childitem. It also so far has worked the fastest out of all the options due to threading.
Permissions are not changeable, and I do NOT have backup administrator rights to override ACL denies.
If anyone knows how to capture the deny/failure logs, or has a clue, I'd be super happy. I know I have denies because the bottom of each log will list how many times it failed. But if it's listing how many times it failed, surely there's a way to write each time individually to a log?
To set up an artificial fail at home, I used a standard account to run ISE, and using my admin, created a directory with some files, and then another subdirectory inside, removing all permissions for everyone for the subdirectory and files inside it, and changed the owner to system. All the files showed up, however, the subdirectory only showed with the top folder, and was unable to search any deeper.
$ErrorActionPreference = 'continue'
remove-item C:\Users\$env:username\documents\log.txt
$measure= Measure-Command -Expression{
$path = "C:\Users\$env:username\New folder"
robocopy $path null /V /L /E /FP /XJD /XJF /R:1 /W:1 /MT:8 | % {
[string] $out=$_;
if(!($out.Contains("%"))){ECHO $out | Out-File "c:\users\$env:username\documents\log.txt" -Append}
} }
write-host $measure

/MT:x Definitely messes with output in a live copy.
I suspect the errors are going through the error stream (stderr), not standard output (stdout)
First I would add /np to your options. This way you don't have to deal with the "%" later.
Then try using redirection. I don't do it very often, but I think 2>&1 > will do it. It might look something like:
robocopy $path $FakeDest /V /L /E /FP /XJD /XJF /R:1 /W:1 /MT:8 2>&1 > "c:\users\$env:username\documents\log.txt"
Also, If you wrap this in start-process there are parameters to redirect both Standard and error streams. Again not tested but it might look something like:
Start-Process -FilePath C:\Windows\System32\Robocopy.exe -ArgumentList "$path $FakeDestination /V /L /E /FP /XJD /XJF /R:1 /W:1 /MT:8" -RedirectStandardError C:\temp\Errors.txt -RedirectStandardOutput c:\temp\Output.txt
Obviously change the file names...
If you really want to get crazy there's a module for NTFS permissions. It has cmdlets like Get-ChildItem2 these are based on a 3rd part API called AlphaFS (I think). it can access paths beyond the typical 260 character limit. I'm sure it will be slower, but failing the first 2 it may be worth it. I don't know how they react to access denied though...
I'm sorry I don't have time to test any of these approaches at the moment. However, do let me know if this helps at all. I'll be around...

Related

Only with certain users: UNC paths are not supported Defaulting to windows directory

I have a batch file, whose purpose is to copy a directory from a network location and place it in the C: location of the user's system if it doesn't already exist. The problem is when the main user attempts this, the above message is displayed and the application subsequently errors out. BUT, when I run on my machine, to try and debug, it works just as it should with no problems.
I've seen a lot out there on this, but none where it works depending on the user. Also, most are about only working with network drives and not locals.
Here is the code. I am not the author of this .bat so let me know if something doesn't look quite right.
#echo off
echo Starting Application...
rem copy files over to the users local computer to prevent .dll problems when running App
C:
CD \
if not exist "C:\App" mkdir c:\App
CD App
robocopy "\\server\shared\fuller\Apps\ThisApp" c:\App /S robocopy.log
echo You may close this window or it will close by itself when the program is done.
ThisApp1.exe
I've tried using popd and pushd in various spots, but I'm not sure where I would put those, or even if it's applicable to this situation.
Put this line as the 2nd line (right after #echo off)
pushd %~dp0 & REM needed in case 'Run as Administrator' or executed from network drive
https://ss64.com/nt/pushd.html
Note that 'Run as Administrator' changes the current directory... this will put it back to where it was.

Icacls, problems with permission in subfolder of Program Files (x86)

The problem I have is very persistent.
I have a self-extracting rar file, say files.rar.
This is being uncompressed in C:\Program Files (x86)\CompanyName\XDS\bin
Now this starts a batch file, install.bat, which is in the bin folder.
A 32 bit version of cmd.exe executes this batch file.
So far so good, this all works well.
But: I have to rename one file in the bin folder and delete another. You may wonder, why put a file in there that needs to be deleted? Well, I have a 32 and 64 bit version of a driver. Depending on the target, either one of these must be renamed to NatUSB.dll, while the other must be deleted.
If I run this batch file (simplified):
icacls natusb*.dll /inheritance:r /grant:r everyone:(OI)(CI)F
icacls natusb*.dll /T
rename NatUSB_32_ezusb.dll NatUSB32.dll
del NatUSB_32_winusb.dll
I have the following problems:
I need administrator rights and that is not acceptable for the
purpose of my company
Even then I get "Access is denied" for the
renaming, possibly since there exists already a NatUSB32.dll.
However, even as administrator I can not delete this file: "Access is
denied"
The del NatUSB_32_winusb.dll also gives "Access is denied"
Then I tried to copy the two files to bin2. Reason for this: if I do "icacls bin /T" the list of files I get permission info on is too long, so I hoped to get this info just for a few files. Unfuntunately, I get "Access is denied" (while trying to copy to bin2), even while file permissions for administrator and system are "Full control".
BTW, just icacls natusb*.dll /T does not give any permissions, this only seems to work for entire folders.
When I am in the XSD (parent) folder and do "icacls bin2 /inheritance:r /grant:r everyone:(OI)(CI)F"
Next, I do "icacls bin2 /T"
bin2 Everyone:(OI)(CI)(F)
BUILTIN\Users:(OI)(CI)(F)
I still cant copy the few nat*.dll from bin to bin2.
If I try to set access rights on bin folder (very similar to above), I get "bin: The handle is invalid".
BTW, the permissions on bin are as follows:
bin NT SERVICE\TrustedInstaller:(I)(F)
NT SERVICE\TrustedInstaller:(I)(CI)(IO)(F)
NT AUTHORITY\SYSTEM:(I)(F)
NT AUTHORITY\SYSTEM:(I)(OI)(CI)(IO)(F)
BUILTIN\Administrators:(I)(F)
BUILTIN\Administrators:(I)(OI)(CI)(IO)(F)
BUILTIN\Users:(I)(RX)
BUILTIN\Users:(I)(OI)(CI)(IO)(GR,GE)
CREATOR OWNER:(I)(OI)(CI)(IO)(F)
and as stated before, with icacls I can't change any of this. BTW, the security tab doesn't show duplicate groups or names.
In short: I have problems with renaming, deleting files (must urgent, to solve current problem) and also with copying files from C:\Program Files (x86)\XXX\XDS\bin to XDS\bin2.
To solve the problems, I need to use icacls or another build in command in a script to force the access rights. Setting access rights in the security tab of a folder is not acceptable: it needs to be done from script. Also: we can not install 3rd party software, certainly not if it is large or has a commercial license. A few kB, perhaps 1MB is for a free to use tool may be acceptable if all else fails.
The solution should also not require administrator rights.
This is your main problem:
icacls natusb*.dll /inheritance:r /grant:r everyone:(OI)(CI)F
The (OI)(CI) flags are only valid for directories, not for files. Because the flags are invalid, the permission is ignored. So the net effect of this command is to remove all permissions from the file, implicitly denying access to everyone, including administrators.
If you take out the invalid flags, the command will work as expected:
icacls natusb*.dll /inheritance:r /grant:r everyone:F
However, given your description of the problem, it looks as if you will require admin access, because the permissions on the existing folder are read-only for non-administrators. There's no way around that, unless you can install into a different folder (e.g., in the user's Application Data folder) or change the destination folder permissions in advance.
Note that changing the folder permissions is likely to introduce a security vulnerability.

How to run batch file from network share without "UNC path are not supported" message?

I am trying to run a batch file from a network share, but I keep getting the following message: "UNC path are not supported. Defaulting to Windows directory." The batch file is located on \\Server\Soft\WPX5\install.bat. While logged in as administrator, from my Windows 7 Desktop, I navigate to \\Server\Soft\WP15\ and double click on install.bat, that's when I get the "UNC path are not supported." message. I found some suggestions online stating that mapping drive will not work, but using a symbolic link will solve this issue, but the symbolic link didn't work for me. Below is my batch file content, I would appreciate any assistance that can help me accomplish what I am trying to do. Basically, I want to be able to run the batch file from \\Server\Soft\WP15\install.bat.
Batch file content
mklink /d %userprofile%\Desktop\WP15 \\server\soft\WP15
\\server\soft\WP15\setup.exe
robocopy.exe "\\server\soft\WP15\Custom" /copyall "C:\Program Files (x86)\WP\Custom Templates"
Regedit.exe /s \\server\soft\WPX5\Custom\Migrate.reg
Also, how do I remove the symbolic link after the install is completed?
PUSHD and POPD should help in your case.
#echo off
:: Create a temporary drive letter mapped to your UNC root location
:: and effectively CD to that location
pushd \\server\soft
:: Do your work
WP15\setup.exe
robocopy.exe "WP15\Custom" /copyall "C:\Program Files (x86)\WP\Custom Templates"
Regedit.exe /s WPX5\Custom\Migrate.reg
:: Remove the temporary drive letter and return to your original location
popd
Type PUSHD /? from the command line for more information.
I feel cls is the best answer. It hides the UNC message before anyone can see it. I combined it with a #pushd %~dp0 right after so that it would seem like opening the script and map the location in one step, thus preventing further UNC issues.
cls
#pushd %~dp0
:::::::::::::::::::
:: your script code here
:::::::::::::::::::
#popd
Notes:
pushd will change your working directory to the scripts location in the new mapped drive.
popd at the end, to clean up the mapped drive.
There's a registry setting to avoid this security check (use it at your own risks, though):
Under the registry path
HKEY_CURRENT_USER
\Software
\Microsoft
\Command Processor
add the value DisableUNCCheck REG_DWORD and set the value to 0 x 1
(Hex).
Note:
On Windows 10 version 1803, the setting seems to be located under HKLM:
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Command Processor
Basically, you can't run it from a UNC path without seeing that message.
What I usually do is just put a CLS at the top of the script so I don't have to see that message. Then, specify the full path to files in the network share that you need to use.
I needed to be able to just Windows Explorer browse through the server share, then double-click launch the batch file. #dbenham led me to an easier solution for my scenario (without the popd worries):
:: Capture UNC or mapped-drive path script was launched from
set NetPath=%~dp0
:: Assumes that setup.exe is in the same UNC path
%NetPath%setup.exe
:: Note that NetPath has a trailing backslash ("\")
robocopy.exe "%NetPath%Custom" /copyall "C:\Program Files (x86)\WP\Custom Templates"
Regedit.exe /s %NetPath%..\WPX5\Custom\Migrate.reg
:: I am not sure if WPX5 was typo, so use ".." for parent directory
set NetPath=
pause
Instead of launching the batch directly from explorer - create a shortcut to the batch and set the starting directory in the properties of the shortcut to a local path like %TEMP% or something.
To delete the symbolic link, use the rmdir command.
I ran into the same issue recently working with a batch file on a network share drive in Windows 7.
Another way that worked for me was to map the server to a drive through Windows Explorer: Tools -> Map network drive. Give it a drive letter and folder path to \yourserver. Since I work with the network share often mapping to it makes it more convenient, and it resolved the “UNC path are not supported” error.
My situation is just a little different. I'm running a batch file on startup to distribute the latest version of internal business applications.
In this situation I'm using the Windows Registry Run Key with the following string
cmd /c copy \\serverName\SharedFolder\startup7.bat %USERPROFILE% & %USERPROFILE%\startup7.bat
This runs two commands on startup in the correct sequence. First copying the batch file locally to a directory the user has permission to. Then executing the same batch file. I can create a local directory c:\InternalApps and copy all of the files from the network.
This is probably too late to solve the original poster's question but it may help someone else.
This is the RegKey I used:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Command Processor]
"DisableUNCCheck"=dword:00000001
My env windows10 2019 lts version and I add this two binray data ,fix this error
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Command Processor DisableUNCCheck value 1
Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Command Processor
DisableUNCCheck value 1
This is a very old thread, but I still use Windows 7. :-)
There is one point that no one seems to have taken into account, which probably would help Windows 10 users also.
If Command Extensions are enabled, the PUSHD command accepts network paths in addition to the normal drive letter and path.
So the obvious - and simplest - answer might be to enable command extensions in the batch script, if you intend to use PUSHD. At the very least, this ought to reduce the problems you might have in using PUSHD wqith a network path.
I stumbled upon this question while searching for a solution to a specific problem. I needed to make a batch script that sits in a network folder (UNC path) with a Python script. The goal was to be able to double click on the batch script and have it run the Python script:
with the network folder containing the script as the working directory,
without modifications to the Python script (no command line parameters or hard-coded paths).
without creating another Python file.
The pushd and popd solutions were unsatisfactory. They work, but if the user were to get in the habit of forcefully terminating the script while it was running, they would end up with a bunch of mapped drives in My Computer since popd wasn't run.
I start by using cls to clear the UNC path error. I then assign the path containing the batch script to a variable. I slice the path to remove the trailing backslash (otherwise, Python throws a SyntaxError). Finally, I run a couple Python commands inside the batch file that change the working directory and execute the target script:
cls
#echo off
set pyfile=myscript.py
set batchdir=%~dp0
set wdir=%batchdir:~0,-1%
python -c "import os; import runpy; os.chdir(r""%wdir%""); runpy.run_path(r""%pyfile%"")"
pause
Editing Windows registries is not worth it and not safe, use Map network drive and load the network share as if it's loaded from one of your local drives.

How to delete a folder that name ended with a dot (".")?

I got some folders created by malware whose name ended with a dot like C:\a.\ or C:\b.\, etc.
I found a solution that can remove such folder with command rd /q /s "C:\a.\" but if I call win API RemoveDirectory, it returns ERROR_FILE_NOT_FOUND.
And I just wonder how to write a function to delete such directory, thanks
I test on my own Windows XP SP3 system like this
create a folder C:\>mkdir a..\\\ and I cannot double click to access this folder. and I can remove with command rd /q /s "C:\a.\"
what Windows system API(s) that rd /q /s command call?
Here's a solution to this problem:
rd /s "\\?\C:\Documents and Settings\User\Desktop\Annoying Folder."
Solution:
When you call RemoveDirectory, make sure that you prefix the path with the string "\\?\".
Explanation:
It has everything to do with the dot. According to MSDN, there are certain cases where you may not be able to delete a file or folder on an NTFS volume, specifically when the file name is invalid in the Win32 name space (which is why you are unable to open the file using the normal methods in Windows Explorer).
You may not be able to delete a file if the file name includes an invalid name (for example, the file name has a trailing space or a trailing period or the file name is made up of a space only). To resolve this issue, use a tool that uses the appropriate internal syntax to delete the file. You can use the "\\?\" syntax with some tools to operate on these files, for example:
del "\\?\c:\path_to_file_that contains a trailing space.txt "
The cause of this issue is similar to Cause 4. However, if you use typical Win32 syntax to open a file that has trailing spaces or trailing periods in its name, the trailing spaces or periods are stripped before the actual file is opened. Therefore, if you have two files in the same folder named "AFile.txt" and "AFile.txt " (note the space after the file name), if you try to open the second file by using standard Win32 calls, you open the first file instead. Similarly, if you have a file whose name is just " " (a space character) and you try to open it by using standard Win32 calls, you open the file's parent folder instead. In this situation, if you try to change security settings on these files, you either may not be able to do this or you may unexpectedly change the settings on different files. If this behavior occurs, you may think that you have permission to a file that actually has a restrictive ACL.
(Source: http://support.microsoft.com/?kbid=320081)
Ive posted this on SU and I decided to post it here too. Its the simplest and fastest and easiest way to achieve this. I am now laughing at how much simple it is.
Install WinRAR
Follow the Step by Step procedure from pictures:
I myself had WinRaR installed so I decided to demonstrate the workaround in it.
This workaround is also possible by using 7zip.
One another thing I should mention is that, as it seems the problem is caused by using windows explorer and any other file browser (like winrar file browser itself, ftp explorers etc.) will treat this files as normal.
You could try using any file browser and simply delete those files and not bother archiving them though!
Cheers!
If you have git installed (you can get ir from here) then it is as simple as:
Navigate File Explorer to location where problematic folder is located.
Context menu (right mouse button) > Git Bash Here.
rm -rf Foldername./
When you see the name is "a.", but the actual name is "a.."
Try this:
rd /q /s "C:\a..\"
And you can try explore the folder by this code:
for /f "tokens=3 delims=<>" %%a in ('dir /ad /x "C:\*" ^| findstr " a\.\.$"') do (
for /f "tokens=1" %%b in ("%%a") do start "" "%%~fb"
)
I used "WinRar" A simple RAR, ZIP processor. You can use any sort of file name editor. Just open the directory where your file is into WinRar and select rename after right clicking the file/folder you want to rename and fill in the new name.
If you need to keep the data you can also use the \\?\ trick for renaming the folder.
ren "\\?\C:\Documents and Settings\User\Desktop\Annoying Folder." "\\?\C:\Documents and Settings\User\Desktop\Annoying Folder"
This is an ideal solution if you need to know what is inside the folder or if the data is important.
This works in both Command Prompt and PowerShell.
Try to use unlocker program to delete files and folders that you can't delete normally.
if you want to keep the files theres options in bash as well.
you will require the Windows Subsystem for Linux package (i have Ubuntu installed)
to keep the files. open a command prompt and cd over to where the file or folder is located.
now type "bash"
this will open bash in the prompt. now enter mv '[folder or file you want to move]' '[new name (can include path)]' (theres more to mv so if you want to read up on all of its options use 'man mv' this will open its manual page (then use q to return to bash))
the mv command is short for move, but its has a secondary function of renaming things.
also in bash use 'single quotes' and not a normal "double quote", as bash expects 'single quotes'.
heres a example. assume your folder is named "data 1." located in c:\users (so the full path to the error folder is c:\users\data 1.
1. open command prompt using any method
2. enter cd c:\users
3. now type bash this loads bash in the folder you previously were in
4. finally type mv 'data 1.' 'data 1'
5. the folder is now accessible and you can choose to delete it.
Use bash rm command from Ubuntu on Windows 10

Can't copy big DB files using "copy", "robocopy" or even "eseutil"

I am having a problem copying large DB files (~100GB) in an automated script I am trying to write for a Windows Server. I have tried using "copy", "robocopy", and even "eseutil".
My script is running on a Windows 2008 Server (destination of the file) and is pulling from a Windows 2003 Server (source of the file).
I have already tried changing the IRPStackSize registry setting, as well as both of the ones in the HKLM/SYSTEM/CurrentControlSet/Control/SessionManager/MemoryManagement hive. This was all done on the 2008 server and rebooted with no effect. Does anyone have a good workaround?
Copy and Robocopy both give me this:
Not enough server storage is available to process this command.
Eseutil.exe gives me this:
H:\TempSQLBackups>eseutil /y \\SRC_SERVER\SQL_BACKUPS\BIG_DB.BAK /d H:\TempSQLBackups\BIG_DB.bak
Extensible Storage Engine Utilities for Microsoft(R) Exchange Server
Version 08.01
Copyright (C) Microsoft Corporation. All Rights Reserved.
Initiating COPY FILE mode...
Source File: \\SRC_SERVER\SQL_BACKUPS\BIG_DB.BAK
Destination File: H:\TempSQLBackups\BIG_DB.bak
Copy Progress (% complete)
0 10 20 30 40 50 60 70 80 90 100
|----|----|----|----|----|----|----|----|----|----|
........FAILURE: ReadFile: The specified network name is no longer available.
Operation terminated unsuccessfully after 11336.16 seconds.
H:\TempSQLBackups>
I figured out how to fix this!
Use a LINUX machine to samba mount the source and destination directories/drives and copy them via the network. Personally, I use rsync since it will recreate the directory structure and only copy files that aren't there or are different. Thus, you can stop and/or restart at anytime without losing your progress.
I can't believe we're still paying Microsoft for this trash of an OS. I had similar problems and there seems to be no fix other than this one. Its a little slow but not nearly as slow as doing on natively since it will fail EVERY TIME.
At one point I thought robocopy would surely do it using the /IPG:xx option (InterPacketGap in milliseconds). Nope. It just PROLONGS the stack overflow and remote console lockout. I thought, maybe, Microsoft got it right with this OS. So much for Win2K8 being solid. Ugh! Windoze is for workstations. For servers you need a server OS not tinkertoy code.
Use the XCOPY with the /J option to avoid network failures of large files. This will ONLY works in 2008 R2 and Windows 7 though. This solved my timeout issue.
Please check.
Have you tried to copy the files with the old fasion way of drag and drop?
I would do this once, to make sure its not your network failing. Make sure that works, and then try look at other solutions.
1) Make sure your destination drive, is NTFS and NOT Fat32.
2) Check when its failing to copy, is it always at the same point? ( IE if it always failing after 2gb )
Have you tried xcopy? It works better for large files and recursive copy. doc
Also, from my own experience working with network drives and command line is a pain and buggy. It is also a good idea to map the network drive and use drive letter such as z:\
xcopy /K /R /E /I /S /C /H /G /X /Y s:\*.* t:\
/K Copies attributes. Normal Xcopy will reset read-only attributes.
/R Overwrites read-only files.
/E Copies directories and subdirectories, including empty ones.
/I If destination does not exist and copying more than one file, assumes that destination must be a directory.
/S Copies directories and subdirectories except empty ones.
/C Continues copying even if errors occur.
/H Copies hidden and system files also.
/Y Suppresses prompting to confirm you want to overwrite an existing destination file.
/G Allows the copying of encrypted files to destination that does not support encryption.
/X Copies file audit settings (implies /O).
The format for the second part of eseutil should be:
/d\\server\folder\filename
Notice there is no space after the /d

Resources