Cannot remove item, The Directory is not empty - windows

Question

When using the Remove-Item command, even utilizing the -r and -Force parameters, sometimes the following error message is returned:
Remove-Item : Cannot remove item C:\Test Folder\Test Folder\Target: The directory is not empty.
Particularly, this happens when the directory to be removed is opened in Windows Explorer.
Now, while it is possible to avoid this simply by having Windows Explorer closed or not browsing that location, I work my scripts in a multi-user environment where people sometimes just forget to close Windows Explorer windows, I am interested in a solution for deleting entire folders and directories even if they are opened in Windows Explorer.
Is there an option more powerful than -Force that I can set to achieve this?
To reliably reproduce this, create the folder C:\Test Folder\Origin and populate it with some files and subfolders (important), then take the following script or one like it and execute it once. Now open one of the subfolders of C:\Test Folder\Target (in my case, I used C:\Test Folder\Target\Another Subfolder containing A third file.txt), and try running the script again. You will now get the error. If you run the script a third time, you will not get the error again (depending on circumstances that I have yet to determine, though, the error sometimes occurs the second time and then never again, and at other times it occurs every second time).
$SourcePath = "C:\Test Folder\Origin"
$TargetPath = "C:\Test Folder\Target"
if (Test-Path $TargetPath) {
Remove-Item -r $TargetPath -Force
}
New-Item -ItemType directory -Path $TargetPath
Copy-Item $SourcePath -Destination $TargetPath -Force -Recurse -Container

Answer 1

Update: Starting with (at least [1]) Windows 10 version 20H2 (I don't know that Windows Server version and build that corresponds to; run winver.exe to check your version and build), the DeleteFile Windows API function now exhibits synchronous behavior on supported file-systems, including NTFS, which implicitly solves the problems with PowerShell's Remove-Item and .NET's System.IO.File.Delete / System.IO.Directory.Delete (but, curiously, not with cmd.exe's rd /s).
This is ultimately only a timing issue: the last handle to a subdirectory may not be closed yet at the time an attempt is made to the delete the parent directory - and this is a fundamental problem, not restricted to having File Explorer windows open:
Incredibly, the Windows file and directory removal API is asynchronous: that is, by the time the function call returns, it is not guaranteed that removal has completed yet.
Regrettably, Remove-Item fails to account for that - and neither do cmd.exe's rd /s and .NET's [System.IO.Directory]::Delete() - see this answer for details.
This results in intermittent, unpredictable failures.
The workaround comes courtesy of in this YouTube video (starts at 7:35), a PowerShell implementation of which is below:
Synchronous directory-removal function Remove-FileSystemItem:
Important:
The synchronous custom implementation is only required on Windows, because the file-removal system calls on Unix-like platforms are synchronous to begin with. Therefore, the function simply defers to Remove-Item on Unix-like platforms. On Windows, the custom implementation:
requires that the parent directory of a directory being removed be writable for the synchronous custom implementation to work.
is also applied when deleting directories on any network drives.
What will NOT prevent reliable removal:
File Explorer, at least on Windows 10, does not lock directories it displays, so it won't prevent removal.
PowerShell doesn't lock directories either, so having another PowerShell window whose current location is the target directory or one of its subdirectories won't prevent removal (by contrast, cmd.exe does lock - see below).
Files opened with FILE_SHARE_DELETE / [System.IO.FileShare]::Delete (which is rare) in the target directory's subtree also won't prevent removal, though they do live on under a temporary name in the parent directory until the last handle to them is closed.
What WILL prevent removal:
If there's a permissions problem (if ACLs prevent removal), removal is aborted.
If an indefinitely locked file or directory is encountered, removal is aborted. Notably, that includes:
cmd.exe (Command Prompt), unlike PowerShell, does lock the directory that is its current directory, so if you have a cmd.exe window open whose current directory is the target directory or one of its subdirectories, removal will fail.
If an application keeps a file open in the target directory's subtree that was not opened with file-sharing mode FILE_SHARE_DELETE / [System.IO.FileShare]::Delete (using this mode is rare), removal will fail. Note that this only applies to applications that keep files open while working with their content. (e.g., Microsoft Office applications), whereas text editors such as Notepad and Visual Studio Code, by contrast, do not keep they've loaded open.
Hidden files and files with the read-only attribute:
These are quietly removed; in other words: this function invariably behaves like Remove-Item -Force.
Note, however, that in order to target hidden files / directories as input, you must specify them as literal paths, because they won't be found via a wildcard expression.
The reliable custom implementation on Windows comes at the cost of decreased performance.
function Remove-FileSystemItem {
<#
.SYNOPSIS
Removes files or directories reliably and synchronously.
.DESCRIPTION
Removes files and directories, ensuring reliable and synchronous
behavior across all supported platforms.
The syntax is a subset of what Remove-Item supports; notably,
-Include / -Exclude and -Force are NOT supported; -Force is implied.
As with Remove-Item, passing -Recurse is required to avoid a prompt when
deleting a non-empty directory.
IMPORTANT:
* On Unix platforms, this function is merely a wrapper for Remove-Item,
where the latter works reliably and synchronously, but on Windows a
custom implementation must be used to ensure reliable and synchronous
behavior. See https://github.com/PowerShell/PowerShell/issues/8211
* On Windows:
* The *parent directory* of a directory being removed must be
*writable* for the synchronous custom implementation to work.
* The custom implementation is also applied when deleting
directories on *network drives*.
* If an indefinitely *locked* file or directory is encountered, removal is aborted.
By contrast, files opened with FILE_SHARE_DELETE /
[System.IO.FileShare]::Delete on Windows do NOT prevent removal,
though they do live on under a temporary name in the parent directory
until the last handle to them is closed.
* Hidden files and files with the read-only attribute:
* These are *quietly removed*; in other words: this function invariably
behaves like `Remove-Item -Force`.
* Note, however, that in order to target hidden files / directories
as *input*, you must specify them as a *literal* path, because they
won't be found via a wildcard expression.
* The reliable custom implementation on Windows comes at the cost of
decreased performance.
.EXAMPLE
Remove-FileSystemItem C:\tmp -Recurse
Synchronously removes directory C:\tmp and all its content.
#>
[CmdletBinding(SupportsShouldProcess, ConfirmImpact='Medium', DefaultParameterSetName='Path', PositionalBinding=$false)]
param(
[Parameter(ParameterSetName='Path', Mandatory, Position = 0, ValueFromPipeline, ValueFromPipelineByPropertyName)]
[string[]] $Path
,
[Parameter(ParameterSetName='Literalpath', ValueFromPipelineByPropertyName)]
[Alias('PSPath')]
[string[]] $LiteralPath
,
[switch] $Recurse
)
begin {
# !! Workaround for https://github.com/PowerShell/PowerShell/issues/1759
if ($ErrorActionPreference -eq [System.Management.Automation.ActionPreference]::Ignore) { $ErrorActionPreference = 'Ignore'}
$targetPath = ''
$yesToAll = $noToAll = $false
function trimTrailingPathSep([string] $itemPath) {
if ($itemPath[-1] -in '\', '/') {
# Trim the trailing separator, unless the path is a root path such as '/' or 'c:\'
if ($itemPath.Length -gt 1 -and $itemPath -notmatch '^[^:\\/]+:.$') {
$itemPath = $itemPath.Substring(0, $itemPath.Length - 1)
}
}
$itemPath
}
function getTempPathOnSameVolume([string] $itemPath, [string] $tempDir) {
if (-not $tempDir) { $tempDir = [IO.Path]::GetDirectoryName($itemPath) }
[IO.Path]::Combine($tempDir, [IO.Path]::GetRandomFileName())
}
function syncRemoveFile([string] $filePath, [string] $tempDir) {
# Clear the ReadOnly attribute, if present.
if (($attribs = [IO.File]::GetAttributes($filePath)) -band [System.IO.FileAttributes]::ReadOnly) {
[IO.File]::SetAttributes($filePath, $attribs -band -bnot [System.IO.FileAttributes]::ReadOnly)
}
$tempPath = getTempPathOnSameVolume $filePath $tempDir
[IO.File]::Move($filePath, $tempPath)
[IO.File]::Delete($tempPath)
}
function syncRemoveDir([string] $dirPath, [switch] $recursing) {
if (-not $recursing) { $dirPathParent = [IO.Path]::GetDirectoryName($dirPath) }
# Clear the ReadOnly attribute, if present.
# Note: [IO.File]::*Attributes() is also used for *directories*; [IO.Directory] doesn't have attribute-related methods.
if (($attribs = [IO.File]::GetAttributes($dirPath)) -band [System.IO.FileAttributes]::ReadOnly) {
[IO.File]::SetAttributes($dirPath, $attribs -band -bnot [System.IO.FileAttributes]::ReadOnly)
}
# Remove all children synchronously.
$isFirstChild = $true
foreach ($item in [IO.directory]::EnumerateFileSystemEntries($dirPath)) {
if (-not $recursing -and -not $Recurse -and $isFirstChild) { # If -Recurse wasn't specified, prompt for nonempty dirs.
$isFirstChild = $false
# Note: If -Confirm was also passed, this prompt is displayed *in addition*, after the standard $PSCmdlet.ShouldProcess() prompt.
# While Remove-Item also prompts twice in this scenario, it shows the has-children prompt *first*.
if (-not $PSCmdlet.ShouldContinue("The item at '$dirPath' has children and the -Recurse switch was not specified. If you continue, all children will be removed with the item. Are you sure you want to continue?", 'Confirm', ([ref] $yesToAll), ([ref] $noToAll))) { return }
}
$itemPath = [IO.Path]::Combine($dirPath, $item)
([ref] $targetPath).Value = $itemPath
if ([IO.Directory]::Exists($itemPath)) {
syncremoveDir $itemPath -recursing
} else {
syncremoveFile $itemPath $dirPathParent
}
}
# Finally, remove the directory itself synchronously.
([ref] $targetPath).Value = $dirPath
$tempPath = getTempPathOnSameVolume $dirPath $dirPathParent
[IO.Directory]::Move($dirPath, $tempPath)
[IO.Directory]::Delete($tempPath)
}
}
process {
$isLiteral = $PSCmdlet.ParameterSetName -eq 'LiteralPath'
if ($env:OS -ne 'Windows_NT') { # Unix: simply pass through to Remove-Item, which on Unix works reliably and synchronously
Remove-Item #PSBoundParameters
} else { # Windows: use synchronous custom implementation
foreach ($rawPath in ($Path, $LiteralPath)[$isLiteral]) {
# Resolve the paths to full, filesystem-native paths.
try {
# !! Convert-Path does find hidden items via *literal* paths, but not via *wildcards* - and it has no -Force switch (yet)
# !! See https://github.com/PowerShell/PowerShell/issues/6501
$resolvedPaths = if ($isLiteral) { Convert-Path -ErrorAction Stop -LiteralPath $rawPath } else { Convert-Path -ErrorAction Stop -path $rawPath}
} catch {
Write-Error $_ # relay error, but in the name of this function
continue
}
try {
$isDir = $false
foreach ($resolvedPath in $resolvedPaths) {
# -WhatIf and -Confirm support.
if (-not $PSCmdlet.ShouldProcess($resolvedPath)) { continue }
if ($isDir = [IO.Directory]::Exists($resolvedPath)) { # dir.
# !! A trailing '\' or '/' causes directory removal to fail ("in use"), so we trim it first.
syncRemoveDir (trimTrailingPathSep $resolvedPath)
} elseif ([IO.File]::Exists($resolvedPath)) { # file
syncRemoveFile $resolvedPath
} else {
Throw "Not a file-system path or no longer extant: $resolvedPath"
}
}
} catch {
if ($isDir) {
$exc = $_.Exception
if ($exc.InnerException) { $exc = $exc.InnerException }
if ($targetPath -eq $resolvedPath) {
Write-Error "Removal of directory '$resolvedPath' failed: $exc"
} else {
Write-Error "Removal of directory '$resolvedPath' failed, because its content could not be (fully) removed: $targetPath`: $exc"
}
} else {
Write-Error $_ # relay error, but in the name of this function
}
continue
}
}
}
}
}
[1] I've personally verified that the issue is resolved in version 20H2, by running the tests in GitHub issue #27958 for hours without failure; this answer suggests that the problem was resolved as early as version 1909, starting with build 18363.657, but Dinh Tran finds that the issue is not resolved as of build 18363.1316 when removing large directory trees such as node_modules. I couldn't find any official information on the subject.