Cannot recursively remove certain files with Remove-Item -Recurse -Force - windows

On powershell 5.1
Using this within a foreach loop, ex:
foreach ($i in $folder.subfolders) {
if( $i.path -like "*node_modules" ){
Remove-Item $i.path -Force -Recurse
}
}
I keep running into this particular error:
+ CategoryInfo : WriteError: (_node_modules_r...dationpath.html:FileInfo) [Remove-Item], DirectoryNotFoundException
+ FullyQualifiedErrorId : RemoveFileSystemItemIOError,Microsoft.PowerShell.Commands.RemoveItemCommand
Remove-Item : Cannot remove item C:\topDir\js\node_modules\ram
l-1-parser\documentation\interfaces\_node_modules_raml_definition_system_node_modules_raml_typesystem_dist_src_index_d_.numbertype.html:
Could not find a part of the path '_node_modules_raml_definition_system_node_modules_raml_typesystem_dist_src_index_d_.numbertype.html'.
At C:\topDir\re.ps1:12 char:11
+ Remove-Item $i.path -Force -Recurse
The files causing the error all exist and I've checked the path and as far as my eyes can tell, they're correct?
Only thing I can make out is that the filenames are abnormally long. But surely that can't be the cause of this?

Long UNC names are problematic to delete. This is a OS specific thing, that limit is 260 characters.
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396
On Windows 10 you can enable long path names, but that does not work in all situations. I some cases you have to result to using the short name, and this requires some conversion effort.
And ditto with what 'Beacon Bits' states, because there is no .subfolder property or method. You can see this by doing...
(gci D:\Temp) | Get-Member | Select Name,MemberType
If you are after the file in a parent or subfolder, you use the...
FullName Property
Something like...
(gci 'D:\Temp\*.txt' -Recurse).FullName
Results
D:\Temp\diff\TestFile_2.txt
...
D:\Temp\Duplicates\BeforeRename1\FsConfig.txt
...
D:\Temp\Duplicates\dup5\SomeRandomThing.txt
...

Related

How to properly format a Copy-Item when filenames and file paths have apostrophes and brackets? [duplicate]

I modified PowerShell script from PowerShell - Batch change files encoding To UTF-8.
# Modified version of https://stackoverflow.com/q/18684793
[Threading.Thread]::CurrentThread.CurrentUICulture = 'en-US'
$Encoding = New-Object System.Text.UTF8Encoding($True) # If UTF8Encoding($False), It will be UTF-8 without BOM
$source = "C:\Users\AKULA\Desktop\SRC" # source directory
$destination = "C:\Users\AKULA\Desktop\DST" # destination directory
if (!(Test-Path $destination)) {
New-Item -Path $destination -ItemType Directory | Out-Null
}
# Delete all previously generated file
Get-ChildItem -Path $destination -Include * -File -Recurse | ForEach-Object {$_.Delete()}
# Recursively convert all files into UTF-8
foreach ($i in Get-ChildItem $source -Force -Recurse -Exclude "desktop.ini") {
if ($i.PSIsContainer) {
continue
}
$name = $i.Fullname.Replace($source, $destination)
$content = Get-Content $i.Fullname
if ($null -ne $content) {
[System.IO.File]::WriteAllLines($name, $content, $Encoding)
} else {
Write-Host "No content from: $i"
}
}
But after using it, I've found that PS cannot handle [ or ] well.
I made some test files that has diversity in name/content.
Get-Content : An object at the specified path C:\Users\AKULA\Desktop\SRC\FILENAME[[[[[[]]]]]]]].txt does not exist, or
has been filtered by the -Include or -Exclude parameter.
At C:\Users\AKULA\Desktop\Convert_to_UTF-8.ps1:24 char:16
+ $content = Get-Content $i.Fullname
+ ~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (System.String[]:String[]) [Get-Content], Exception
+ FullyQualifiedErrorId : ItemNotFound,Microsoft.PowerShell.Commands.GetContentCommand
Since I cannot embed images in question, here is link of IMGUR album.
Full image list: https://imgur.com/a/aN1RG2L
These are what I've tested:
Test files have different names. Their name contains space, ',
[]. Also made up different language(Japanese, Korean).
These files have same content, encoded with UCS-2 BE BOM(UTF-16 BE) so
that I can check if it has re-encoded to UTF-8.
How can I make my script handle [ or ] in file name well?
tl;dr
Indeed, use of the -LiteralPath parameter is the best solution (in PowerShell (Core) v6+, you can shorten to -lp):
$content = Get-Content -LiteralPath $i.Fullname
-LiteralPath ensures that $i.Fullname is taken verbatim (literally); that is, [ and ] in the path are interpreted as themselves rather than having special meaning, as they would have as a -Path argument, due to being interpreted as a wildcard expression - note that -Path is positionally implied if you only pass a value (a string) as the first argument, as you did (Get-Content $i.FullName)
Note: This answer analogously applies to all cmdlets that have both -Path and -LiteralPath parameters, such as Set-Content, Out-File, and Set-Location.
As for what you tried:
$content = Get-Content $i.Fullname
is effectively the same as:
$content = Get-Content -Path $i.Fullname
That is, the (first) positional argument passed to Get-Content is implicitly bound to the
-Path parameter.
The -Path parameter accepts wildcard expressions to allow matching paths by patterns; in addition to support for * (any run of characters) and ? (exactly 1 character), [...] inside a wildcard pattern denotes a character set or range (e.g., [12] or [0-9]).
Therefore an actual path that contains [...], e.g., foo[10].txt, is not recognized as such, because the [10] is interpreted as a character set matching a single character that is either 1 or 0; that is foo[10].txt would match foo0.txt and foo1.txt, but not a file literally named foo[10].txt.
When (implicitly) using -Path, it is possible to escape [ and ] instances that should be interpreted verbatim, namely via the backtick (`), but note that this can get tricky to get right when quoting and/or variable references are involved.
If you know a path to be a literal path, it is best to form a habit of using -LiteralPath (which in PowerShell Core you can shorten to -lp).
However, if your path contains literal [ and ] and you also need wildcard matching, you must use `-escaping - see this answer.
There are at least two situations where the solution's good advice doesn't hold, unfortunately.
Selective error handling
Get-Content -LiteralPath "nobox[]" gives an error message and exception type as if wildcards are involved:
Get-Content : An object at the specified path box[] does not exist, or has been filtered by the -Include or -Exclude parameter.
At line:1 char:1
+ Get-Content -Path "nobox[]"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (System.String[]:String[]) [Get-Content], Exception
+ FullyQualifiedErrorId : ItemNotFound,Microsoft.PowerShell.Commands.GetContentCommand
whereas without the brackets, we get:
Get-Content : Cannot find path 'nobox' because it does not exist.
At line:1 char:1
+ Get-Content -LiteralPath "nobox"
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ObjectNotFound: (nobox:String) [Get-Content], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetContentCommand
Therefore, to silently deal with an optional file, something like:
try {
$lines = Get-Content -LiteralPath $path -ErrorAction Stop
}
catch [System.Management.Automation.ItemNotFoundException] {
$lines = #()
}
chokes on paths with brackets.
Creating a hard or symbolic link
A minor and a major caveat:
The Path parameter, the name of the new item, "works like the LiteralPath parameter of other cmdlets", says the documentation of New-Item clearly, and that seems true and makes sense. Though I wish we could clarify that by writing -LiteralPath.
The Value parameter, the target of the link (also known as Target secretly in v5 and openly later), does not accept wildcard characters according to the same documentation, but that's a lie. The command:
New-Item -ItemType "HardLink" -Path "whatever" -Target "*"
makes Powershell squeal "Cannot set the location because path '*' resolved to multiple containers.".
So you always need the escapes for the target. If you have a file named "f[]", then this will display an error:
New-Item -ItemType "HardLink" -Path "whatever" -Target "f[]"
and this will create a link:
New-Item -ItemType "HardLink" -Path "f[2]" -Target ([WildcardPattern]::Escape("f[]"))
Same for ItemType "SymbolicLink".

In Powershell, why is it so much slower to find all directory names, than to add up all file sizes?

I'm confused by how much slower one of these commands is compared to the other.
This command finds the size of the C:\ drive, by recursively finding the size of every subfolder in C:\
$len = 0
gci -path C:\ -recurse -force -ErrorAction SilentlyContinue | % { $len += $_.length}
This command finds the name of every folder on the C:\ drive, by recursively looking through every folder on the C:\ drive
$result = gci -path C:\ –recurse -force -directory -ErrorAction SilentlyContinue | %{$_.fullName}
Why is the first one so much faster than the second? Is there some kind of caching going on for folder sizes? Is it just a matter of string manipulation being orders of magnitude slower than integer manipulation?
Note: as written your first command pushes out a bunch of error messages I cancelled the command because it ran on for some minutes! Although directories have a length property it is the length of the Full Diretory Path and not the size of the directory in bytes.
The property 'length' cannot be found on this object. Verify that the property
exists.
At line:2 char:68
+ ... ecurse -force -ErrorAction SilentlyContinue | % { $len += $_.length}
+ ~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], PropertyNotFoundException
+ FullyQualifiedErrorId : PropertyNotFoundStrict
So I added the -File switch parameter.
PS> $len = 0
Measure-Command -Expression {
gci -path C:\ -File -recurse -force -ErrorAction SilentlyContinue | % { $len += $_.length}}
TotalSeconds : 58.5945269
PS> $len = 0
Measure-Command -Expression {
$x = gci -path C:\ -File -recurse -force -ErrorAction SilentlyContinue
ForEach ($File in $x) { $Len += $file.length}
}
TotalSeconds : 48.99365
As you can see using the Pipe is considerably slower that saving to a variable, an array in this case, and then processing from that.
I edited out the redundant info output by Measure-Command
In the second command you are not processing files only directories (-Directory) and you are not calculating size thus making it faster.
As you can see this command only returns an array of Strings of the directory paths. (again edited for brevity).
PS> $result.count
118148
PS> $result[0] | gm
TypeName: System.Stri
Name MemberType Definition
---- ---------- ----------
Chars ParameterizedProperty char Chars(int index) {get;}
Length Property int Length {get;}
PS> $result[0].Length
14
PS> $result[0]
C:\$GetCurrent
HTH

Rename certain characters from filenames in all files in a folder and his sub-folders

I Want to replace certain character from my files
for example
i want to change > "original" -> "something else"
i use this script to replace it in one folder only using powershell
Dir |>> Rename-Item -NewName { $_.Name -replace "original","something " }
the issue is , with that code , that's only replace the files inside one folder and it don't replace them in the subfolders
i need it to replace it in file on the folder and in the sub folder files too , is that possible?
( i'm kinda new to this )
note : this code shows this error when there's any subfolders with the files
erorr =
Rename-Item : Source and destination path must be different.
At line:2 char:1
+ Rename-Item -NewName { $_.Name -replace "original","something " }
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : WriteError: (F:\folder-name\...ff\py-compat:String) [Rename-Item], IOException
+ FullyQualifiedErrorId : RenameItemIOError,Microsoft.PowerShell.Commands.RenameItemCommand
Operating system : windows
info
Operating system model : windows 10
In case of directories the rename-item commandlet gets the same path as input and as value of the -newname parameter, so it errors out.
Adding the -verbose and -whatif switch parameters to the command will show that.
Use the -Path parameter to specify the base path you want to recurse, the -recurse switch parameter to recursively visit each subfolder, the -file switch parameter to filter out directories, and/or use the -filter parameter to include a subset of files.
Example:
get-childitem -Recurse -File -Path 'c:\temp' | rename-item -newname { $_.name -replace "original","replacement }

Metadata export error when running on large directory structures

This script exports the basic file metadata in PowerShell to a .csv.
PS K:\> Get-childitem -recurse -file | select-object length,lastwritetime,fullname | export-csv filelist.csv -notypeinformation
It works great on simple directory structures of 10-20K files, but when I run it on 500K+ files of complex, multi-level directory structures it freezes or gives me error:
At line:1 char:1
+ Get-childitem -recurse -file | select-object length,lastwritetime,ful ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidOperation: (0:Int32) [], RuntimeException
+ FullyQualifiedErrorId : CallDepthOverflow
Is this a problem with the -recurse or else? Using PowerShell 5 1 (Major Minor).
Test to see how long the filepaths are.
If the filepaths are more than 248 characters... That may be why you are getting the error.
To check to see if any come back with more than 248 characters you can do
Get-childitem "filepath" -recurse | % {
$filepath = $_.FullName
$charactercount = ($filepath | Measure-Object -Character).Characters
If ($charactercount -gt 248){write-host $filepath}
}

Remove-Item doesn't work, Delete does

Does anyone have any idea why Remove-Item would fail while Delete works?
In below script, I get a list of files I'd like to delete.
Using Remove-Item I get following error message:
VERBOSE: Performing the operation "Remove File" on target
"\\UncPath\Folder\test.rtf". Remove-Item : Cannot remove item
\\UncPath\Folder\test.rtf: Access to the path is denied.
but using Delete is deleting those files as we speak.
Script
$files = gci \\UncPath\Folder| ?{ $_.LastWriteTime -le (Get-Date).addDays(-28) }
# This doesn't work
$files | Remove-Item -force -verbose
# But this does
$files | % { $_.Delete() }
powershell may act strange with UNC path, I think it prepends the UNC Path with the current provider you can verify this with :
cd c:
test-path \\127.0.0.1\c$
returns TRUE
cd HKCU:
test-path \\127.0.0.1\c$
returns FALSE
when specifying the fullpath we're telling powershell to use the filesystem provider, that solves the problem. you could also specify the provider like remove-item filesystem::\\uncpath\folder
I can finally repro this and IMO it appears to be a bug. The repro is to have an open share like C$ but to set Deny Modify perms for the user on the file. When I do that, I observe this:
PS> gci '\\Keith-PC\C$\Users\Keith\foo.txt' | ri -for
ri : Cannot remove item \\Keith-PC\C$\Users\Keith\foo.txt: Access to the path is denied.
At line:1 char:43
+ gci '\\Keith-PC\C$\Users\Keith\foo.txt' | ri -for
+ ~~~~~~~
+ CategoryInfo : InvalidArgument: (\\Keith-PC\C$\Users\Keith\foo.txt:FileInfo) [Remove-Item], ArgumentExc
eption
+ FullyQualifiedErrorId : RemoveFileSystemItemArgumentError,Microsoft.PowerShell.Commands.RemoveItemCommand
PS> gci '\\Keith-PC\C$\Users\Keith\foo.txt' | %{$_.Delete()} # <== this works!
I also observe that removing the -Force parameter deletes the file without error as well. The deny perms still allow me to delete the file from Windows Explorer so that leads me to believe that the file should delete. So what is up with using the -Force parameter? When I delve into the ErrorRecord I see this:
Message : Access to the path is denied.
ParamName :
Data : {}
InnerException :
TargetSite : Void set_Attributes(System.IO.FileAttributes)
StackTrace : at System.IO.FileSystemInfo.set_Attributes(FileAttributes value)
at Microsoft.PowerShell.Commands.FileSystemProvider.RemoveFileSystemItem(FileSystemInfo
fileSystemInfo, Boolean force)
It seems that the -Force parameter is trying to set (more likely reset) attributes and the permissions on the file don't allow it e.g.:
PS> gci '\\Keith-PC\C$\Users\Keith\foo.txt' | %{$_.Attributes = 'Normal'}
Exception setting "Attributes": "Access to the path is denied."
At line:1 char:45
+ gci '\\Keith-PC\C$\Users\Keith\foo.txt' | %{$_.Attributes = 'Normal'}
+ ~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], SetValueInvocationException
+ FullyQualifiedErrorId : ExceptionWhenSetting
So it seems to me that PowerShell should first try as if the -Force weren't present and if that fails, then try resetting attributes.

Resources