PowerShell - Determine the existence of certain files in a folder hierarchy efficiently - sorting

I'm looking to thin down how many folders I need to recover after a cryptolocker outbreak at a clients site and started looking into powershell as a good way to do this. What I need to do is recover a folder if it has any file inside with the extension .encrypted.
I can run the below
get-childitem C:\ -recurse -filter “*.encrypted” | %{$_.DirectoryName} | Get-Unique
And get a list of all folders that have .encrypted files in them but what I would like to do is thin down the list for example if we have the below file list and assume * means the folder contains encrypted files.
C:\Folder1
C:\Folder1\Folder2\Folder4*
C:\Folder1\Folder2*
C:\Folder1\Folder3\Folder5*
C:Folder1\Folder3\Folder6\
rather than returning
C:\Folder1\Folder2\Folder4*
C:\Folder1\Folder2*
C:\Folder1\Folder3\Folder5*
I would like it just to return as this would be the optimal recovery option.
C:\Folder1\Folder2*
C:\Folder1\Folder3\Folder5*
I know this is a fairly complex problem so I'm not asking anyone to solve it for me just some pointers in the right direction would be awesome as my brain is fried at the moment and I need to write this fairly quickly.

Here's a simple way to do this that should be pretty efficient:
PS C:\> dir -ad -rec | where { test-path (join-path $_.FullName *.encrypted) }
dir is an alias for get-childitem
where is an alias for where-object
-ad means return directories only
-rec means recurse
test-path returns $true if the path exists (yes, it handles wildcards)
S, we recurse through all folders forwarding the folder object down the pipeline. We get the full name of the folder and append *.encrypted to it. If test-path returns $true for this path, we forward the folder down the pipeline. The folder ends up in the console output.
Now, if you want to get a little fancier, here's a more fleshed out one-liner than will report the folders and the encrypted files count into a csv file named after the machine:
dir -ad -rec | ? { test-path (join-path $_.FullName *.txt) } | % {
[pscustomobject]#{"Path"=$_.fullname;"Count"=(dir (join-path $_ *.txt)).count}} |`
Export-Csv "c:\temp\$(hostname).csv" -NoTypeInformation
(? and % are aliases for where-object and foreach-object respectively)
With a little more effort, you could use a fan-out scan of the entire company assuming powershell remoting is enabled on each target machine and have it return all results to you from all machines.
Good luck!

This is too much for a comment, but I don't know that it would be a good answer, just a kind of hackish way to get it done...
The only thing I could think of is to get your list of folders, then start matching them all against each other, and when you get two that at least partially match remove the longer one.
$FullList = GCI C:\*.encrypted | Select -Expand DirectoryName -Unique | Sort -Property Length
$ToRemove = #()
foreach($Folder in $FullList){$ToRemove+=$FullList| Where{$_ -ne $Folder -and ($_ -match [regex]::Escape($Folder))}}
$FinalList = $FullList | Where{$ToRemove -notcontains $_}
That's going to be slow though, there has to be a better way to do it. I just haven't thought of a better way yet.
Don't get me wrong, this will work, and it's faster than going through things by hand for sure, but I'm sure that there has to be a better way to do it.

Related

Is there any way to find 'metadata' on folders using powershell

So I want to know if any of the folders in a directory have any subfolders or files in them, I tried just looking at the directory in PowerShell but it gave me only mode, last write time, and name. Is there any way of adding to this list to include metadata of the folder like size or number of subfiles/folders all I want to know is if they are empty or not so there may be a simpler way I'm missing.
Thanks for any help you can give!
I see the question is tagged 'windows', so on Windows you could also use a COM object.
$fso = New-Object -ComObject Scripting.FileSystemObject
$folder = $fso.GetFolder($pathToFolder)
$folder will be an object with a bunch of interesting metadata on it, including SubFolders and Files. One of the interesting ones is Size. If Size is zero, there are no files in that directory, or in any nested subdirectories either.
If you just want to know if there are folders/subfolders and/or files then this will work:
$folder="C:\Test"
Get-ChildItem $folder -Recurse | Measure-Object
Output (in my case)
Count : 2
Average :
Sum :
Maximum :
Minimum :
Property :
If you want to see more properties then this might work for you:
Get-ChildItem -Path $folder -Recurse | Format-List *
alternatively you can also select the first x, last x, or even skip items:
Get-ChildItem -Path $folder -Recurse |Select-Object -First 2| Format-List *
*-Recurse will check all folders below

Powershell "LastWriteTime" not working

Is there a way to stop Powershell from sorting by default? I need to read in files from a directory and in the order which they are listed in the directory I need them to also be listed in the array (variable). Even when I use -lastwritetime on the get-childitem command, it seems to have no affect. The primary reason why I want to do this is because the files have names that are all the same except each file has a number after it like the following:
document1.doc
document2.doc
document3.doc
.....
document110.doc
The problem is if it's sorted by name, it will sort in this manner:
document1.doc
document10.doc
document111.doc
Which is horribly wrong!
Right now I have this command and it doesn't work:
$filesnames1 = get-childItem -name *.doc -Path c:\fileFolder\test | sort-object -LastWriteTime
You probably want something more along these lines.
$filesnames1 = Get-ChildItem -Path c:\fileFolder\test\*.doc |
Sort-Object -Property LastWriteTime
I don't think either of those two cmdlets have a -LastWriteTime parameter.
If you need only the names from those filesystem objects, you can use ($filesnames1).Name after the code above. There are other ways.
Thanks for responding Mike. What I did is put a "-filter *.pdf" just before -path which gave me the headers. Then I piped in a "select-object -ExpandProperty name" to list it exactly how I needed it to. It was a little trial and error but I did eventually figure it out.
$filesnames1 = Get-ChildItem -filter *.doc -Path c:\fileFolder\test |
Sort-Object -LastWriteTime | -ExpandProperty name

How to use PowerShell Remove-Item to delete a directory with long name? [duplicate]

I'm writing a simple script to delete USMT migration folders after a certain amount of days:
## Server List ##
$servers = "Delorean","Adelaide","Brisbane","Melbourne","Newcastle","Perth"
## Number of days (-3 is over three days ago) ##
$days = -3
$timelimit = (Get-Date).AddDays($days)
foreach ($server in $servers)
{
$deletedusers = #()
$folders = Get-ChildItem \\$server\USMT$ | where {$_.psiscontainer}
write-host "Checking server : " $server
foreach ($folder in $folders)
{
If ($folder.LastWriteTime -lt $timelimit -And $folder -ne $null)
{
$deletedusers += $folder
Remove-Item -recurse -force $folder.fullname
}
}
write-host "Users deleted : " $deletedusers
write-host
}
However I keep hitting the dreaded Remove-Item : The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
I've been looking at workarounds and alternatives but they all revolve around me caring what is in the folder.
I was hoping for a more simple solution as I don't really care about the folder contents if it is marked for deletion.
Is there any native Powershell cmdlet other than Remove-Item -recurse that can accomplish what I'm after?
I often have this issue with node projects. They nest their dependencies and once git cloned, it's difficult to delete them. A nice node utility I came across is rimraf.
npm install rimraf -g
rimraf <dir>
Just as CADII said in another answer: Robocopy is able to create paths longer than the limit of 260 characters. Robocopy is also able to delete such paths. You can just mirror some empty folder over your path containing too long names in case you want to delete it.
For example:
robocopy C:\temp\some_empty_dir E:\temp\dir_containing_very_deep_structures /MIR
Here's the Robocopy reference to know the parameters and various options.
I've created a PowerShell function that is able to delete a long path (>260) using the mentioned robocopy technique:
function Remove-PathToLongDirectory
{
Param(
[string]$directory
)
# create a temporary (empty) directory
$parent = [System.IO.Path]::GetTempPath()
[string] $name = [System.Guid]::NewGuid()
$tempDirectory = New-Item -ItemType Directory -Path (Join-Path $parent $name)
robocopy /MIR $tempDirectory.FullName $directory | out-null
Remove-Item $directory -Force | out-null
Remove-Item $tempDirectory -Force | out-null
}
Usage example:
Remove-PathToLongDirectory c:\yourlongPath
This answer on SuperUser solved it for me: https://superuser.com/a/274224/85532
Cmd /C "rmdir /S /Q $myDir"
I learnt a trick a while ago that often works to get around long file path issues. Apparently when using some Windows API's certain functions will flow through legacy code that can't handle long file names. However if you format your paths in a particular way, the legacy code is avoided. The trick that solves this problem is to reference paths using the "\\?\" prefix. It should be noted that not all API's support this but in this particular case it worked for me, see my example below:
The following example fails:
PS D:\> get-childitem -path "D:\System Volume Information\dfsr" -hidden
Directory: D:\System Volume Information\dfsr
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a-hs 10/09/2014 11:10 PM 834424 FileIDTable_2
-a-hs 10/09/2014 8:43 PM 3211264 SimilarityTable_2
PS D:\> Remove-Item -Path "D:\System Volume Information\dfsr" -recurse -force
Remove-Item : The specified path, file name, or both are too long. The fully qualified file name must be less than 260
characters, and the directory name must be less than 248 characters.
At line:1 char:1
+ Remove-Item -Path "D:\System Volume Information\dfsr" -recurse -force
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : WriteError: (D:\System Volume Information\dfsr:String) [Remove-Item], PathTooLongExcepti
on
+ FullyQualifiedErrorId : RemoveItemIOError,Microsoft.PowerShell.Commands.RemoveItemCommand
PS D:\>
However, prefixing the path with "\\?\" makes the command work successfully:
PS D:\> Remove-Item -Path "\\?\D:\System Volume Information\dfsr" -recurse -force
PS D:\> get-childitem -path "D:\System Volume Information\dfsr" -hidden
PS D:\>
If you have ruby installed, you can use Fileman:
gem install fileman
Once installed, you can simply run the following in your command prompt:
fm rm your_folder_path
This problem is a real pain in the neck when you're developing in node.js on Windows, so fileman becomes really handy to delete all the garbage once in a while
This is a known limitation of PowerShell. The work around is to use dir cmd (sorry, but this is true).
http://asysadmin.tumblr.com/post/17654309496/powershell-path-length-limitation
or as mentioned by AaronH answer use \?\ syntax is in this example to delete build
dir -Include build -Depth 1 | Remove-Item -Recurse -Path "\\?\$($_.FullName)"
If all you're doing is deleting the files, I use a function to shorten the names, then I delete.
function ConvertTo-ShortNames{
param ([string]$folder)
$name = 1
$items = Get-ChildItem -path $folder
foreach ($item in $items){
Rename-Item -Path $item.FullName -NewName "$name"
if ($item.PSIsContainer){
$parts = $item.FullName.Split("\")
$folderPath = $parts[0]
for ($i = 1; $i -lt $parts.Count - 1; $i++){
$folderPath = $folderPath + "\" + $parts[$i]
}
$folderPath = $folderPath + "\$name"
ConvertTo-ShortNames $folderPath
}
$name++
}
}
I know this is an old question, but I thought I would put this here in case somebody needed it.
There is one workaround that uses Experimental.IO from Base Class Libraries project. You can find it over on poshcode, or download from author's blog. 260 limitation is derived from .NET, so it's either this, or using tools that do not depend on .NET (like cmd /c dir, as #Bill suggested).
Combination of tools can work best, try doing a dir /x to get the 8.3 file name instead. You could then parse out that output to a text file then build a powershell script to delete the paths that you out-file'd. Take you all of a minute. Alternatively you could just rename the 8.3 file name to something shorter then delete.
For my Robocopy worked in 1, 2 and 3
First create an empty directory lets say c:\emptydir
ROBOCOPY c:\emptydir c:\directorytodelete /purge
rmdir c:\directorytodelete
This is getting old but I recently had to work around it again. I ended up using 'subst' as it didn't require any other modules or functions be available on the PC this was running from. A little more portable.
Basically find a spare drive letter, 'subst' the long path to that letter, then use that as the base for GCI.
Only limitation is that the $_.fullname and other properties will report the drive letter as the root path.
Seems to work ok:
$location = \\path\to\long\
$driveLetter = ls function:[d-z]: -n | ?{ !(test-path $_) } | random
subst $driveLetter $location
sleep 1
Push-Location $driveLetter -ErrorAction SilentlyContinue
Get-ChildItem -Recurse
subst $driveLetter /D
That command is obviously not to delete files but can be substituted.
PowerShell can easily be used with AlphaFS.dll to do actual file I/O stuff
without the PATH TOO LONG hassle.
For example:
Import-Module <path-to-AlphaFS.dll>
[Alphaleonis.Win32.Filesystem.Directory]::Delete($path, $True)
Please see at Codeplex: https://alphafs.codeplex.com/ for this .NET project.
I had the same issue while trying to delete folders on a remote machine.
Nothing helped but... I found one trick :
# 1:let's create an empty folder
md ".\Empty" -erroraction silentlycontinue
# 2: let's MIR to the folder to delete : this will empty the folder completely.
robocopy ".\Empty" $foldertodelete /MIR /LOG+:$logname
# 3: let's delete the empty folder now:
remove-item $foldertodelete -force
# 4: we can delete now the empty folder
remove-item ".\Empty" -force
Works like a charm on local or remote folders (using UNC path)
Adding to Daniel Lee's solution,
When the $myDir has spaces in the middle it gives FILE NOT FOUND errors considering set of files splitted from space. To overcome this use quotations around the variable and put powershell escape character to skip the quatations.
PS>cmd.exe /C "rmdir /s /q <grave-accent>"$myDir<grave-accent>""
Please substitute the proper grave-accent character instead of <grave-accent>
SO plays with me and I can't add it :). Hope some one will update it for others to understand easily
Just for completeness, I have come across this a few more times and have used a combination of both 'subst' and 'New-PSDrive' to work around it in various situations.
Not exactly a solution, but if anyone is looking for alternatives this might help.
Subst seems very sensitive to which type of program you are using to access the files, sometimes it works and sometimes it doesn't, seems to be the same with New-PSDrive.
Any thing developed using .NET out of the box will fail with paths too long. You will have to move them to 8.3 names, PInVoke (Win32) calls, or use robocopy

In PowerShell, how can I extract a file from HKEY_Users from all SIDs?

I am writing a PowerShell module to look for data that each user who has logged onto the computer at some point might have in their directory in HKEY_USERS. My initial thought was to mount HKEY_USERS, find a way to store each user's SID in a string variable, and then loop through all folders like so:
dir HKU\<STRING VARIABLE HOLDING SID>\Software\MyApp\Mydesireddata
Is there a way I can avoid having to loop through SIDs (because I won't know them ahead of time), and extract that file info from each SID on the system while remembering which SID it came from?
EDIT: Here is an example of the key I'm trying to extract from each user's SID using regedit (vncviewer's EulaAccepted)
Use Get-ChildItem to retrieve each user-specific subkey:
$UserHives = Get-ChildItem Registry::HKEY_USERS\ |Where-Object {$_.Name -match '^HKEY_USERS\\S-1-5-21-[\d\-]+$'}
Then loop over each entry and retrieve the desired registry value:
foreach($Hive in $UserHives)
{
# Construct path from base key
$Path = Join-Path $Hive.PSPath "SOFTWARE\MyApp\DataKey"
# Attempt to retrieve Item property
$Item = Get-ItemProperty -Path $Path -Name ValueName -ErrorAction SilentlyContinue
# Check if item property was there or not
if($Item)
{
$Item.ValueName
}
else
{
# doesn't exist
}
}
I tackled this issue a slightly different way; preferring to make use of a conspicuously placed wildcard.
Get-ItemProperty -Path Registry::HKEY_USERS\*\SOFTWARE\TestVNC\viewer\ -Name EulaAccepted |
Select-Object -Property #{n="SID";e={$_.PSPath.Split('::')[-1].Split('\')[1]}},EulaAccepted
The wildcard will automatically check all available paths and return what you need as well as the SID from the parent path.
As for the username (which is probably more useful than a SID), you didn't specifically ask for it, but I added it in for grins; this should cover local and domain accounts.
mind the line breaks
Get-ItemProperty -Path Registry::HKEY_USERS\*\SOFTWARE\TestVNC\viewer\ -Name EulaAccepted |
Select-Object -Property #{n="SID";e={$_.PSPath.Split('::')[-1].Split('\')[1]}},EulaAccepted |
Select-Object -Property #{n="User";e={[System.Security.Principal.SecurityIdentifier]::new($_.SID).`
Translate([System.Security.Principal.NTAccount]).Value}},SID,EulaAccepted
Getting the username is just ugly; there's likely a cleaner way to get it, but that's what I have in my head. The double-select really makes my skin crawl - there's something unpleasant about it. I could just do a one shot thing, but then it gets so unwieldly you don't even know what you're doing by looking at it.
I've included a screenshot of the registry below, and a screenshot of the screen output from running the few lines.

Move list of empty folders from one volume to another with PowerShell

I am using PowerShell 2.0 on a Windows SBS 2008 machine with the latest service packs. I have a two line script that finds all empty folders in a directory:
$a = Get-ChildItem E:\File_Server -recurse | Where-Object {$_.PSIsContainer -eq $True}
$a | Where-Object {$_.GetFiles().Count -eq 0} | [what now?!]
The second line finds all folders that are empty, however I'm stumped on the last pipe. Here's what I've tried so far (keep in mind that the following were tried after the second pipe in the second line above):
$_.move(F:\path) Yes, I tried that. Yes, I'm a PowerShell noob. Of course, I received the error "Expressions are only allowed as the first element of a pipeline."
move-item -destination F:\Path I receive the lamest error ever: "Move-Item : Source and destination path must have identical roots. Move will not work across volumes." Seriously? What kind of asinine limitation is that?! Moving on...
copy-item -destination F:\empty_folders Apparently I can get around the limitation of move-item by using copy-item and then use remove-item. No such luck, however. I started the script with copy-item first. PowerShell didn't throw any errors, but also didn't do any thing else. It just sat there for the better part of an hour. No directories were moved.
Summary:
How do I take the list of empty folders that I have and, using PowerShell, move those empty directories elsewhere (across volumes!) deleting the originals after the move? Notice the caveat "using PowerShell" as I've already thought about adding RoboCopy to the mix but would like to keep it within PowerShell.
I was pretty frustrated with this as well, but like you said copy-item followed by remove-item will work:
gci e:\file_server | ?{$_.PSIsContainer -eq $true} | ?{$_.GetFiles().count -eq 0} |
%{copy-item -LiteralPath $_.fullname -Destination f:\path; remove-item $_.fullname}
Note this is not just a limitation with Move-Item and Powershell but applicable to .Net System.IO.Directory.Move and to be fair, the documentation says so:
Move-Item will move files between drives that are supported by the
same provider, but it will move directories only within the same
drive.
http://technet.microsoft.com/en-us/library/dd315310.aspx
PS: You got the "Expressions are only allowed as the first element of a pipeline." error because the $_.move(F:\path) that you were trying must be inside a foreach-object like so - %{$_.move(F:\path)}

Resources