I have written a powershell script for archival of old log files or say some output file of web application which is in TBs but the script is taking very long time. I have done some improvement but not able to speed up more from here.
Code:
#region Archive Files using 7zip
[cmdletbinding()]
Param
(
[Parameter(Mandatory=$true, HelpMessage = "Path needs to be with trailing slash at the end of location." )]
[string]$SourceFilesPath
)
$7zip = "C:\Program Files\7-Zip\7z.exe"
$FilePath = ""
foreach ( $filename in $(Get-ChildItem $SourceFilesPath -Force -Recurse | where {$_.LastWriteTime -lt (get-date).AddDays(-1).ToShortDateString()}))
{
$FilePath = Get-ItemProperty $filename.FullName
$ZipFilePath = $filename.Directory.ToString() + "\ZippedFiles" + "\Archive_" + $filename.LastWriteTime.ToString("MMddyyyy") + ".7z"
$tempPath = ("-w"+"C:\Temp")
$OutputData = &$7zip a $tempPath -t7z $ZipFilePath $FilePath
$OutputData
if ($OutputData -contains "Everything is OK")
{
Remove-Item $FilePath -Force
Write-Output "File removed $FilePath"
}
Get-Item $ZipFilePath | ForEach-Object {$_.LastWriteTime = $filename.LastWriteTime}
}
#endregion
#region Archive Files using 7zip
[cmdletbinding()]
Param
(
[Parameter(Mandatory = $true, HelpMessage = 'Path needs to be with trailing slash at the end of location.' )]
[string]$SourceFilesPath
)
Import-Module ..\Invoke-Parallel.ps1 # Download from PSGallery
$7zip = 'C:\Program Files\7-Zip\7z.exe'
$fileToArchive = $(Get-ChildItem $SourceFilesPath -Force -Recurse | Where-Object -FilterScript {
$_.LastWriteTime -lt (Get-Date).AddDays(-1).ToShortDateString()
})
$counter = 0
$groupSize = 2000 # Will group items by 2,000 increments
$groups = $fileToArchive | Group-Object -Property {
[math]::Floor($counter++ / $groupSize)
}
$groups
# This will spawn multiple instances of 7zip - depending on how many groups of 2,000 files exist
$groups.Group | Invoke-Parallel -ScriptBlock {
$FilePath = $null
$fileName = $_
$FilePath = Get-ItemProperty -Path $fileName.FullName
$ZipFilePath = $fileName.Directory.ToString() + '\ZippedFiles' + '\Archive_' + $fileName.LastWriteTime.ToString('MMddyyyy') + '.7z'
$tempPath = ('-w'+'C:\Temp')
$OutputData = &$Using:7zip a $tempPath -t7z $ZipFilePath $FilePath
$OutputData
if ($OutputData -contains 'Everything is OK')
{
Remove-Item $FilePath -Force
Write-Output -InputObject "File removed $FilePath"
}
Get-Item $ZipFilePath | ForEach-Object -Process {
$_.LastWriteTime = $fileName.LastWriteTime
}
}
Related
I am working on a Windows Server File Security project, and need the explicit ACL permissions for our " O:" and all of the subfolders. I am wanting to export it to an CSV for easy formatting. I am looking for a Powershell script that can list the folder name and the Security Group or User that has access to that folder.
$rootpath = "O:\ADSMO"
$outfile = "ExplicitACLs.txt"
New-Variable acl
$Report = #"
Explicit permissions on folders under parent $rootpath.
"#
$Report | out-file -Encoding ASCII -FilePath $outfile
Get-ChildItem -Recurse $rootpath -Exclude "*.*" | Where-Object {$_.PSisContainer } | ForEach-Object {
$acl = Get-Acl -Path $_.FullName
$access = $acl.access
if ( $access | Where-Object { $_.IsInherited -eq $False }) {
Add-Content -Path $outfile $_
$access | Where-Object { $_.IsInherited -eq $False } | ForEach-Object {
$i = $_.IdentityReference
$t = "`t"
$r = $_.FileSystemRights
$c = "$i"+"$t"+"$t"+"$t"+"$r"
Add-Content -Path $outfile $c
}
Add-Content -Path $outfile ""
}
Clear-Variable acl
Clear-Variable access
}
Add-Content -Path $outfile ""
Seems like you're trying to build a CSV manually, which is definitely not recommended. You can use Export-Csv to export your report to CSV. From what I'm seeing, your code could be simplified to this:
Get-ChildItem O:\ADSMO -Directory -Recurse | ForEach-Object {
foreach($access in (Get-Acl $_.FullName).Access) {
# if `IsInherited = $true` go to next iteration
if($access.IsInherited) { continue }
[pscustomobject]#{
FolderName = $_.Name
FolderPath = $_.FullName
IdentityReference = $access.IdentityReference
FileSystemRights = $access.FileSystemRights
}
}
} | Export-Csv 'C:\path\to\acls.csv' -NoTypeInformation
I am attempting to extract the date last modified from the files in a Windows directory. Here is my basic script:
Function Get-FolderItem {
[cmdletbinding(DefaultParameterSetName='Filter')]
Param (
[parameter(Position=0,ValueFromPipeline=$True,ValueFromPipelineByPropertyName=$True)]
[Alias('FullName')]
[string[]]$Path = $PWD,
[parameter(ParameterSetName='Filter')]
[string[]]$Filter = '*.*',
[parameter(ParameterSetName='Exclude')]
[string[]]$ExcludeFile,
[parameter()]
[int]$MaxAge,
[parameter()]
[int]$MinAge
)
Begin {
$params = New-Object System.Collections.Arraylist
$params.AddRange(#("/L","/E","/NJH","/NDL","/BYTES","/FP","/NC","/XJ","/R:0","/W:0","T:W","/TS","/UNILOG:c:\temp\test.txt"))
#params.AddRange(#("/L","/S","/NJH","/BYTES","/FP","/NC","/NDL","/TS","/XJ","/R:0","/W:0"))
If ($PSBoundParameters['MaxAge']) {
$params.Add("/MaxAge:$MaxAge") | Out-Null
}
If ($PSBoundParameters['MinAge']) {
$params.Add("/MinAge:$MinAge") | Out-Null
}
}
Process {
ForEach ($item in $Path) {
Try {
$item = (Resolve-Path -LiteralPath $item -ErrorAction Stop).ProviderPath
If (-Not (Test-Path -LiteralPath $item -Type Container -ErrorAction Stop)) {
Write-Warning ("{0} is not a directory and will be skipped" -f $item)
Return
}
If ($PSBoundParameters['ExcludeFile']) {
$Script = "robocopy `"$item`" NULL $Filter $params /XF $($ExcludeFile -join ',')"
} Else {
$Script = "robocopy `"$item`" NULL $Filter $params"
}
Write-Verbose ("Scanning {0}" -f $item)
Invoke-Expression $Script | Out-Null
get-content "c:\temp\test.txt" | ForEach {
Try {
If ($_.Trim() -match "^(?<Children>\d+)\s(?<FullName>.*)") {
$object = New-Object PSObject -Property #{
FullName = $matches.FullName
Extension = $matches.fullname -replace '.*\.(.*)','$1'
FullPathLength = [int] $matches.FullName.Length
FileHash = Get-FileHash -LiteralPath "\\?\$($matches.FullName)" |Select -Expand Hash
Created = ([System.IO.FileInfo] $matches.FullName).creationtime
LastWriteTime = ([System.IO.FileInfo] $matches.FullName).LastWriteTime
Characters = (Get-Content -LiteralPath "\\?\$($matches.FullName)" | Measure-Object -ignorewhitespace -Character).Characters
Owner = (Get-ACL $matches.Fullname).Owner
}
$object.pstypenames.insert(0,'System.IO.RobocopyDirectoryInfo')
Write-Output $object
} Else {
Write-Verbose ("Not matched: {0}" -f $_)
}
} Catch {
Write-Warning ("{0}" -f $_.Exception.Message)
Return
}
}
} Catch {
Write-Warning ("{0}" -f $_.Exception.Message)
Return
}
}
}
}
$a = Get-FolderItem "C:\TargetDirectory\Folder" | Export-Csv -Path C:\Temp\output.csv -Encoding Unicode
The script extracts the date last modified of filepaths less than 260 characters. It returns a nonsense date of 1600-12-31 4:00:00 PM for files longer than 260 characters. Here is the line that is not working:
LastWriteTime = ([System.IO.FileInfo] $matches.FullName).LastWriteTime
My first attempt to solve this problem was to find a command that began with Get- because such commands were useful in extracting filehashes, filepaths, character counts and owner names of files longer than 260 characters. For example:
Owner = (Get-ACL $matches.Fullname).Owner
Characters = (Get-Content -LiteralPath "\\?\$($matches.FullName)" | Measure-Object-ignorewhitespace -Character).Characters
FileHash = Get-FileHash -LiteralPath "\\?\$($matches.FullName)" |Select -Expand Hash
Get-Date however seemed to be about getting the current date.
In my second attempt, I went back to Boe Prox's original blogpost on this script and noticed that his script had two components that were missing from mine:
a robocopy switch /TS
Date = [datetime]$matches.Date
I added to my script however doing so return an error: WARNING: Cannot convert null to type "System.DateTime". I rechecked the file in the directory, and it clearly has a date.
I reexamined the documentation on Get-Date and tried
Date = Get-Date -Format o | ForEach-Object { $matches -replace ":", "." }
However, this returned WARNING: Cannot convert value "2018/03/05 18:06:54 C:TargetDirectory\Folder\Temp.csv to type "System.IO.FileInfo". Error: " Illegal characters in path."
(N.B. In other posts, people have suggested changing the server settings to permit the existence of files longer than 260 characters. This is not an option for me because I do not have access to the servers.)
Once you hit 260 characters in the path, you hit the old Windows MAX_PATH limitation. In order to get around that, you have to prepend your path with \\?\.
In your code above, you do that for Characters and FileHash but you don't do that when retrieving LastWriteTime. e.g. Changing the path to this will work:
Created = ([System.IO.FileInfo] "\\?\$($matches.FullName)").creationtime
LastWriteTime = ([System.IO.FileInfo] "\\?\$($matches.FullName)").LastWriteTime
The alternative way is to use the Get-ChildItem cmdlet along with \\?\ prepended to the path to retrieve most of the fields you want without having to query it multiple times:
get-content "c:\temp\test.txt" | ForEach {
Try {
If ($_.Trim() -match "^(?<Children>\d+)\s(?<FullName>.*)") {
$file = Get-ChildItem "\\?\$($matches.FullName)"
$object = New-Object PSObject -Property #{
FullName = $file.FullName
Extension = $file.Extension
FullPathLength = $file.FullName.Length
FileHash = Get-FileHash -LiteralPath "\\?\$($matches.FullName)" |Select -Expand Hash
Created = $file.CreationTime
LastWriteTime = $file.LastWriteTime
Characters = (Get-Content -LiteralPath "\\?\$($matches.FullName)" | Measure-Object -ignorewhitespace -Character).Characters
Owner = (Get-ACL $matches.Fullname).Owner
}
$object.pstypenames.insert(0,'System.IO.RobocopyDirectoryInfo')
Write-Output $object
} Else {
Write-Verbose ("Not matched: {0}" -f $_)
}
} Catch {
Write-Warning ("{0}" -f $_.Exception.Message)
Return
}
}
I have a script set to enter a for each loop every-time a file is created. Once in the loop it will move a file to a another folder and if the same file has to be moved 3 times it will move the file to a different table and remove the record of it from the hash table.
My issue is when I run the script it does not do anything that I write inside the for each loop. Only if I write script above it. Can someone please advise?
$folder = 'C:\Users\jnwankwo\Documents\IUR Test\r' # Enter the root path you want to monitor.
$Failedfolder = 'C:\Users\jnwankwo\Documents\IUR Test\r'
$filter = '*.*' # You can enter a wildcard filter here.
$Files = #{}
$Counter = 1
$folder = 'C:\Users\jnwankwo\Documents\IUR Test\r' # Enter the root path you want to monitor.
$Failedfolder = 'C:\Users\jnwankwo\Documents\IUR Test\r'
$filter = '*.*' # You can enter a wildcard filter here.
$Files = #{}
$Counter = 1
# In the following line, you can change 'IncludeSubdirectories to $true if required.
$fsw = New-Object IO.FileSystemWatcher $folder, $filter -Property #{IncludeSubdirectories = $false;NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'}
Register-ObjectEvent $fsw Created -SourceIdentifier FileCreated -Action {
ForEach ($file in $folder)
{
$fName = $file.Name
if (-not $Files.ContainsKey($fName))
{
$Files.Add($fName,$Counter)
}
if (($Files.ContainsKey($fName)) -and ($Files.Item($fName) -lt 3))
{
Move-Item 'C:\Users\jnwankwo\Documents\IUR Test\r\*.txt' 'C:\Users\jnwankwo\Documents\IUR Test' -force
$Files.Set_Item($fName,$Counter++)
}
ElseIf (($Files.ContainsKey($fName)) -and ($Files.Item($fName) -eq 3))
{
$Files.clear()
Move-Item 'C:\Users\jnwankwo\Documents\Failed\' $Failedfolder -force
}
}
}
# To stop the monitoring, run the following commands:
# Unregister-Event FileCreated
I have found one thing in your code.
Change ForEach ($file in $folder) to ForEach ($file in (gci $folder))
Here you go, you will have to change the folders back though :)
$folder = 'C:\temp\test' # Enter the root path you want to monitor.
$filter = '*.*' # You can enter a wildcard filter here.
# In the following line, you can change 'IncludeSubdirectories to $true if required.
$fsw = New-Object IO.FileSystemWatcher $folder, $filter -Property #{IncludeSubdirectories = $false;NotifyFilter = [IO.NotifyFilters]'FileName,LastWrite'}
Register-ObjectEvent $fsw Created -SourceIdentifier FileCreated -Action {
$folder = 'C:\temp\test' # Enter the root path you want to monitor.
$Failedfolder = 'C:\temp\test3'
$Files = #{}
$Counter = 1
ForEach ($file in (gci $folder))
{
$fName = $file.Name
if (-not $Files.ContainsKey($fName))
{
$Files.Add($fName,$Counter)
}
if (($Files.ContainsKey($fName)) -and ($Files.Item($fName) -lt 3))
{
Move-Item $file.Fullname 'C:\Users\jnwankwo\Documents\IUR Test' -force
$Files.Item($fName) = $Counter++
}
ElseIf (($Files.ContainsKey($fName)) -and ($Files.Item($fName) -eq 3))
{
$Files.clear()
Move-Item $file.Fullname $Failedfolder -force
}
}
}
Addition:
To store your Hashtable to a file and re-import it on the next run you can use the following code:
#store hashtable to file
$Files.GetEnumerator() | % { "$($_.Name)=$($_.Value)" } | Out-File files_ht.txt
#to import again
$Files = Get-Content files_ht.txt | Convertfrom-StringData
This should enable you to have the data from the hashtable persistent
Today we try to list all installed programs on each VM with following script to query WMI.
We find out it will list out all 64 bit applications, plus some of 32 bit applications.
But not all applications (32bit + 64bit) will list out.
param(
[string] $ExportPath = ''
)
$InstalledProducts = get-wmiobject -class Win32_Product
if (($InstalledProducts -ne $null) -and ($InstalledProducts.Count -gt 0))
{
$fileName = ($env:COMPUTERNAME) + "-" + (Get-Date -f "yyyy-mm-dd-hhmmss") + ".csv"
$fileExport = $fileName
if(Test-Path $ExportPath) {
$fileExport = Join-Path (Resolve-Path $ExportPath) $fileName
}
$InstalledProducts |
Select-Object #{Name="HostName"; Expression={"$env:COMPUTERNAME"}}, Name, Version, Vendor |
Export-CSV -Path $fileExport -Encoding UTF8
}
else
{
Write-Host "!!!ERROR!!!"
}
We also try "wmic product" it has the similar issue.
https://superuser.com/questions/681564/how-to-list-all-applications-displayed-from-add-remove-winxp-win7-via-command-li
At last, we need to merge all items both in
HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall
HKLM:\SOFTWARE\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall
Code:
param (
[String] $ExportPath = '<NetworkPath>'
)
$fileName = ($ENV:COMPUTERNAME) + "-" + (Get-Date -f "yyyy-mm-dd-HHmmss") + ".csv"
$fileExport = $fileName
if (Test-Path $ExportPath) {
$fileExport = Join-Path (Resolve-Path $ExportPath) $fileName
}
$UninstallRegList = ('HKLM:\Software\Microsoft\Windows\CurrentVersion\Uninstall\*',
'HKLM:\Software\Wow6432Node\Microsoft\Windows\CurrentVersion\Uninstall\*')
$UninstallRegList |
Get-ItemProperty |
foreach{
if (($_.DisplayName -ne $NULL) -and ($_.DisplayName -ne "")){
$_
}
} | Select-Object DisplayName, DisplayVersion, Publisher |
Export-CSV -Path $fileExport -Encoding UTF8
Here post explains that
The Win32_InstalledSoftwareElement and Win32_Product will only give
you information about software that is installed by Microsoft
Installer.
ref: WMI "installed" query different from add/remove programs list?
With PowerShell (although other suggestions are welcome), how does one recursively loop a directory/folder and
replace text A with B in all files,
rename all files so that A is replaced by B, and last
rename all folders also so that A is replaced by B?
With a few requirements refinements, I ended up with this script:
$match = "MyAssembly"
$replacement = Read-Host "Please enter a solution name"
$files = Get-ChildItem $(get-location) -filter *MyAssembly* -Recurse
$files |
Sort-Object -Descending -Property { $_.FullName } |
Rename-Item -newname { $_.name -replace $match, $replacement } -force
$files = Get-ChildItem $(get-location) -include *.cs, *.csproj, *.sln -Recurse
foreach($file in $files)
{
((Get-Content $file.fullname) -creplace $match, $replacement) | set-content $file.fullname
}
read-host -prompt "Done! Press any key to close."
I would go with something like this:
Get-ChildItem $directory -Recurse |
Sort-Object -Descending -Property { $_.FullName } |
ForEach-Object {
if (!$_.PsIsContainer) {
($_|Get-Content) -replace 'A', 'B' | Set-Content $_.FullName
}
$_
} |
Rename-Item { $_.name -replace 'A', 'B' }
The Sort-Object is there to ensure that first children (subdirs, files) are listed and then directories after them. (12345)
Untested, but should give you a starting point:
$a = 'A';
$b = 'B';
$all = ls -recurse;
$files = = $all | where{ !$_.PSIsContainer );
$files | %{
$c = ( $_ | get-itemcontent -replace $a,$b );
$c | out-file $_;
}
$all | rename-item -newname ( $_.Name -replace $a,$b );
Untested, may be I'm more lucky ;-)
$hello = 'hello'
$world = 'world'
$files = ls -recurse | ? {-not $_.PSIsContainer}
foearch ($file in $files) {
gc -path $file | % {$_ -replace $hello, $world} | Set-Content $file
ri -newname ($file.name -replace $hello, $world)
}
ls -recurse | ? {$_.PSIsContainer} | ri -newname ($_.name -replace $hello, $world)
To use the same recursion:
$hello = 'hello'
$world = 'world'
$everything = ls -recurse
foearch ($thing in $everything) {
if ($thing.PSIsContainer -ne $true) {
gc -path $thing | % {$_ -replace $hello, $world} | Set-Content $thing
}
ri -newname ($thing.name -replace $hello, $world)
}
I needed this for myself and below slightly better version of the script.
I added followings:
Support for verbose parameter so you can actually see what changes script has made.
Ability to specify folders so you can limit changes.
Adding bower.json, txt and md in to include extensions.
Search and replace content first, do rename later.
Do not replace content if search string is not found (this avoids unnecessary change in modified date).
[CmdletBinding(SupportsShouldProcess=$true)]
Param()
$match = "MyProject"
$replacement = Read-Host "Please enter project name"
$searchFolders = #("MyProject.JS", "MyProject.WebApi", ".")
$extensions = #("*.cs", "*.csproj", "*.sln", "bower.json", "*.txt", "*.md")
foreach($searchFolderRelative in $searchFolders)
{
$searchFolder = join-path (get-location) $searchFolderRelative
Write-Verbose "Folder: $searchFolder"
$recurse = $searchFolderRelative -ne "."
if (test-path $searchFolder)
{
$files = Get-ChildItem (join-path $searchFolder "*") -file -include $extensions -Recurse:$recurse |
Where-Object {Select-String -Path $_.FullName $match -SimpleMatch -Quiet}
foreach($file in $files)
{
Write-Verbose "Replaced $match in $file"
((Get-Content $file.fullname) -creplace $match, $replacement) | set-content $file.fullname
}
$files = Get-ChildItem $searchFolder -filter *$match* -Recurse:$recurse
$files |
Sort-Object -Descending -Property { $_.FullName } |
% {
Write-Verbose "Renamed $_"
$newName = $_.name -replace $match, $replacement
Rename-Item $_.FullName -newname $newName -force
}
}
else
{
Write-Warning "Path not found: $searchFolder"
}
}
Note that one change from the answer is that above recurses folder only in specified folders, not in root. If you don't want that then just set $recurse = true.