I have 50 sub-Folders inside a Single Parent Folder.
Inside each sub folders there are multiple .txt files. I want to merge all the text files in a single sub-folder into 1 .txt file.
But I want a command so that it can be done in one go for all the subfolder, like i don't want to write command for each sub-folder.
For example:-
ABCD (Parent Folder ):-
A
B ; Here A and B are sub-folder
A\0001.txt
A\0002.txt
I want to merge and make a single text file A\0001.txt.
B\0001.txt
B\0002.txt
I want to merge both the text files in B Folder.
Can it be done in one go ?
This is probably a lot easier using powershell.
Try the following and change the basedir to the parent folder of all your subdirectories.
$basedir = "C:\Basedir"
$folderlist = Get-childitem -Path $basedir
foreach ($folder in $folderlist)
{
$dir = $folder
$outFile = Join-Path $dir "merged.txt"
# Build the file list
$fileList = Get-ChildItem -Path $dir -Filter File*.txt -File
# Get the header info from the first file
Get-Content $fileList[0] | select -First 2 | Out-File -FilePath $outfile -Encoding ascii
# Cycle through and get the data (sans header) from all the files in the list
foreach ($file in $filelist)
{
Get-Content $file | select -Skip 2 | Out-File -FilePath $outfile -Encoding ascii -Append
}
}
Maybe old but useful: this version works with folders and subfolders recursively:
$basedir = "..."
$folderlist = Get-childitem -Path $basedir -Recurse -Directory | Select-Object FullName
foreach ($folder in $folderlist)
{
Write-Host $folder.FullName
$dir = $folder.FullName
$outFile = Join-Path $basedir "merged.txt"
# Build the file list
$fileList = Get-ChildItem -Path $dir -Filter *.log | Select-Object FullName
# Get the header info from the first file
#Get-Content $fileList[0] | select -First 2 | Out-File -FilePath $outfile -Encoding ascii
# Cycle through and get the data (sans header) from all the files in the list
foreach ($file in $filelist)
{
Write-Host $file.FullName
Get-Content $file.FullName | Out-File -FilePath $outfile -Encoding ascii -Append
}
}
Related
I am trying to create a PowerShell script to fetch the root folder's name where in their subdirectories files with error names are present with today's date. Below is the sample code I have tried so far to pick the folder names.
Root Log folder - C:\Errorlogs, contains many other application log level folders.
$targetDir="C:\Errorlogs"
Get-ChildItem $targetDir -Recurse -ErrorAction SilentlyContinue -Force -Filter "*Error*"|
where {([datetime]::now.Date -eq $_.lastwritetime.Date)} |
select FullName
I have tried the above code; however, it's giving me the whole path as result, whereas I only need the folder name.
Result - C:\Errorlogs\AsyncCreateUsersAPIProcessor\202302\04\Error.txt
Required - AsyncCreateUsersAPIProcessor
Use string LastIndexOf and SubString
$rootPath = "C:\Temp\Errorlogs"
$date = [DateTime]::Now.ToString("yyyyMM\\\\dd")
$pattern = '\\(?<folder>\w+)\\' + $date + '\\Error.*$'
$files = Get-ChildItem -Path $rootPath -Recurse | Select-Object -Property Fullname | Where-Object {$_.Fullname -Match $pattern}
foreach($file in $files)
{
$file.Fullname -match $pattern
Write-Host "folder = " $Matches.folder
}
Looks like you can do it just with splitting the path using \ as delimiter then picking the 3rd token (2nd index of an array):
$targetDir = "C:\Errorlogs"
Get-ChildItem $targetDir -Recurse -ErrorAction SilentlyContinue -Force -Filter "*Error*" |
Where-Object { [datetime]::Now.Date -eq $_.LastWriteTime.Date } |
Select-Object #{ N='Name'; E={ $_.FullName.Split('\')[2] }}
Another option if you want 2 levels up in the folder hierarchy is to query the .Directory property of the file then the .Parent property of the parent folder (2 times or as many times as needed):
$targetDir = "C:\Errorlogs"
Get-ChildItem $targetDir -Recurse -ErrorAction SilentlyContinue -Force -Filter "*Error*" |
Where-Object { [datetime]::Now.Date -eq $_.LastWriteTime.Date } |
Select-Object #{ N='Name'; E={ $_.Directory.Parent.Parent.Name }}
As long as the subfolders inside the folder you are after all have numeric-only names, you can loop backwards to get at the first non-numeric foldername and output that.
$targetDir = "C:\Errorlogs"
Get-ChildItem -Path $targetDir -File -Filter "*Error*" -Recurse -Force -ErrorAction SilentlyContinue |
Where-Object { [datetime]::Now.Date -eq $_.LastWriteTime.Date } | ForEach-Object {
$parentDir = $_.Directory
while ($parentDir.Name -match '^\d+$') { $parentDir = $parentDir.Parent }
$parentDir.Name
}
That way, even a path C:\Errorlogs\AsyncCreateUsersAPIProcessor\202302\02\04\1234\567\Error.txt would produce folder name AsyncCreateUsersAPIProcessor
I have the following problem and I would really appreciate it if I could get some help on that front. I am getting a constant flow of xml files into a folder. A XML file name can look like this. It only goes up to 1005.
1001.order-asdf1234.xml
1002.order-asdf4321.xml
I want to sort the files into uniquely named folders that are not based on the file names. A example for that would be
C:\Directory Path...\Peter (All files starting with 1001 go in there)
C:\Directory Path...\John (All files starting with 1002 go there)
How can I create a batch or a powershell script to continuously sorts files into the specified folders? Since I only have 5 folders I would like to simply specify the target folders for each and not have elaborate loops but I don't know how to do that.
The easiest way is to create a lookup Hashtable where you define which prefix ('1001' .. '1005') maps to which destination folder:
# create a Hasthable to map the digits to a foldername
$folderMap = #{
'1001' = 'Peter'
'1002' = 'John'
'1003' = 'Lucretia'
'1004' = 'Matilda'
'1005' = 'Henry'
}
# set source and destination paths
$rootFolder = 'X:\Where\the\files\are'
$destination = 'Y:\Where\the\files\should\go'
# loop over the files in the root path
Get-ChildItem -Path $rootFolder -Filter '*.xml' -File |
Where-Object { $_.BaseName -match '^\d{4}\.' } |
ForEach-Object {
$prefix = ($_.Name -split '\.')[0]
$targetPath = Join-Path -Path $destination -ChildPath $folderMap[$prefix]
$_ | Move-Item -Destination $targetPath -WhatIf
}
Remove the -WhatIf safety-switch if you are satisfied with the results shown on screen
You could use a switch statement to decide on the target folder based on the first part of the file name:
$files = Get-ChildItem path\to\folder\with\xml\files -Filter *.xml
switch($files)
{
{$_.Name -like '1001*'} {
$_ |Move-Item -Destination 'C:\path\to\Peter'
}
{$_.Name -like '1002*'} {
$_ |Move-Item -Destination 'C:\path\to\John'
}
{$_.Name -like '1003*'} {
# etc...
}
default {
Write-Warning "No matching destination folder for file '$($_.Name)'"
}
}
If you change your mind about loops, my preference would be to store the mapping in a hashtable and loop over the entries for each file:
$files = Get-ChildItem path\to\folder\with\xml\files -Filter *.xml
$targetFolders = #{
'1001' = 'C:\path\to\Peter'
'1002' = 'C:\path\to\John'
'1003' = 'C:\path\to\Paul'
'1004' = 'C:\path\to\George'
'1005' = 'C:\path\to\Ringo'
}
foreach($file in $files){
$targetFolder = $targetFolders.Keys.Where({$file.Name -like "${_}*"}, 'First')
$file |Move-Item -Destination $targetFolder
}
I have hundreds of file in working directory that needs to be processed. It looks similar to this
a.txt
b.txt
c.txt
d.txt
All of these file can be processed manually like this
$lines = Get-Content "a.txt"
foreach ($line in $lines){
Out-File -FilePath "a-done.txt" -Encoding UTF8 -Append -InputObject ($line.Split(","))[0]
}
How to automate this process using loop by passing all filename to variable above.
I have tried foreach loop but it's not working
$lines = Get-Content "path/*.txt"
foreach ($line in $lines){
Out-File -FilePath "$lines-processed.txt" -Encoding UTF8 -Append -InputObject ($line.Split(","))[0]
}
What do I miss here?
To discover the files themselves, you'll want to use Get-ChildItem instead of Get-Content! To reference the file name without the extension (ie. a from a.txt), reference the BaseName property:
foreach($file in Get-ChildItem .\path\ -Filter *.txt){
foreach ($line in $file |Get-Content){
Out-File -FilePath "$($file.BaseName)-done.txt" -Encoding UTF8 -Append -InputObject ($line.Split(","))[0]
}
}
Youre looping through the lines of get-content, not where the filenames are saved. You need probably an extra step e.g.
$items = Get-ChildItem 'C:\Users\Alex\Desktop\oop'
foreach ($item in $items) {
<#your processin with get-content here#>
echo $item.name
echo "$item-processed.txt"
}
I misunderstodd in the first time. I hope I am right now:
You want so save one done-File per one input file.
The Problem with your code is that you are collection all the content of all Files in your $lines-Variable. And there is no Information about the underlying File(-names) any more.
Instead you have to loop over all the files an handle them seperately.
The solution suggested:
$files = dir *.txt -Exclude *done.txt
foreach ($f in $files) {
Get-Content $f | % {$_.split(',')[0]} |
Out-File ($f.DirectoryName + '\' + $f.Basename + '-done.txt') -Encoding UTF8}
Regards Martin
Here's an example of how I combine csv files into one big CSV.
$txtFilter = "D:\Temp\*.csv"
$fileOutputSummary = "D:\Temp\Summary.csv"
$list = Get-ChildItem -Path $txtFilter | select FullName
$iItems = $list.Count
$i = 0
ForEach($file in $list){
$i++
Write-Host "Combining ($i of $iItems) `r"
Write-Progress -Activity "Combining CSV files" -PercentComplete ($i / $iItems*100)
Import-Csv -Path $file.FullName | Export-Csv -Path $fileOutputSummary -Append -NoTypeInformation
Sleep 1
} # end ForEach file
I hope my example helps.
I'm working on a script that checks folders in specific directory. For example, I run the script for first time, it generates me a txt file containing folders in the directory.
I need the script to add any new directories that are found to the previously created txt file when the script is run again.
Does anyone have any suggestions how to make that happen?
Here is my code so far:
$LogFolders = Get-ChildItem -Directory mydirectory ;
If (-Not (Test-Path -path "txtfilelocated"))
{
Add-Content txtfilelocated -Value $LogFolders
break;
}else{
$File = Get-Content "txtfilelocatedt"
$File | ForEach-Object {
$_ -match $LogFolders
}
}
$File
something like this?
You can specify what directory to check adding path to get-childitem cmdlet in first line
$a = get-childitem | ? { $_.psiscontainer } | select -expand fullname #for V2.0 and above
$a = get-childitem -Directory | select -expand fullname #for V3.0 and above
if ( test-path .\list.txt )
{
compare-object (gc list.txt) ($a) -PassThru | Add-Content .\list.txt
}
else
{
$a | set-content .\list.txt
}
I'm trying to copy one file to any subfolder in a directory that has a specific name. I am part way there, but just can't quite get it to work.
I am able to find all of the subfolders called "help" using:
Get-ChildItem -Path Y:\folder1\subfolder -Directory -Recurse | ? { ($_.PSIsContainer -eq $true) -and ($_.Name -like 'help')}
That will get any folder in Y:\folder1\subfolder named help. So have been trying:
$folder = Get-ChildItem -Path Y:Y:\folder1\subfolder -Directory -Recurse | ? { ($_.PSIsContainer -eq $true) -and ($_.Name -like 'help')}
foreach ($f in $folder){
Copy-Item Y:\Info.html -Destination $folder[$f]
}
and that does not work. Bonus points if you can also tell me how to have it write out to a csv file all of the directories it copies the file to.
Thanks
I wrote this with version 3, but I think it will work with 1 and 2 since I used Set-StrictMode -Version <number> to test them.
The CSV output will look something like this for every line: Y:\Info.html,Y:\folder1\subfolder\help
$logpath = 'C:\log.csv'
$logopts = #{filepath=$logpath; append=$true; encoding='ascii'}
$file = 'Y:\Info.html'
$path = 'Y:\folder1\subfolder'
$search = 'help'
gci $path -d -s `
| ?{ $_.psIsContainer -and $_.name -match $search } `
| %{
cp $file $_.fullName; # copy file
$line = $file, $_.fullName -join ','; # build output
$line | out-file #logopts; # write output
}
Version 1
$folders = #(
(gci Y:\folder1\subfolder -dir -r | ? {$_.Name -like 'help'}).fullname
)
ForEach ($f in $folders) {
Copy-Item Y:\Info.html $f
}
Version 2
(gci Y:\folder1\subfolder -dir -r | ? {$_.Name -like 'help'}).fullname | % {cp Y:\Info.html $_}