I have this script that stores filenames into a text file:
Get-ChildItem -Path "C:\Users\MyUser\Documents\Files" -Name |
Out-File "C:\Users\MyUser\Documents\Files\FileList.txt"
The result is:
file1.txt
file2.txt
file3.txt
file4.txt
I would like it to produce a list of PowerShell statements in a text file so that I could use them later like this:
Add-Content -Path "C:\Users\MyUser\Documents\Files\file1.txt" -Value "foo"
Add-Content -Path "C:\Users\MyUser\Documents\Files\file2.txt" -Value "foo"
Add-Content -Path "C:\Users\MyUser\Documents\Files\file3.txt" -Value "foo"
Add-Content -Path "C:\Users\MyUser\Documents\Files\file4.txt" -Value "foo"
Is it possible to concatenate a string to the file name the way I need?
I am also not sure whether I understand your question correctly but maybe this is what you are looking for:
Get-ChildItem -Path "C:\Users\MyUser\Documents\Files" |
% {
"Add-Content -Path `"{0}`" -Value `"foo`"" -f $_.FullName
} | Out-File "C:\Users\MyUser\Documents\Files\FileList.txt"
I'm a little unsure about what you're asking, but if I have it correct you want to get a list of files, add a line to the end of the file, and put the names of the files in text file.
Get-ChildItem -Path "C:\Users\MyUser\Documents\Files" |
ForEach-Object {
$_ | Add-Content -Value "foo"
$_.Name
} | Out-File "C:\Users\MyUser\Documents\Files\FileList.txt"
This is one possible way. It passes each item to ForEach-Object, where a value is added to the file, and the file name is sent to the pipeline. That gets piped into Out-File which writes the list.
There are many ways to achieve this; it's best to study and play with pipelining and then look at the various cmdlets and how they handle pipeline input.
Related
I am having a helluva time trying to understand why this script is not working as intended. It is a simple script in which I am attempting to import a CSV, select a few columns that I want, then export the CSV and copy over itself. (Basically we have archived data that I only need a few columns from for another project due to memory size constraints). This script is very simple, which apparently has an inverse relationship with how much frustration it causes when it doesn't work... Right now the end result is I end up with an empty csv instead of a csv containing only the columns I selected with Select-Object.
$RootPath = "D:\SomeFolder"
$csvFilePaths = Get-ChildItem $RootPath -Recurse -Include *.csv |
ForEach-Object{
Import-CSV $_ |
Select-Object Test_Name, Test_DataName, Device_Model, Device_FW, Data_Avg_ms, Data_StdDev |
Export-Csv $_.FullName -NoType -Force
}
Unless you read the input file into memory in full, up front, you cannot safely read from and write back to the same file in a given pipeline.
Specifically, a command such as Import-Csv file.csv | ... | Export-Csv file.csv will erase the content of file.csv.
The simplest solution is to enclose the command that reads the input file in (...), but note that:
The file's content (transformed into objects) must fit into memory as a whole.
There is a slight risk of data loss if the pipeline is interrupted before all (transformed) objects have been written back to the file.
Applied to your command:
$RootPath = "D:\SomeFolder"
Get-ChildItem $RootPath -Recurse -Include *.csv -OutVariable csvFiles |
ForEach-Object{
(Import-CSV $_.FullName) | # NOTE THE (...)
Select-Object Test_Name, Test_DataName, Device_Model, Device_FW,
Data_Avg_ms, Data_StdDev |
Export-Csv $_.FullName -NoType -Force
}
Note that I've used -OutVariable csvFiles in order to collect the CSV file-info objects in output variable $csvFiles. Your attempt to collect the file paths via $csvFilePaths = ... doesn't work, because it attempts to collects Export-Csv's output, but Export-Csv produces no output.
Also, to be safe, I've changed the Import-Csv argument from $_ to $_.FullName to ensure that Import-Csv finds the input file (because, regrettably, file-info object $_ is bound as a string, which sometimes expands to the mere file name).
A safer solution would be to output to a temporary file first, and (only) on successful completion replace the original file.
With either approach, the replacement file will have default file attributes and permissions; if the original file had special attributes and/or permissions that you want to preserve, you must recreate them explicitly.
As Matt commented, your last $PSItem ($_) not related to the Get-ChildItem anymore but for the Select-Object cmdlet which don't have a FullName Property
You can use differnt foreach approach:
$RootPath = "D:\SomeFolder"
$csvFilePaths = Get-ChildItem $RootPath -Recurse -Include *.csv
foreach ($csv in $csvFilePaths)
{
Import-CSV $csv.FullName |
Select-Object Test_Name,Test_DataName,Device_Model,Device_FW,Data_Avg_ms,Data_StdDev |
Export-Csv $csv.FullName -NoType -Force
}
Or keeping your code, add $CsvPath Variable containing the csv path and use it later on:
$RootPath = "D:\SomeFolder"
Get-ChildItem $RootPath -Recurse -Include *.csv | ForEach-Object{
$CsvPath = $_.FullName
Import-CSV $CsvPath |
Select-Object Test_Name,Test_DataName,Device_Model,Device_FW,Data_Avg_ms,Data_StdDev |
Export-Csv $CsvPath -NoType -Force
}
So I have figured it out. I was attempting to pipe through the Import-Csv cmdlet directly instead of declaring it as a variable in the o.g. code. Here is the code snippet that gets what I wanted to get done, done. I was trying to pipe in the Import-Csv cmdlet directly before, I simply had to declare a variable that uses the Import-Csv cmdlet as its definition and pipe that variable through to Select-Object then Export-Csv cmdlets. Thank you all for your assistance, I appreciate it!
$RootPath = "\someDirectory\"
$CsvFilePaths = #(Get-ChildItem $RootPath -Recurse -Include *.csv)
$ColumnsWanted = #('Test_Name','Test_DataName','Device_Model','Device_FW','Data_Avg_ms','Data_StdDev')
for($i=0;$i -lt $CsvFilePaths.Length; $i++){
$csvPath = $CsvFilePaths[$i]
Write-Host $csvPath
$importedCsv = Import-CSV $csvPath
$importedCsv | Select-Object $ColumnsWanted | Export-CSV $csvPath -NoTypeInformation
}
I want to check files for integrity with a checksum. To make it easier I put the hash into an alternate data stream of the file. When someone alters the file I can verify this with the checksum.
However, when I add a data stream the file's LastWriteTime gets updated, so I added functionality to reverse it.
It works like a charm - mostly. But it fails with some files, about 5%. I have no idea why. It looks like it fails with file names that contain spaces or extra dots, but many other that have spaces and multiple dots in the file name work just fine.
Does anyone know what's going on, how to prevent these failures or how to improve the code?
Thanks!
The code:
$filenames = Get-ChildItem *.xl* -Recurse | % { $_.FullName }
foreach( $filename in $filenames ) { ForEach-Object { $timelwt = Get-ItemProperty $filename | select -expand LastWriteTime | select -expand ticks } {add-content -stream MD5 -value (Get-FileHash -a md5 $filename).hash $filename } { Set-ItemProperty $filename -Name LastWriteTime -Value $timelwt}}```
Your code can be reduced to this:
Get-ChildItem *.xl* -Recurse | ForEach-Object {
$lastWriteTime = $_.LastWriteTime
$_ | Add-Content -Stream MD5 -Value ($_ | Get-FileHash -a md5).Hash
$_.LastWriteTime = $lastWriteTime
}
Get-ChildItem with the -Filter you have in place will return FileInfo objects, which have a settable LastWriteTime property, there is no reason for using Get-ItemProperty nor Set-ItemProperty over them.
As for, why your code could be failing, the likeable explanation is that you have some file paths with wildcard metacharacters, and since you're not using -LiteralPath, the cmdlets are defaulting to the -Path parameter (which allows wildcard metacharacters).
As aside, I would personally recommend you to create a separate checksum file for the files instead of adding an alternative data stream.
consider I have a below CSV file.
input:
ID;ITEM_ID;STATUS;
001;;RELEASED;
002;36530;RELEASED;
003;86246;RELEASED;
004;;RELEASED;
I want to remove the row that has ;; (ITEM_ID) missing and save it.I tried doing it on one sample file and it worked as expected.
Import-Csv -Path ".\TestFile.CSV" | where {$_.ITEM_ID -ne ""} | Export-Csv -Path ".\TestFile-temp.CSV" -NoTypeInformation
Remove-Item -Path '.\TestDir\TestFile.csv'
Rename-Item -Path '.\TestDir\TestFile-temp.csv' -NewName 'TestFile.csv'
output:
ID;ITEM_ID;STATUS;
002;36530;RELEASED;
003;86246;RELEASED;
The challenge is, i have multiple csv files and it doesn't has value in different columns, but in single column when i opened in excel file.
so it's not taking the condition < where {$_.ITEM_ID -ne ""} >.
Now i have to search/parse each row of each csv file, search special character (;;) in that row and delete the line and save the file.
i am good at shell scripting but, i am very new to powershell scripting. can anybody please help me to get the logic here or use other cmdlet that can do the job?
$fileDirectory = "C:\Users\Administrator\Documents\check";
foreach($file in Get-ChildItem $fileDirectory)
{
$csvFileToCheck = Import-Csv -Path $fileDirectory\$file
$noDoubleSemiComma = foreach($line in $csvFileToCheck)
{
if(Select-String << i want the logic here>>)
{
$line
}
}
$noDoubleSemiComma | Export-Csv -Path $fileDirectory\tmp.csv -NoTypeInformation
Remove-Item -Path $fileDirectory\$file
Rename-Item -Path $fileDirectory\tmp.csv -NewName $file
}
As commented, you need to add parameter -Delimiter ';' to the cmdlet otherwise a comma is used to parse the fields in the CSV.
As I understand, you also want to remove the quotes Export-Csv outputs around all fields and headers and for PowerShell version 7 you have the option to use parameter -UseQuotes AsNeeded.
As this is not available for version 5.1, I made a function ConvertTo-CsvNoQuotes some time ago to remove the quotes in a safe way. (simply replacing them all with an empty string is dangerous, because sometimes values do need quotes)
Copy that function into your script at the top, then below that, your code could be simplified like this:
$fileDirectory = "C:\Users\Administrator\Documents\check"
Get-ChildItem -Path $fileDirectory -Filter '*.csv' -File | ForEach-Object {
# for better readability store the full path of the file in a variable
$filePath = $_.FullName
(Import-Csv -Path $filePath -Delimiter ';') | ConvertTo-CsvNoQuotes -Delimiter ';' | Set-Content $filePath -Force
Write-Host "File '$filePath' modified"
}
After all helpful suggestion, i finally nailed it down. AS my power-shell version was 5.1 , i had to use logic for trimming double quotes after export-csv. Powershell version 7 and later has -UseQuotes that could have solve that too.
Hope this help others.
$fileDirectory = "C:\Users\Administrator\Documents\check";
foreach($file in Get-ChildItem $fileDirectory)
{
Import-Csv -Path $fileDirectory\$file -Delimiter ';' | where {$_..ITEM_ID -ne ""} | Export-Csv -Path $fileDirectory\temp.csv -Delimiter ';' -NoTypeInformation
$Test = Get-Content $fileDirectory\temp.csv
$Test.Replace('";"',";").TrimStart('"').TrimEnd('"') | Out-File $fileDirectory\temp.csv -Force -Confirm:$false
Remove-Item -Path $fileDirectory\$file
Rename-Item -Path $fileDirectory\temp.csv -NewName $file
Write-Output "$file file modified."
}
Any suggestion to trim down number of lines of code is welcomed.
While trying to transfer file from Windows to Unix Azure environment, I am getting error dos2unix format error
dos2unix -o /xyz/home/ABC_efg.txt failed to execute dos2unix format change.
I tried to run a PS script to fix it but does seem to work .
Get-ChildItem -File -Recurse *.txt | % { $x = get-content -raw -path $_.fullname; $x -replace "`r`n","`n" | set-content -NoNewline -path $_.fullname }
Instead of using -replace, I would prefer to read the content(s) as string array and join these strings with "`n".
Something like this:
$files = Get-ChildItem -File -Recurse -Filter '*.txt' | Select-Object -ExpandProperty FullName
$files | ForEach-Object {
(Get-Content -Path $_) -join "`n" | Set-Content -Path $_ -NoNewline -WhatIf
}
Remove the -WhatIf switch if you are satisfied with the outout shown in the console.
Well, part of the issue is that you are piping a string to Set-Content and then trying to use that string to determine where to save the file. Try changing the last part from:
$x -replace "`r`n","`n" | set-content -NoNewline -path $_.fullname
to this:
set-content -NoNewline -path $_.fullname -value ($x -replace "`r`n","`n")
If that doesn't update the formatting like you expect it to you may need to use the -Encoding parameter for Set-Content. I'm not real familiar with encoding though, so I am not sure about that.
I have multiple machines uploading files to one FTP directory. The first part of the filename is the machine, the rest is a timestamp, e.g. AAAAA_20130312_125113.
Now I want to get a sorted list of all Unique machines that have uploaded to this directory.
I managed to write the lost of all filenames.substring(0,5) to the host but I still don't have the unique machine names.
$files=Get-ChildItem $strMOVETO -Name -Include TAS*.csv -Recurse
ForEach ($i in $files) { Write-Host $i.Substring(0,5) }
Any hints on how to do this? Does not necessary have to be a one liner, although that would be a nice challenge ;-).
Thanks!
What happens when you have an 8-character machine name? Your substring will break. Since the machine name, date & time are delimited by an _, split on that & get the first item.
Get-ChildItem $strMOVETO -recurse -name -include TAS*.csv|%{$_.split("_")[0]}|sort-object -unique
To filter on date as well:
Get-ChildItem $strMOVETO -recurse -include TAS*.csv|where-object{$_.lastwritetime -ge (get-date).adddays(-1)}|%{$_.basename.split("_")[1]}|sort-object -unique
Not tested but something like this:
Get-ChildItem $strMOVETO -Name -Include TAS*.csv -Recurse | % { $_.Name.Substring(0,5) } | Sort -Unique
You don't need to do the Write-Host inside the loop and it's easier to use % instead of a foreach loop.
pipe the results of your command into a | sort -unique
$files=Get-ChildItem $strMOVETO -Name -Include TAS*.csv -Recurse
ForEach ($i in $files) { Write-Host $i.Substring(0,5) } | sort -unique
...but better still would be to simplify the script...
$filter = "TAS*.csv"
Get-ChildItem -Path $strMOVETO -Filter $filter -Recurse | % {$_.BaseName.Substring(0,5) } | sort -unique