I currently have a CSV which contains 1 column that lists many file FullNames. (ie. "\\server\sub\folder\file.ext").
I am attempting to import this CSV, move the file to a separate location and append a GUID to the beginning of the filename in the new location (ie GUID_File.ext). I've been able to move the files, generate the GUID_ but haven't been able to store and reuse the existing filename.ext, it just gets cut off and the file ends up just being a GUID_. I just am not sure how to store the existing filename for reuse.
$Doc = Import-CSV C:\Temp\scripttest.csv
ForEach ($line in $Doc)
{
$FileBase = $Line.basename
$FileExt = $Line.extension
Copy-Item -path $line.File -Destination "\\Server\Folder\$((new-guid).guid.replace('-',''))_$($Filebase)$($FileExt)"
}
If possible, I'm going to also need to store and place all the new GUID_File.ext back into a CSV and store any errors to another file.
I currently have a CSV which contains 1 column that lists many file FullNames. (ie. "\server\sub\folder\file.ext").
This isn't a CSV. It's just a plaintext file with a list.
Here's how you can accomplish your goal, however:
foreach ($path in (Get-Content -Path C:\Temp\scripttest.csv))
{
$file = [System.IO.FileInfo]$path
$prefix = (New-Guid).Guid -replace '-'
Copy-Item -Path $file.FullName -Destination "\\Server\Folder\${prefix}_$file"
}
This will take your list, convert the item into a FileInfo type it can work with, and do the rest of your logic.
Based on:
$FileBase = $line.basename
$FileExt = $line.extension
it sounds like you mistakenly think that the $line instances representing the objects returned from Import-Csv C:\Temp\scripttest.csv are [System.IO.FileInfo] instances, but they're not:
What Import-Csv outputs are [pscustomobject] instances whose properties reflect the column values of the input CSV, and the values of these properties are invariably strings.
You must therefore use $line.<column1Name> to refer to the column containing the full filenames, where <column1Name> is the name defined for the column of interest in the header line (the 1st line) of the input CSV file.
If the CSV file has no header line, you can specify the column names by passing an array of column names to Import-Csv's -Header parameter, e.g.,
Import-Csv -Header Path, OtherCol1, OtherCol2, ... C:\Temp\scripttest.csv
I'll assume that the column of interest is named Path in the following solution:
$Doc = Import-Csv C:\Temp\scripttest.csv
ForEach ($rowObject in $Doc)
{
$fileName = Split-Path -Leaf $rowObject.Path
Copy-Item -Path $rowObject.Path `
-Destination "\\Server\Folder\$((new-guid).guid.replace('-',''))_$fileName"
}
Note how Split-Path -Leaf is used to extract the filename, including extension, from the full input path.
If I read your question carefully, you want to:
copy the files listed in the CSV file in the 'File' column.
the new files should have a GUID prepended to the filename
you need a new CSV file where the new filenames are stored for later reference
you want to track any errors and write those to a (log) file
Assuming you have an input CSV file looking something like this:
File,Author,MoreStuff
\\server\sub\folder\file.ext,Someone,Blah
\\server\sub\folder\file2.ext,Someone Else,Blah2
\\server\sub\folder\file3.ext,Same Someone,Blah3
Then below script does hopefully what you want.
It creates new filenames by prepending them with a GUID and copies the files in the CSV listed in column File to some destination path.
It outputs a new CSV file in the destination folder like this:
OriginalFile,NewFile
\\server\sub\folder\file.ext,\\anotherserver\sub\folder\38f7bec9e4c0443081b385277a9d253d_file.ext
\\server\sub\folder\file2.ext,\\anotherserver\sub\folder\d19546f7a3284ccb995e5ea27db2c034_file2.ext
\\server\sub\folder\file3.ext,\\anotherserver\sub\folder\edd6d35006ac46e294aaa25526ec5033_file3.ext
Any errors are listed in a log file (also in the destination folder).
$Destination = '\\Server\Folder'
$ResultsFile = Join-Path $Destination 'Copy_Results.csv'
$Logfile = Join-Path $Destination 'Copy_Errors.log'
$Doc = Import-CSV C:\Temp\scripttest.csv
# create an array to store the copy results in
$result = #()
# loop through the csv data using only the column called 'File'
ForEach ($fileName in $Doc.File) {
# check if the given file exists; if not then write to the errors log file
if (Test-Path -Path $fileName -PathType Leaf) {
$oldBaseName = Split-Path -Path $fileName.Path -Leaf
# or do $oldBaseName = [System.IO.Path]::GetFileName($fileName)
$newBaseName = "{0}_{1}" -f $((New-Guid).toString("N")), $oldBaseName
# (New-Guid).toString("N") returns the Guid without hyphens, same as (New-Guid).Guid.Replace('-','')
$destinationFile = Join-Path $Destination $newBaseName
try {
Copy-Item -Path $fileName -Destination $destinationFile -Force -ErrorAction Stop
# add an object to the results array to store the original filename and the full filename of the copy
$result += New-Object -TypeName PSObject -Property #{
'OriginalFile' = $fileName
'NewFile' = $destinationFile
}
}
catch {
Write-Error "Could not copy file to '$destinationFile'"
# write the error to the log file
Add-content $Logfile -Value "$((Get-Date).ToString("yyyy-MM-dd HH:mm:ss")) - ERROR: Could not copy file to '$destinationFile'"
}
}
else {
Write-Warning "File '$fileName' does not exist"
# write the error to the log file
Add-content $Logfile -Value "$((Get-Date).ToString("yyyy-MM-dd HH:mm:ss")) - WARNING: File '$fileName' does not exist"
}
}
# finally create a CSV with the results of this copy.
# the CSV will have two headers 'OriginalFile' and 'NewFile'
$result | Export-Csv -Path $ResultsFile -NoTypeInformation -Force
Thank you to everyone for the solutions. All of them worked and worked well. I chose Theo as the answer for the fact that his solution solved the error logging and stored all the new renamed files with GUID_File.ext new to the existing CSV info.
Thank you all.
Related
In windows, how can I batch convert base64 file names in a folder to their original names assuming every file name in the folder is encoded with base64
You can do this by iterating the path of the files and try to decode the base64 basenames of those files. If that succeeds, rename the file.
Get-ChildItem -Path 'TheFolderWhereTheFilesAre>' -File | ForEach-Object {
# store the file name for when we hit the catch block
$file = $_.FullName
try {
$newBase = [System.Text.Encoding]::Default.GetString([System.Convert]::FromBase64String($_.BaseName))
$_ | Rename-Item -NewName ('{0}{1}' -f $newBase, $_.Extension) -ErrorAction Stop
}
catch {
Write-Warning "Error renaming file '$file':`r`n$_.Exception.Message"
}
}
I have a bunch of files in folder A and their corresponding metadata files in folder B. I want to loop though the data files and check if the columns are the same in the metadata file, (since incoming data files could have new columns added at any position without notice). If the columns in both files match, no action to is to be taken. If Data file has more columns than metadata file, then those columns should be deleted from incoming data file. Any help would be appreciated. Thanks!
Data file is ps_job.txt
“empid”|”name”|”deptid”|”zipcode”|”salary”|”gender”
“1”|”Tom”|”10″|”11111″|”1000″|”M”
“2”|”Ann”|”20″|”22222″|”2000″|”F”
Meta data file is ps_job_metadata.dat
“empid”|”name”|”zipcode”|”salary”
I would like my output to be
“empid”|”name”|”zipcode”|”salary”
“1”|”Tom”|”11111″|”1000″
“2”|”Ann”|”22222″|”2000″
That's a seemingly simple question with a very complicated answer. However, I've broken down the code for what you will need to do. Here are the steps that need to happen in order for powershell to do everything you're asking of it.
Read the .dat file
Save the .dat data into an object
Read the .txt file
Save the .txt header into an object
Check for the differences
Delete the old text file (that had too many columns)
Create a new text file with the new columns
I've made some assumptions in how this looks. However, with the way I've structured the code, it should be easy enough to make modifications as necessary if my assumptions are wrong. Here are my assumptions:
The text file will always have all of the columns that the DAT file has (even though it will sometimes have more)
The dat file is structured like a text file and can be directly imported into powershell.
And here is the code, with comments. I've done my best to explain the purpose of each section, but I've written this with the expectation that you have a basic knowledge of powershell, especially arrays. If you have questions I'll do my best to answer, though I'll ask that you refer to the section of code you have questions on.
###
### The paths. I'm sure you will have multiples of each file. However, I didn't want to attempt to pull in
### the files with this sample code as it can vary so much in your environment.
###
$dat = "C:\StackOverflow\thingy.dat"
$txt = "C:\stackoverflow\ps_job.txt"
###
### This is the section to process the DAT file
###
# This will read the file and put it in a variable
$dat_raw = get-content -Path $dat
# Now, let's seperate out the punctuation and give us our object
$dat_array = $dat_raw.split("|")
$dat_object = #()
foreach ($thing in $dat_array)
{
$dat_object+=$thing.Replace("""","")
}
###
### This is the section to process the TXT file
###
# This will read the file and put it into a variable
$txt_raw = get-content -Path $txt
# Now, let's seperate out the punctuation and give us our object
$txt_header_array = $txt_raw[0].split("|")
$txt_header_object = #()
foreach ($thing in $txt_header_array)
{
$txt_header_object += $thing.Replace("""","")
}
###
### Now, let's figure out which columns we're eliminating (if any)
###
$x = 0
$total = $txt_header_object.count
$to_keep = #()
While ($x -le $total)
{
if ($dat_object -contains $txt_header_object[$x])
{
$to_keep += $x
}
$x++
}
### Now that we know which objects to keep, we can apply the changes to each line of the text file.
### We will save each line to a new variable. Then, once we have the new variable, we will delete
### The existing file with a new file that has only the data we want.Note, we will only run this
### Code if there's a difference in the files.
if ($total -ne $to_keep.count)
{
### This first section will go line by line and 'fix' the number of columns
$new_text_file = #()
foreach ($line in $txt_raw)
{
if ($line.Length -gt 0)
{
# Blank out the array each time
$line_array = #()
foreach ($number in $to_keep)
{
$line_array += ($line.split("|"))[$number]
}
$new_text_file += $line_array -join "|"
}
else
{
$new_text_file +=""
}
}
### This second section will delete the original file and replace it with our good
### file that has been created.
Remove-item -Path $txt
$new_text_file | out-file -FilePath $txt
}
This small example can be a start for your solution :
$ps_job = Import-Csv D:\ps_job.txt -Delimiter '|'
$ps_job_metadata = (Get-Content D:\ps_job_metadata.txt) -split '\|'-replace '"'
foreach( $d in (Compare-Object $column $ps_job_metadata))
{
if($d.SideIndicator -eq '<=')
{
$ps_job | %{ $_.psobject.Properties.Remove($d.InputObject) }
}
}
$ps_job | Export-Csv -Path D:\output.txt -Delimiter '|' -NoTypeInformation
I tried this and it works.
$outputFile = "C:\Script_test\ps_job_mod.dat"
$sample = Import-Csv -Path "C:\Script_test\ps_job.dat" -Delimiter '|'
$metadataLine = Get-Content -Path "C:\Script_test\ps_job_metadata.txt" -First 1
$desiredColumns = $metadataLine.Split("|").Replace("`"","")
$sample | select $desiredColumns | Export-Csv $outputFile -Encoding UTF8 -NoTypeInformation -Delimiter '|'
Please note that the smart quotes are in consistent over the rows and there are empty lines between the rows (I highly recommend to reformat/update your question).
Anyways, as long as the quoting of the header is consistent between the two (ps_job.txt and ps_job_metadata.dat) files:
# $JobTxt = Get-Content .\ps_job.txt
$JobTxt = #'
“empid”|”name”|”deptid”|”zipcode”|”salary”|”gender”
“1”|”Tom”|”10″|”11111″|”1000″|”M”
“2”|”Ann”|”20″|”22222″|”2000″|”F”
'#
# $MetaDataTxt = Get-Content .\ps_job_metadata.dat
$MetaDataTxt = #'
“empid”|”name”|”zipcode”|”salary”
'#
$Job = ConvertFrom-Csv -Delimiter '|' $JobTxt
$MetaData = ConvertFrom-Csv -Delimiter '|' (#($MetaDataTxt) + 'x|')
$Job | Select-Object $MetaData.PSObject.Properties.Name
“empid” ”name” ”zipcode” ”salary”
------- ------ --------- --------
“1” ”Tom” ”11111″ ”1000″
“2” ”Ann” ”22222″ ”2000″
Here's the same answer I posted to your question on Powershell.org
$jobfile = "ps_job.dat"
$metafile = "ps_job_metadata.dat"
$outputfile = "some_file.csv"
$meta = ((Get-Content $metafile -First 1 -Encoding UTF8) -split '\|')
Class ColumnSelector : System.Collections.Specialized.OrderedDictionary {
Select($line,$meta)
{
$meta | foreach{$this.add($_,(iex "`$line.$_"))}
}
ColumnSelector($line,$meta)
{
$this.select($line,$meta)
}
}
import-csv $jobfile -Delimiter '|' |
foreach{[pscustomobject]([columnselector]::new($_,$meta))} |
Export-CSV $outputfile -Encoding UTF8 -NoTypeInformation -Delimiter '|'
Output
PS C:\>Get-Content $outputfile
"empid"|"name"|"zipcode"|"salary"
"1"|"Tom"|"11111"|"1000"
"2"|"Ann"|"22222"|"2000"
Provided you want to keep those curly quotes and your code page and console font supports all the characters, you can do the following:
# Create array of properties delimited by |
$headers = (Get-Content .\ps_job_metadata.dat -Encoding UTF8) -split '\|'
Import-Csv ps_job.dat -Delimiter '|' -Encoding utf8 | Select-Object $headers
I'm trying to automate renaming of many multiple files in a Windows 7 directory. I need to search a source index file (.txt or .csv which is a list of extended file names) and, where there is a partial match to the original file name, copy the first 12 characters (of the relevant string in the index file) and rename the original file accordingly (and preserving original file extension).
e.g.
(a) Files currently in the Windows directory are named as follows (hundreds of files):
23456abc.doc
76543cab.doc
92837bca.doc
(b) Values in the .txt/.csv file as follows (hundreds of values - NOTE: these do not have file extensions):
BetterName1.RandomText1.23456abc.MoreRandomText1
BetterName2.RandomText2.76543cab.MoreRandomText2
BetterName3.RandomText3.92837bca.MoreRandomText3
(c) Desired Result is for the files to be auto renamed as follows:
[by searching for filename in (a) within the list of values in (b) and, where there is a match, returning the first 12 characters as the new filename whilst preserving the original file extension]
BetterName1.doc
BetterName2.doc
BetterName3.doc
NOTE: My preference is to use an Index file for the look-up that is in .txt format. However in need I can also use a .csv
I have never used PowerShell before and am new to Windows batch scripting. I have searched around and tried to cobble together snippets of code into a Windows batch script (also tried a PowerShell script) to achieve this but my knowledge in this area is seriously lacking so unfortunately I'm still struggling away at square one.
Any assistance would be greatly appreciated. Thank you in advance.
P.S. Here is a PowerShell script that I tried to get working but to no avail.
$fdPath = 'C:\TEST\Data'
$sourcelistFiles = Get-ChildItem -Path $FDPATH\*.txt | ForEach-Object {$_.user } FullName
$findReplaceList = Import-Csv -Path $FDPATH\AllNames.csv
$totalitems = $sourcelistFiles.count
$currentrow = 0
foreach ($sourcelistFile in $sourcelistFiles)
{
$currentrow += 1
Write-Progress -Activity "Processing record $currentrow of $totalitems" -Status "Progress:" -PercentComplete (($currentrow / $totalitems) * 100)
[string] $txtSourceListFile = Get-Content $sourcelistFile | Out-String
ForEach ($findReplaceItem in $findReplaceList)
{
$txtSourceListFile = $txtSourceListFile -replace "$($findReplaceitem.FindString)", "$($findReplaceitem.ReplaceString)"
}
$txtSourceListFile | Set-Content ($sourcelistFile)
-NoNewLine
}
$FDPATH = 'C:\TEST\Data'
foreach($obj in (Import-Csv -Path $FDPATH\AllNames.csv)){
foreach($thing in $(gci -Path $FDPATH\*.txt)){
if("123.hash.avocado" -match $thing.basename){$ret = $thing.fullname}
}
$stuff = $obj -split "."
ren -Path $ret -NewName $stuff[0]
}
See if this works, it iterates through the csv then iterates through the directory to see if the directory's name is in the csv's line that is being iterated, then sets a variable to be the fullname of the file and renames it to the first name before the period.
Import-CSV LISTA.csv -Header newFileName | % { Copy-Item -Path archivo_convert.pdf -Destination "$($_.newfilename).pdf" }
With Powershell i'm trying to split a text file into multiple files using the the beginning of each line as a delimiter
Input file (transfer.txt):
3M|9935551876|11.99|2235641|001|1|100|N|780
3M|1135741031|13.99|8735559|003|1|100|N|145
3M|5835551001|20.50|4556481|002|1|100|N|222
3M|4578420001|33.00|1125785|001|1|100|N|652
8L|00811444243|134148|4064080040|1|02/05/2017 21:15:13|8|170502707|19.85
8L|00811444243|130925|4189133003|1|02/05/2017 21:15:13|8|170502707|4.69
8L|00811444243|136513|4186144003|2|02/05/2017 21:15:13|8|170502707|10.83
Output file (Article.txt):
3M|9935551876|11.99|2235641|001|1|100|N|780
3M|1135741031|13.99|8735559|003|1|100|N|145
3M|5835551001|20.50|4556481|002|1|100|N|222
3M|4578420001|33.00|1125785|001|1|100|N|652
Here's a snippet of my code:
$Path = "D:\BATCH\"
$InputFile = (Join-Path $Path "transfer.txt")
$Reader = New-Object System.IO.StreamReader($InputFile)
while (($Line = $Reader.ReadLine()) -ne $null) {
if ($Line.StartsWith("3M")) {
$OutputFile = "Article.txt"
}
Add-Content (Join-Path $Path $OutputFile) $Line
}
This as a result, creates the same file as the input file. What's wrong with the code?
The below line is the problem. It is outside the If loop and adding the content of each line to the output file. But as I understand, that is not what you want. You want only the content that pass the If condition to be added to the output file. Hence, it needs to be inside the If loop.
Add-Content (Join-Path $Path $OutputFile) $Line
Although I am not too found of this approach because you would be making as many Disk I/O operations as there are lines that pass the if condition. Not very good for scalability.
You can change your code to something like this to reduce number of Disk I/O to just 1.
$out = While (($Line = $Reader.ReadLine()) -ne $null) {
If ($Line.StartsWith("3M")) {
$Line
}
}
$OutputFile = "Article.txt"
Add-Content (Join-Path $Path $OutputFile) $Out
As others have already pointed out, you never change the output file to anything different from "Article.txt", and you write all input lines to the defined output file.
If you want to write the lines of the input file to different files depending on the value of the first field I'd recommend naming the output files after that value. And since you're writing the output with Add-Content I'd also suggest reading the input file via Get-Content for simplicity reasons. Use a StreamReader when performance is an issue (in which case you'll want to use a StreamWriter too), but not just because.
Get-Content $InputFile | ForEach-Object {
$basename, $null = $_.Split('|', 2)
Add-Content (Join-Path $Path "${basename}.txt") $_
}
i am trying to loop through all files no matter the type, in a folder, and change a string with one that is input by the user..
i can do this now, with the code below, but only with one type of file extension..
This is my code:
$NewString = Read-Host -Prompt 'Input New Name Please'
$scriptPath = split-path -parent $MyInvocation.MyCommand.Definition
$InputFiles = Get-Item "$scriptPath\*.md"
$OldString = 'SolutionName'
$InputFiles | ForEach {
(Get-Content -Path $_.FullName).Replace($OldString,$NewString) | Set-Content -Path $_.FullName
}
echo 'Complete'
How do i loop through the files, no matter the extension ?
so no matter if it is a md, txt or cshtml or some other, it will replace the string as instructed.
To get all the files in a folder you can get use Get-ChildItem. Add the -Recurse switch to also include files inside of sub-folders.
E.g. you could rewrite your script like this
$path = 'c:\tmp\test'
$NewString = Read-Host -Prompt 'Input New Name Please'
$OldString = 'SolutionName'
Get-ChildItem -Path $path | where {!$_.PsIsContainer} | foreach { (Get-Content $_).Replace($OldString,$NewString) | Set-Content -Path $_.FullName }
this will first get all the files from inside the folder defined in $path, then replace the value given in $OldString with what the user entered in when prompted and finally save the files.
Note: the scripts doesn't make any difference regarding if the content of the files changed or not. This will cause all files modified date to get updated. If this information is important to you then you need to add a check to see if the files contains the $OldString before changing them and saving.