I manage to generate a csv through PowerShell script on collecting a group of server disk info, but the result output on the csv file require some data massage.
below will be the script for ref:
foreach($pc in $comp)
$diskvalue += Get-WmiObject #Params | Select #{l='drives';e='DeviceID'}, #{l='server',e='SystemName'}, #{Name=”size(MB)”;Expression={“{0:N1}” -f($_.size/1mb)}}, #{Name=”freespace(MB)”;Expression={“{0:N1}” -f($_.freespace/1mb)}}, #{Name=”UsedSpace(MB)”;Expression={“{0:N2}” -f(($_.size - $_.FreeSpace)/1mb)}}
$diskvalue | Export-Csv C:\disk_info\DiskReport.csv -NoTypeInformation
The output csv file on "drives" column will contain:
C:
Yet I would like to remove the ":" at the back that data output.
C
Change:
#{l='drives';e='DeviceID'}
to
#{l='drives';e={"$($_.DeviceID)".Trim(": ")}
Trim(": ") will remove any whitespace and : characters from the DeviceID string
Related
I wish to pipe aws cli output which appears on my screen as text output from a powershell session into a text file in csv format.
I have researched the Export-CSV cmdlet from articles such as the below:
https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/export-csv?view=powershell-7.1
I cannot see how to use this to help me with my goal. From my testing, it only seems to work with specific windows programs, not general text output.
An article on this site shows how you can achieve my goal with unix commands, by replacing spaces with commas.
Output AWS CLI command with filters to CSV without jq
The answer with unix is to use sed at the end of the command like so:
aws rds describe-db-instance-automated-backups --query 'DBInstanceAutomatedBackups[*].{ARN:DBInstanceArn,EarliestTime:RestoreWindow.EarliestTime,LatestTime:RestoreWindow.LatestTime}' --output text | sed -E 's/\s+/,/g'
Export-csv` appears to not be able to do this.
Does anyone know how I might replicate what sed is doing here with powershell?
Here is an example of the output that I would like in csv format:
arn:aws:rds:ap-southwest-2:9711387875370:db:catflow--prod 2019-03-03T09:54:29.402Z 2019-03-05T01:25:53Z
arn:aws:rds:ap-southwest-2:9711387875370:db:xyz-prod-rds-golf 2019-03-01T09:04:31.477Z 2019-03-05T01:28:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-asm-prod-rds-stardb 2019-02-01T09:07:30.648Z 2019-03-05T01:27
:20Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-asm-prod-rds-domaindb 2019-02-02T09:04:30.771Z 2019-03-05T01:28
:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-ctz-prod-rds-datavault 2019-02-26T14:14:30.254Z 2019-03-05T01:29
:13Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-gcp-prod-rds-datavault 2019-02-01T14:05:40.456Z 2019-03-05T01:31
:05Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-conformed-datavault-prod 2019-02-02T14:06:26.050Z 2019-03-
05T01:27:02Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-dqm-datavault-prod 2019-02-01T14:12:05.286Z 2019-03-05T01:26
:53Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-prod-dgc-cde-lineage 2019-03-02T09:54:29.053Z 2019-03-05T01:29
:11Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-rec-prod 2019-02-02T22:09:00.673Z 2019-03-05T01:29:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-serve-prod 2019-03-02T09:54:20.729Z 2019-03-05T01:30:21Z
It's possible that you are working with a tab delimited text file, with no headers.
The tab separator can look like multiple spaces when it is displayed on your screen.
If this is the case, If so, you can actually read this file with import-csv, but you have to use the -header parameter to supply your own field names, and the -delimiter character to use tab as the delimiter. The tab character has to be specified using the backtick escape mechanism.
For details, see the accepted answer to this question.
If you have control over your data feed, there is an alternative. The aws cli interface has an option to format the output in JSON format. That format will be much easier to import into Powershell in a form you can use.
Edit:
The following script uses the mockup provided by Theo, except that the multiple spaces have been replaced by a tab character. It uses ConvertFrom-Csv rather than Import-Csv, but it's the same idea:
$awsReturn = #"
arn:aws:rds:ap-southwest-2:9711387875370:db:catflow--prod 2019-03-03T09:54:29.402Z 2019-03-05T01:25:53Z
arn:aws:rds:ap-southwest-2:9711387875370:db:xyz-prod-rds-golf 2019-03-01T09:04:31.477Z 2019-03-05T01:28:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-asm-prod-rds-stardb 2019-02-01T09:07:30.648Z 2019-03-05T01:27:20Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-asm-prod-rds-domaindb 2019-02-02T09:04:30.771Z 2019-03-05T01:28:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-ctz-prod-rds-datavault 2019-02-26T14:14:30.254Z 2019-03-05T01:29:13Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-gcp-prod-rds-datavault 2019-02-01T14:05:40.456Z 2019-03-05T01:31:05Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-conformed-datavault-prod 2019-02-02T14:06:26.050Z 2019-03-05T01:27:02Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-dqm-datavault-prod 2019-02-01T14:12:05.286Z 2019-03-05T01:26:53Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-prod-dgc-cde-lineage 2019-03-02T09:54:29.053Z 2019-03-05T01:29:11Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-rec-prod 2019-02-02T22:09:00.673Z 2019-03-05T01:29:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-serve-prod 2019-03-02T09:54:20.729Z 2019-03-05T01:30:21Z
"#
$myarray = $awsreturn | ConvertFrom-Csv -header "Prod","DateStart","DateEnd" -delimiter "`t"
$myarray | Format-Table
$myarray | gm
When I ran it in my environment, it produced the following:
Prod DateStart DateEnd
---- --------- -------
arn:aws:rds:ap-southwest-2:9711387875370:db:catflow--prod 2019-03-03T09:54:29.402Z 2019-03-05T01:25:53Z
arn:aws:rds:ap-southwest-2:9711387875370:db:xyz-prod-rds-golf 2019-03-01T09:04:31.477Z 2019-03-05T01:28:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-asm-prod-rds-stardb 2019-02-01T09:07:30.648Z 2019-03-05T01:27:20Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-asm-prod-rds-domaindb 2019-02-02T09:04:30.771Z 2019-03-05T01:28:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-ctz-prod-rds-datavault 2019-02-26T14:14:30.254Z 2019-03-05T01:29:13Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-gcp-prod-rds-datavault 2019-02-01T14:05:40.456Z 2019-03-05T01:31:05Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-conformed-datavault-prod 2019-02-02T14:06:26.050Z 2019-03-05T01:27:02Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-dqm-datavault-prod 2019-02-01T14:12:05.286Z 2019-03-05T01:26:53Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-prod-dgc-cde-lineage 2019-03-02T09:54:29.053Z 2019-03-05T01:29:11Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-rec-prod 2019-02-02T22:09:00.673Z 2019-03-05T01:29:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-serve-prod 2019-03-02T09:54:20.729Z 2019-03-05T01:30:21Z
TypeName: System.Management.Automation.PSCustomObject
Name MemberType Definition
---- ---------- ----------
Equals Method bool Equals(System.Object obj)
GetHashCode Method int GetHashCode()
GetType Method type GetType()
ToString Method string ToString()
DateEnd NoteProperty string DateEnd=2019-03-05T01:25:53Z
DateStart NoteProperty string DateStart=2019-03-03T09:54:29.402Z
Prod NoteProperty string Prod=arn:aws:rds:ap-southwest-2:9711387875370:db:catflow--prod
Lets assume the data returned looks like this mockup (in the question it is strangely formatted):
$awsReturn = #"
arn:aws:rds:ap-southwest-2:9711387875370:db:catflow--prod 2019-03-03T09:54:29.402Z 2019-03-05T01:25:53Z
arn:aws:rds:ap-southwest-2:9711387875370:db:xyz-prod-rds-golf 2019-03-01T09:04:31.477Z 2019-03-05T01:28:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-asm-prod-rds-stardb 2019-02-01T09:07:30.648Z 2019-03-05T01:27:20Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-asm-prod-rds-domaindb 2019-02-02T09:04:30.771Z 2019-03-05T01:28:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-ctz-prod-rds-datavault 2019-02-26T14:14:30.254Z 2019-03-05T01:29:13Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-gcp-prod-rds-datavault 2019-02-01T14:05:40.456Z 2019-03-05T01:31:05Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-conformed-datavault-prod 2019-02-02T14:06:26.050Z 2019-03-05T01:27:02Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-dqm-datavault-prod 2019-02-01T14:12:05.286Z 2019-03-05T01:26:53Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-prod-dgc-cde-lineage 2019-03-02T09:54:29.053Z 2019-03-05T01:29:11Z
arn:aws:rds:ap-southwest-2:9711387875370:db:prod-rec-prod 2019-02-02T22:09:00.673Z 2019-03-05T01:29:40Z
arn:aws:rds:ap-southwest-2:9711387875370:db:-serve-prod 2019-03-02T09:54:20.729Z 2019-03-05T01:30:21Z
"#
Then, you can do this:
# Since I don't know if that is one single string or a string array:
if ($awsReturn -isnot [array]) { $awsReturn = $awsReturn -split '\r?\n' }
# write it to csv file
$awsReturn -replace '\s+', ',' | Set-Content -Path 'WhereEver.csv' -PassThru # PassThru also displays on screen
to get a file that can serve as CSV (although it has no headers or quoted fields)
If you want to use Export-CSV to get a csv file with headers and quoted fields, you need to split the lines and output objects.
Something like this:
# Since I don't know if that is one single string or a string array:
if ($awsReturn -isnot [array]) { $awsReturn = $awsReturn -split '\r?\n' }
# write it to csv file (without headers or quotes values)
$awsReturn | ForEach-Object {
$data = $_ -split '\s+' # in this case we know we have 3 fields
[PsCustomObject]#{
Prod = $data[0]
DateStart = $data[1]
DateEnd = $data[2]
}
} | Export-Csv -Path 'WhereEver.csv' -NoTypeInformation
The WhereEver.csv file will then look like this:
"Prod","DateStart","DateEnd"
"arn:aws:rds:ap-southwest-2:9711387875370:db:catflow--prod","2019-03-03T09:54:29.402Z","2019-03-05T01:25:53Z"
"arn:aws:rds:ap-southwest-2:9711387875370:db:xyz-prod-rds-golf","2019-03-01T09:04:31.477Z","2019-03-05T01:28:40Z"
"arn:aws:rds:ap-southwest-2:9711387875370:db:-asm-prod-rds-stardb","2019-02-01T09:07:30.648Z","2019-03-05T01:27:20Z"
"arn:aws:rds:ap-southwest-2:9711387875370:db:-asm-prod-rds-domaindb","2019-02-02T09:04:30.771Z","2019-03-05T01:28:40Z"
"arn:aws:rds:ap-southwest-2:9711387875370:db:-ctz-prod-rds-datavault","2019-02-26T14:14:30.254Z","2019-03-05T01:29:13Z"
"arn:aws:rds:ap-southwest-2:9711387875370:db:-gcp-prod-rds-datavault","2019-02-01T14:05:40.456Z","2019-03-05T01:31:05Z"
"arn:aws:rds:ap-southwest-2:9711387875370:db:prod-conformed-datavault-prod","2019-02-02T14:06:26.050Z","2019-03-05T01:27:02Z"
"arn:aws:rds:ap-southwest-2:9711387875370:db:prod-dqm-datavault-prod","2019-02-01T14:12:05.286Z","2019-03-05T01:26:53Z"
"arn:aws:rds:ap-southwest-2:9711387875370:db:prod-prod-dgc-cde-lineage","2019-03-02T09:54:29.053Z","2019-03-05T01:29:11Z"
"arn:aws:rds:ap-southwest-2:9711387875370:db:prod-rec-prod","2019-02-02T22:09:00.673Z","2019-03-05T01:29:40Z"
"arn:aws:rds:ap-southwest-2:9711387875370:db:-serve-prod","2019-03-02T09:54:20.729Z","2019-03-05T01:30:21Z"
Can you help me understand why this script wont work?
I need to map tons of printers
$path = 'C:\temp\printers.csv'
Import-Csv -Header ('Printernames') -Path $path
foreach ($Printer in $Printername) {
start \\print01\$Printer
}
It's Looks like it takes the header of the CSV file within everytime it loops.
The Import-Csv cmdlet automatically reads the first line of the input file as the CSV headers. The -Header parameter exists so you can provide custom headers in case your data file comes without headers.
Example:
Consider a file input.csv with the following content:
1,"a",23
2,"b",42
If you read that file normally, the first line would be interpreted as the headers of the CSV:
PS C:\> Import-Csv 'input.csv'
1 a 23
- - --
2 b 42
To import all rows as data rows you provide custom headers via the parameter -Header:
PS C:\> Import-Csv 'input.csv' -Header A,B,C
A B C
- - -
1 a 23
2 b 42
I'm looking for some direction on how to read a file line by line, then copy the line based on a search criteria to a newly created file. Since my description is probably poor, I've tried to illustrate below:
Single Text File Sample:
Name=N0060093G
Name=N0060093H
Name=N400205PW
Name=N400205PX
Name=N966O85Q0
Name=N966O85Q1
The script would read each line and use the "###" after "Name=N", to create a new file name after the identifier, "###" to copy each appropriate line to the new file. So, lines "Name=N0060093G"and "Name=N0060093H" would go to "006.txt"; "Name=N400205PW" and "Name=N400205PX" would write to "400.txt", etc.
A RegEx style approach:
$File = 'test.txt'
Get-Content $File | ForEach {
If ($_ -match '^Name\=N(?<filename>\d{3}).*') {
$_ | Out-File -Append "$($Matches.Filename).txt" -WhatIf
}
}
I need to change the local administrator name and password on servers to those that are contained in a .csv
The CSV file contains a list with all the information in it whereby the Server, Administrator name and Passwords are different on each line
The csv is headed by three columns - Server,Admin,PW
How could this be done using Powershell?
I know i can set them all the same using this but they need to be as per each csv line.
foreach ($strComputer in get-content c:\Servers.txt)
{
$Admin=[adsi]("WinNT://" + $strComputer + "/Administrator, user")
$Admin.psbase.rename("Newname")
$Admin.SetPassword("NewPW")
try this ( not tested ):
import-csv c:\servers.txt | % {
$Admin=[adsi]("WinNT://" + $($_.Server) + "/Administrator, user")
$Admin.psbase.rename($($_.Admin))
$Admin.SetPassword($($_.PW))
$Admin.SetInfo() # I think it's needed
}
you can use the Import-Csv instead of get-content. then you can adress the variables by using the header names.
asuming you have a file like:
Server,Admin,PW
bla1,bla2,bla3
blaA,blaB,blaC
the output of
foreach ($line in Import-Csv c:\Servers.txt) { echo $line.server }
would be:
bla1
blaA
just to complete your code, try this example:
foreach ($line in Import-Csv c:\Servers.txt)
{
$Admin=[adsi]("WinNT://" + $line.Server + "/Administrator, user")
$Admin.psbase.rename($line.Admin)
$Admin.SetPassword($line.PW)
}
How do you create a unix file format in Powershell? I am using the following to create a file, but it always creates it in the windows format.
"hello world" | out-file -filepath test.txt -append
As I understand, the new line characters CRLF make it to be a Windows format file whereas the unix format needs only a LF at the end of the line. I tried replacing the CRLF with the following, but it didn't work
"hello world" | %{ $_.Replace("`r`n","`n") } | out-file -filepath test.txt -append
There is a Cmdlet in the PowerShell Community Extensions called ConvertTo-UnixLineEnding
One ugly-looking answer is (taking input from dos.txt outputting to unix.txt):
[string]::Join( "`n", (gc dos.txt)) | sc unix.txt
but I would really like to be able to make Set-Content do this by itself and this solution does not stream and therefore does not work well on large files...
And this solution will end the file with a DOS line ending as well... so it is not 100%
I've found that solution:
sc unix.txt ([byte[]][char[]] "$contenttext") -Encoding Byte
posted above, fails on encoding convertions in some cases.
So, here is yet another solution (a bit more verbose, but it works directly with bytes):
function ConvertTo-LinuxLineEndings($path) {
$oldBytes = [io.file]::ReadAllBytes($path)
if (!$oldBytes.Length) {
return;
}
[byte[]]$newBytes = #()
[byte[]]::Resize([ref]$newBytes, $oldBytes.Length)
$newLength = 0
for ($i = 0; $i -lt $oldBytes.Length - 1; $i++) {
if (($oldBytes[$i] -eq [byte][char]"`r") -and ($oldBytes[$i + 1] -eq [byte][char]"`n")) {
continue;
}
$newBytes[$newLength++] = $oldBytes[$i]
}
$newBytes[$newLength++] = $oldBytes[$oldBytes.Length - 1]
[byte[]]::Resize([ref]$newBytes, $newLength)
[io.file]::WriteAllBytes($path, $newBytes)
}
make your file in the Windows CRLF format. then convert all lines to Unix format in new file:
$streamWriter = New-Object System.IO.StreamWriter("\\wsl.localhost\Ubuntu\home\user1\.bashrc2")
$streamWriter.NewLine = "`n"
gc "\\wsl.localhost\Ubuntu\home\user1\.bashrc" | % {$streamWriter.WriteLine($_)}
$streamWriter.Flush()
$streamWriter.Close()
not a one-liner, but works for all lines, including EOF. new file now shows as Unix format in Notepad on Win11.
delete original file & rename new file to original, if you like:
ri "\\wsl.localhost\Ubuntu\home\user1\.bashrc" -Force
rni "\\wsl.localhost\Ubuntu\home\user1\.bashrc2" "\\wsl.localhost\Ubuntu\home\user1\.bashrc"
Two more examples on how you can replace CRLF by LF:
Example:
(Get-Content -Raw test.txt) -replace "`r`n","`n" | Set-Content test.txt -NoNewline
Example:
[IO.File]::WriteAllText('C:\test.txt', ([IO.File]::ReadAllText('C:\test.txt') -replace "`r`n","`n"))
Be aware, this does really just replace CRLF by LF. You might need to add a trailing LF if your Windows file does not contain a trailing CRLF.