I wanted to compare two text files using command prompt and I am using two text files with name abc and xyz. I need unique records in other text file. But the output which I get for some string are going on to second line which breaks my urls into two separate lines is there any way to compare them and get output in the same format which in the existing text file.
fc abc.txt xyz.txt > unique.txt
abc File contains data as below
newsroom.associatedbank.com/News-Releases/Associated-Bank-opens-new-Minocqua-branch-5e1.aspx
newsroom.associatedbank.com/content/default.aspx?NewsAreaId=2&SubjectId=75
newsroom.associatedbank.com/content/default.aspx?NewsAreaId=2&SubjectId=76
newsroom.associatedbank.com/content/default.aspx?NewsAreaId=2&SubjectId=202
newsroom.associatedbank.com/News-Releases/Associated-Bank-finances-12M-for-retail-and-residential-projects-5dc.aspx
newsroom.associatedbank.com/News-Releases/Associated-Banc-Corp-completes-purchase-of-risk-and-benefits-consulting-firm-Ahmann-Martin-Co-5db.aspx
newsroom.associatedbank.com/News-Releases/Associated-opens-new-Rochester-branch-5da.aspx
xyz File contains data as below
newsroom.associatedbank.com/News-Releases/Associated-Bank-opens-new-Minocqua-branch-5e1.aspx
newsroom.associatedbank.com/content/default.aspx?NewsAreaId=2&SubjectId=75
newsroom.associatedbank.com/content/default.aspx?NewsAreaId=2&SubjectId=76
newsroom.associatedbank.com/content/default.aspx?NewsAreaId=2&SubjectId=202
newsroom.associatedbank.com/News-Releases/Associated-opens-new-Rochester-branch-5da.aspx
You do not have to download Windows PowerShell 2.0 if you have Windows 7 because it is already installed.
From cmd.exe command line:
powershell Compare-Object -ReferenceObject (Get-Content abc.txt) -DifferenceObject (Get-Content xyz.txt) –IncludeEqual ^| Out-File -FilePath unique.txt -Width 4096
Notes:
remove –IncludeEqual (added for piquancy only);
note that | pipe is ^| escaped to pass it to PowerShell, not to treat it in cmd;
change -Width 4096 to desired output line length (integer range). Any additional characters are truncated, not wrapped. If you omit this parameter, the width is determined by the characteristics of the host. The default for the Windows PowerShell console is 80 (characters);
here is a huge script repository there (the link provided with filter to file manipulation with PowerShell);
here is a Compare-Object Cmdlet reference.
To see SideIndicator output format, omit ^| Out-File ... as follows. You should get truncated output on your screen.
powershell Compare-Object -ReferenceObject (Get-Content abc.txt) -DifferenceObject (Get-Content xyz.txt) –IncludeEqual
Using alias names for Cmdlets and omitting optional parts of PowerShell statements, next command should give the same result:
powershell diff (type abc.txt) (gc xyz.txt) -includeequal
I'd suggest you try
findstr /i /L /x /v /g:xyz.txt abc.txt > unique.txt
which should report any line in abc.txt that isn't present in xyz.txt (/i ignoring case, /L literally, no regex, /x - exact match, not on part-line /v lines which don't match)
Consequently, any lines in abc.txt that don't appear in xyz.txt will be directed to unique.txt (tks JosefZ)
"But the output which I get for some string are going on to second line which breaks my urls into two separate lines"
fc has a bug when a line contains more than 127 characters.
It has been hotfixed for Windows XP and Windows Vista but not for Windows 7.
It does not work correctly in Windows 7 (using either the 32 or 64 bit fc.exe)
when the command compares files that contain any ASCII or UNICODE records that have more than 127 characters in a record.
Source where are known errors in fc.exe for windows 7 published
I have created two test files xxx.txt and yyy.txt which differ at line nnn, but fc/n reports that they differ at line nnn+1. It appears that fc has split an of the earlier line into two lines. Examining the files with a hex editor shows no trace of end of line characters 0D or 0A at the place where fc is splitting the line. For larger files the reported mismatch locations from fc and the actual lines where the mismatches are occuring get badly out of synch.
Is this an already known error in fc, and where is there a published list of such known problems with this program?
...
There are hot fixes for Windows XP and for Windows Vista. I do not see one for Windows 7
.
Article ID: 953930 - The Fc.exe command does not work correctly on a Windows XP-based computer when the two files that you are comparing have the TAB or SPACE character around the 128th byte in a string of characters
http://support.microsoft.com/kb/953930
Article ID: 953932 - The Fc.exe command does not work correctly in Windows Vista or in Windows Server 2008 when the two files that you are comparing have the TAB or SPACE character around the 128th byte in a character string
http://support.microsoft.com/kb/953932
Related
I am trying to extract each line from a CSV that has over 1million (1,000,000) lines, where the first character is a 1.
The 1 in this case, refers to the 1st line of a log. There are several different logs in this file, and I need the first line from all of them. Problem is (as you could understand) 1 is not unique, and can appear in any of the 12 'columns' of data I have in this CSV
Essentially, I would like to extract them all to a new CSV file as well, for further break down.
I know it sounds simple enough, but I cannot seem to get the information I need.
I have searched StackOverflow, Microsoft, Google and my own Tech Team.
PS: Get-Content 'C:\Users\myfiles\Desktop\massivelogs.csv' | Select-String "1" | Out-File "extractedlogs.csv"
The immediate answer is that you must use Select-String '^1 in order to restrict matching to the start (^) of each input line.
However, a much faster solution is to use the switch statement with the -File` option:
$inFile = 'C:\Users\myfiles\Desktop\massivelogs.csv'
$outFile = 'extractedlogs.csv'
& { switch -File $inFile -Wildcard { '1*' { $_ } } } | Set-Content $outFile
Note, however, that the output file won't be a true CSV file, because it will lack a header row.
Also, note that Set-Content applies an edition-specific default character encoding (the active ANSI code page in Windows PowerShell, BOM-less UTF-8 in PowerShell Core); use -Encoding as needed.
Using -Wildcard with a wildcard pattern (1*) speeds things up slightly, compared to -Regex with ^1.
I have a windows batch file that I need some help modifying. I use it to name our new PC's based on their serial number. I can successfully create a simple batch file that runs the command and copy the output, but I would like to add some text in front of the outputted serial number so I can easily name the PC's with our standard format.
I have tried to find a way to add the text before the command but it either breaks the command or doesn't show up all together.
#ECHO OFF
TITLE Serial Number grabber
COLOR 0a
#echo ON
WMIC BIOS GET SERIALNUMBER
#echo off
pause
I want the Output to say something like "ABC-Serialnumber" for example, the output in the CMD line windows would output: ABC-1234XYZ
You can redirect the serial number into a text file. Call PowerShell from your batch script to strip away everything except for the number. Then read that into a var to be read with the desired prefix
set prefix=ABC
WMIC BIOS GET SERIALNUMBER>Serial.txt
powershell -command "(Get-content Serial.txt) -replace 'VMware-| ','' | select-object -skip 1 | Set-content Serial.txt"
set /p serial=<serial.txt
set Computer_Asset=%prefix%-%serial%
echo %Computer_Asset%
The PowerShell line is
reading the file (get-content)
replacing VMWare- and all spaces with null (-replace 'first string to replace|second string, in this case spaces','value to be replaced with'
Skip first line of file (select-object -skip 1)
write the new contents to your file (set-content)
Feed the file contents into a serial var.
Now you just set your asset's name with the prefix and the serial (Set asset_tag=%prefix%-%serial%)
I've written a program that returns keycodes as integers for DOS
but i don't know how to get it's output as a variable.
Note: I'm using MS-DOS 7 / Windows 98, so i can't use FOR /F or SET /P
Does anyone know how i could do that?
A few solutions are described by Eric Pement here. However, for older versions of cmd the author was forced to use external tools.
For example, program tools like STRINGS by Douglas Boling, allows for following code:
echo Greetings! | STRINGS hi=ASK # puts "Greetings!" into %hi%
Same goes for ASET by Richard Breuer:
echo Greetings! | ASET hi=line # puts "Greetings!" into %hi%
One of alternative pure DOS solutions needs the program output to be redirected to the file (named ANSWER.DAT in example below) and then uses a specially prepared batch file. To cite the aforementioned page:
[I]n the batch file we need to be able to issue the command
set MYVAR={the contents of ANSWER.DAT go here}. This is a difficult task, since MS-DOS doesn't offer an easy way to prepend "set MYVAR=" to a file [...]
Normal DOS text files and batch files end all lines with two consecutive bytes: a carriage return (Ctrl-M, hex 0D, or ASCII 13) and a linefeed (Ctrl-J, hex 0A or ASCII 10). In the batch file, you must be able to embed a Ctrl-J in the middle of a line.
Many text editors have a way to do this: via a Ctrl-P followed by Ctrl-J (DOS EDIT with Win95/98, VDE), via a Ctrl-Q prefix (Emacs, PFE), via direct entry with ALT and the numeric keypad (QEdit, Multi-Edit), or via a designated function key (Boxer). Other editors absolutely will not support this (Notepad, Editpad, EDIT from MS-DOS 6.22 or earlier; VIM can insert a linefeed only in binary mode, but not in its normal text mode).
If you can do it, your batch file might look like this:
#echo off
:: assume that the datafile exists already in ANSWER.DAT
echo set myvar=^J | find "set" >PREFIX.DAT
copy PREFIX.DAT+ANSWER.DAT VARIAB.BAT
call VARIAB.BAT
echo Success! The value of myvar is: [%myvar%].
:: erase temp files ...
for %%f in (PREFIX.DAT ANSWER.DAT VARIAB.BAT) do del %%f >NUL
Where you see the ^J on line 3 above, the linefeed should be embedded at that point. Your editor may display it as a square box with an embedded circle.
I've written a program that returns keycodes as integers for DOS
but i don't know how to get it's output as a variable.
Note: I'm using MS-DOS 7 / Windows 98, so i can't use FOR /F or SET /P
Does anyone know how i could do that?
A few solutions are described by Eric Pement here. However, for older versions of cmd the author was forced to use external tools.
For example, program tools like STRINGS by Douglas Boling, allows for following code:
echo Greetings! | STRINGS hi=ASK # puts "Greetings!" into %hi%
Same goes for ASET by Richard Breuer:
echo Greetings! | ASET hi=line # puts "Greetings!" into %hi%
One of alternative pure DOS solutions needs the program output to be redirected to the file (named ANSWER.DAT in example below) and then uses a specially prepared batch file. To cite the aforementioned page:
[I]n the batch file we need to be able to issue the command
set MYVAR={the contents of ANSWER.DAT go here}. This is a difficult task, since MS-DOS doesn't offer an easy way to prepend "set MYVAR=" to a file [...]
Normal DOS text files and batch files end all lines with two consecutive bytes: a carriage return (Ctrl-M, hex 0D, or ASCII 13) and a linefeed (Ctrl-J, hex 0A or ASCII 10). In the batch file, you must be able to embed a Ctrl-J in the middle of a line.
Many text editors have a way to do this: via a Ctrl-P followed by Ctrl-J (DOS EDIT with Win95/98, VDE), via a Ctrl-Q prefix (Emacs, PFE), via direct entry with ALT and the numeric keypad (QEdit, Multi-Edit), or via a designated function key (Boxer). Other editors absolutely will not support this (Notepad, Editpad, EDIT from MS-DOS 6.22 or earlier; VIM can insert a linefeed only in binary mode, but not in its normal text mode).
If you can do it, your batch file might look like this:
#echo off
:: assume that the datafile exists already in ANSWER.DAT
echo set myvar=^J | find "set" >PREFIX.DAT
copy PREFIX.DAT+ANSWER.DAT VARIAB.BAT
call VARIAB.BAT
echo Success! The value of myvar is: [%myvar%].
:: erase temp files ...
for %%f in (PREFIX.DAT ANSWER.DAT VARIAB.BAT) do del %%f >NUL
Where you see the ^J on line 3 above, the linefeed should be embedded at that point. Your editor may display it as a square box with an embedded circle.
I'm not familiar with Windows shell. So, let's say my file is like:
DontAppend this line shouldn't be appended
DontAppend this line shouldn't be either
Some lines
more lines
And I'm appending like this:
type file.txt >> AppendHere.txt
This appends the whole file. How do I make it so it skips lines that begin with "DontAppend"?
The command findstr will let you search for lines not containing a string or regular expression so you can use:
findstr /vrc:"^[^A-Za-z0-9]*DontAppend" file.txt >> AppendHere.txt
The /r option says it should use regular expressions and the caret (^) says it should begin with the string.
Edit: added a filter for non alphanumeric chars that may solve the Unicode issues (Unicode files sometimes have a non-printable indicator characters in the beginning).
Either get grep for windows or you could use Windows' own find command
type so.txt|find /v "DontAppend" >> output.txt
The /v option means output lines that dont match your string.
find works for very simple things like this but any more you will need a real filtering tool like grep