On windows, how would I detect the line ending of a file? - windows

I've seen answers to the questions, but those answers are not from a windows perspective from what I can tell.
Windows uses CR LF, Unix uses LF, Mac uses LF and classic mac uses something else. I don't have the brainpower to tell that somehow, if a file is using a different line ending than what I am typing, I get errors when trying to run the script/program which frankly, don't make much sense. After conversion, the script works just fine.
Is there anyway to preemptively check what line endings a file uses, on Windows?

use a text editor like notepad++ that can help you with understanding the line ends.
It will show you the line end formats used as either Unix(LF) or Macintosh(CR) or Windows(CR LF) on the task bar of the tool.
you can also go to View->Show Symbol->Show End Of Line to display the line ends as LF/ CR LF/CR.

Steps:
From the following link download binaries and dependencies zip files:
http://gnuwin32.sourceforge.net/packages/file.htm
Extract their content under the same directory (merge existing directories).
e.g. under c:\gnuwin32
Then you can execute:
c:\gnuwin32\bin\file.exe my-lf-file.txt
my-lf-file.txt; ASCII text
c:\gnuwin32\bin\file.exe my-crlf-file.txt
my-crlf-file.txt; ASCII text, with CRLF line terminators
Of course you can add c:\gnuwin32\bin to your %PATH% variable, to be able to access it without providing the full path.
UPDATE:
If you have git installed you can launch git-bash and run file command from there.
Or you can install this subsystem, as described in the official Microsoft documentation, and get access to the file command.

I too am looking for a "native" windows scripting solution. So far, just have to read a line or 2 in VB in binary fashion and inspect the characters.
One tool to check "manually" is Notepad++. The status bar has a newline style indicator on the right end next to the file encoding indicator.
It looks like this in version 7.5.6
Other editors with Hex mode can show you also.
In Powershell, this command returns "True" for a Windows style file and "False" for a *nix style file.
(Get-Content '\\FILESERVER0001\Fshares\NETwork Shares\20181206179900.TXT' -Raw) -match "\r\n$"
This came from Matt over here: https://stackoverflow.com/a/35354009/1337544

In a batch file, you can try converting the file to CRLF and checking if its size increases:
rem check-crlf.bat
#echo off
setlocal
call type "%~1" | c:\Windows\System32\find.exe "" /v > "%~1.temp"
set size1=%~z1
rem add 2 in case the file doesn't have a trailing newline, since find will add it
set /a size1plus2=%size1%+2
call :setsize2 "%~1.temp%"
for /f %%a in ('c:\Windows\System32\findstr /R /N "^" "%~1" ^| c:\Windows\System32\find /C ":"') do set lines=%%a
if %size1plus2% equ %size2% (
if %lines% equ 2 (
echo File uses LF line endings!
) else (
echo File uses CRLF or has no line endings!
)
) else (
if %size1% lss %size2% (
echo File uses LF line endings!
) else (
echo File uses CR+LF line endings!
)
)
del "%~1.temp"
exit /b
:setsize2
set size2=%~z1
exit /b
We're handling the special case of a file without a trailing newline, as well as a file with two LF-terminated newlines, which both lead to an increase of 2 bytes.
Usage:
check-crlf.bat file-i-care-about.txt

So the main thing to remember, at least for a computer programmer working on modern software is that any combination of CR and LF, in sequence needs to be treated as a newline. You will almost never see the 'old' mac, which is CR with no LF - I prefer to ignore its relatively minuscule existence.. I tend to use 1-byte file processing, but that is a personal preference (a preference that pays a dividend in this scenario) Show proficiency as a programmer by making your code resilient to line ending format of text files.

Related

set /p or alternative on MS-DOS (Windows ME) [duplicate]

I've written a program that returns keycodes as integers for DOS
but i don't know how to get it's output as a variable.
Note: I'm using MS-DOS 7 / Windows 98, so i can't use FOR /F or SET /P
Does anyone know how i could do that?
A few solutions are described by Eric Pement here. However, for older versions of cmd the author was forced to use external tools.
For example, program tools like STRINGS by Douglas Boling, allows for following code:
echo Greetings! | STRINGS hi=ASK # puts "Greetings!" into %hi%
Same goes for ASET by Richard Breuer:
echo Greetings! | ASET hi=line # puts "Greetings!" into %hi%
One of alternative pure DOS solutions needs the program output to be redirected to the file (named ANSWER.DAT in example below) and then uses a specially prepared batch file. To cite the aforementioned page:
[I]n the batch file we need to be able to issue the command
set MYVAR={the contents of ANSWER.DAT go here}. This is a difficult task, since MS-DOS doesn't offer an easy way to prepend "set MYVAR=" to a file [...]
Normal DOS text files and batch files end all lines with two consecutive bytes: a carriage return (Ctrl-M, hex 0D, or ASCII 13) and a linefeed (Ctrl-J, hex 0A or ASCII 10). In the batch file, you must be able to embed a Ctrl-J in the middle of a line.
Many text editors have a way to do this: via a Ctrl-P followed by Ctrl-J (DOS EDIT with Win95/98, VDE), via a Ctrl-Q prefix (Emacs, PFE), via direct entry with ALT and the numeric keypad (QEdit, Multi-Edit), or via a designated function key (Boxer). Other editors absolutely will not support this (Notepad, Editpad, EDIT from MS-DOS 6.22 or earlier; VIM can insert a linefeed only in binary mode, but not in its normal text mode).
If you can do it, your batch file might look like this:
#echo off
:: assume that the datafile exists already in ANSWER.DAT
echo set myvar=^J | find "set" >PREFIX.DAT
copy PREFIX.DAT+ANSWER.DAT VARIAB.BAT
call VARIAB.BAT
echo Success! The value of myvar is: [%myvar%].
:: erase temp files ...
for %%f in (PREFIX.DAT ANSWER.DAT VARIAB.BAT) do del %%f >NUL
Where you see the ^J on line 3 above, the linefeed should be embedded at that point. Your editor may display it as a square box with an embedded circle.

MS-DOS how to get output of command as variable

I've written a program that returns keycodes as integers for DOS
but i don't know how to get it's output as a variable.
Note: I'm using MS-DOS 7 / Windows 98, so i can't use FOR /F or SET /P
Does anyone know how i could do that?
A few solutions are described by Eric Pement here. However, for older versions of cmd the author was forced to use external tools.
For example, program tools like STRINGS by Douglas Boling, allows for following code:
echo Greetings! | STRINGS hi=ASK # puts "Greetings!" into %hi%
Same goes for ASET by Richard Breuer:
echo Greetings! | ASET hi=line # puts "Greetings!" into %hi%
One of alternative pure DOS solutions needs the program output to be redirected to the file (named ANSWER.DAT in example below) and then uses a specially prepared batch file. To cite the aforementioned page:
[I]n the batch file we need to be able to issue the command
set MYVAR={the contents of ANSWER.DAT go here}. This is a difficult task, since MS-DOS doesn't offer an easy way to prepend "set MYVAR=" to a file [...]
Normal DOS text files and batch files end all lines with two consecutive bytes: a carriage return (Ctrl-M, hex 0D, or ASCII 13) and a linefeed (Ctrl-J, hex 0A or ASCII 10). In the batch file, you must be able to embed a Ctrl-J in the middle of a line.
Many text editors have a way to do this: via a Ctrl-P followed by Ctrl-J (DOS EDIT with Win95/98, VDE), via a Ctrl-Q prefix (Emacs, PFE), via direct entry with ALT and the numeric keypad (QEdit, Multi-Edit), or via a designated function key (Boxer). Other editors absolutely will not support this (Notepad, Editpad, EDIT from MS-DOS 6.22 or earlier; VIM can insert a linefeed only in binary mode, but not in its normal text mode).
If you can do it, your batch file might look like this:
#echo off
:: assume that the datafile exists already in ANSWER.DAT
echo set myvar=^J | find "set" >PREFIX.DAT
copy PREFIX.DAT+ANSWER.DAT VARIAB.BAT
call VARIAB.BAT
echo Success! The value of myvar is: [%myvar%].
:: erase temp files ...
for %%f in (PREFIX.DAT ANSWER.DAT VARIAB.BAT) do del %%f >NUL
Where you see the ^J on line 3 above, the linefeed should be embedded at that point. Your editor may display it as a square box with an embedded circle.

Batch file keeps adding special characters to file name?

This is the first time ive posted here so I apologise if im in the wrong place.
I have a batch file that reads a list of domains from a text file and then does an nslookup ls against them, posting the results in their own text file.
Ive never had a problem with this until recently and I cant for the life of me work out why this has started happening.
All the files are perfect except for the first one! The first file name is always proceeded with "" (without the quotes) These files get read by another program I have written so it tends to cause a problem.
Heres the code that creates the files...
(
del /s /q "D:\Profile\Desktop\New_folder\Records\*.*"
for /f %%a in (D:\Profile\Desktop\New_folder\Domains\Domains.txt) do (
echo ls %%a >temp\tempfile.txt
echo exit >>temp\tempfile.txt
nslookup < temp\tempfile.txt > records\%%a.txt
)
)
Any help is much appreciated.
Cheers,
Aaron
According to IBM Extendend Characterset the characters you mentioned have the hex codes EF BB BF which is the UTF-8 byte order mark ("BOM"), see Wikipedia. This means that the file Domain.txt seems to have been saved using UTF-8 character encoding with BOM recently.
In order to get rid of the characters, simply edit the file and save it without a BOM. See e.g. to How to make Notepad to save text in UTF-8 without BOM? how to do that or search for "remove BOM"
Note that UTF-8 without BOM is compatible to printable ASCII, i.e. "normal" characters encoded as UTF-8 will show correctly in most common charactersets such as IBM Extended Characterset.
If you cannot or do not want to edit the input file then you might get rid of the prefix in your batch script, see Substrings in http://www.robvanderwoude.com/ntset.php#StrSubst - eventually something like
set BOM_REMOVED=false
for ...
set X=%%a
if %BOM_REMOVED%==false set X=%X:~3%
set BOM_REMOVED=true
echo ls %X >temp\tempfile.txt
...

Windows batch: Remove lines from a merged file

I am currently making a batch file that merges both, the output from systeminfo, and ipconfig:
#ECHO OFF
pause
systeminfo > "%computername% SystemInfo.txt"
ipconfig >> "%computername% SystemInfo.txt"
"%computername% systeminfo.txt"
The code runs fine and nicely, also independently from OS version and OS language as far as I can tell. My problem though, lies with the systeminfo dump. It lists all 100+ hotfixes that have ever been installed in the machine that is runs on, making the txt file barely legible:
<useful info>
[01]: File 1
[02]: File 1
[03]: File 1
[04]: File 1
....
[150]: file 1
etc...
<useful info>
There's also another problem, namely that this batch file has to run on computers that either run Dutch windows or English windows, meaning that I can't filter on words, because those hotfixes and the words will be different on every computer. Anybody have a nice sollution to this problem.
Note: I have seen it solved the other way around, leaving only the relevant info using findstr. But, because that depends on the language, it is not a viable option.
Edit: The hotfixes are named differently on different OS'es as well, meaning that I can't filter on those. Example: on the XP SP3 I tested, most of the list will be compromised of hotfixes called "[##]file1" on vista however, you will see hex values in the list.
EDIT
My original answer did not work, but I have another idea that works as long as the number and order of each systeminfo header is consistent. I am relying on the fact that the Hotfix(s): is always the 31st header.
#echo off
setlocal enableDelayedExpansion
>systemInfo.txt (
set cnt=0
for /f "delims=" %%A in ('systeminfo') do (
set "ln=%%A"
if "!ln:~0,1!"==" " (if !cnt! neq 31 echo !ln!) else (
echo !ln!
set /a cnt+=1
)
)
ipconfig
)
If the number and/or order of the headers can change, then I don't see how there can be a solution, other then to bite the bullet and look for the specific header text, accounting for all languages that you need to support.
Original failed answer
I don't know how reliable this is. It works for me on my machine, but it would not surprise me if on some machines it strips things it shouldn't.
>systemInfo.txt (
systeminfo|findstr /vxrc:" \[[0-9]*\]: [^ ]*"
ipconfig
)
All my hotfixes begin with KB, followed by a string of numbers. If this is always true, then the above could be improved as:
>systemInfo.txt (
systeminfo|findstr /vxrc:" \[[0-9]*\]: KB[0-9]*"
ipconfig
)
I don't want to figure it out, but you could format it based on the csv output from systeminfo.
systeminfo /fo csv > info.csv
The output, for any language will basically be:
(headers)"<col>", "<col>", "<col>" <...> "<col>"<newline>
(data)"<col>", "<col>", "<col>" <...> "<col>"
The hotfix column is the second to last column. so you could split it by quotes, and ignore that field. It'll have a bunch of crap in it, but it will still be "hotfix, hotfix, hotfix," so you can just remove the whole thing in quotes.
String manipulation in batch is awful if you ask me. If this were me, I'd do it in a language with a string library and call that instead.

Is it possible to put a new line character in an echo line in a batch file? [duplicate]

This question already has answers here:
How can I echo a newline in a batch file?
(24 answers)
Closed 7 years ago.
Is it possible to put a new line character in an echo line in a batch file?
Basically I want to be able to do the equivalent of:
echo Hello\nWorld
You can do this easily enough in Linux, but I can't work out how to do it in Windows.
echo. prints an empty line.
Example:
echo Hello
echo.
echo world
prints
Hello
world
It can be solved with a single echo.
You need a newline character \n for this.
There are multiple ways to get a new line into the echo
1) This sample use the multiline caret to add a newline into the command,
the empty line is required
echo Hello^
world
2) The next solution creates first a variable which contains one single line feed character.
set \n=^
rem ** Two empty lines are required
Or create the new line with a slightly modified version
(set \n=^
%=DONT REMOVE THIS=%
)
And use this character with delayed expansion
setlocal EnableDelayedExpansion
echo Hello!\n!world
To use a line feed character with the percent expansion you need to create a more complex sequence
echo Hello^%\n%%\n%world
Or you can use the New line hack
REM Creating a Newline variable (the two blank lines are required!)
set \n=^
set NL=^^^%\n%%\n%^%\n%%\n%
REM Example Usage:
echo There should be a newline%NL%inserted here.
But only the delayed expansion of the newline works reliable, also inside of quotes.
After a little experimentation I discovered that it is possible to do it without issuing two separate echo commands as described in How can you echo a newline in batch files?. However to make it work you will need a text editor that does not translate CR to CR+LF.
Type:
#echo First Line
then with NumLock on, hold down the ALT key and type 10 on the numeric keypad before releasing ALT (you must use the numeric keypad, not the top-row number keys). This will insert a CR character. Then type the second line. Depending on your editor and how it handles CR compared with CR+LF you may get:
#echo First Line◙Second Line
or
#echo First Line
Second Line
This works from the command line and will work in a batch file so long as the text editor does not translate CR to CR+LF (which Windows/DOS editors do unless you configure them not to). If the CR is converted to CR+LF, or if you use just LF, the second line is interpreted as a new command.
However, I cannot see why this would be preferable over simply:
#echo First Line
#echo Second Line
Ahaha,
I think I've worked out something close enough...
echo hello&echo.&echo world
Produces:
hello
world
echo.
or
echo(
will do the blank new line. Hope this is helpful.
I found this very informative, so wanted to post a better example using the answers provided
This provides a nicely formatted usage message
if "%1" == """" goto usage
:usage
echo USAGE: %0 [Set properties using -D flag] [Ant Task to Run] &
echo. &
echo Availble Command line properties &
echo -------------------------------- &
...
I think it is not possible. You could only try an ascii-character to this:
http://www.asciitable.com/
But this will perhaps crash your batch-file.

Resources