Hi I want to set value InFile for all text files in a directory which means that I want the batch file to load and do the command for all of the text files in the directory one by one.
Normally I just copy the command how many times files are then I replace Infile with every file name. By this code I get the file names and then I replace them.
cd /d "dir"
dir /a /b /-p /o:gen >names.txt
and here's the example of a command.
#Echo OFF
REM Set These Variables
SET "InFile=123.txt"
SET "OutFile=NowLoad.txt"
SET "Replace=KK"
SET "ReplaceWith=JJ"
REM Get Total Lines Number [including empty lines]
FOR /F %%A IN ('TYPE "%InFile%"^|find /v /c ""') DO SET "Till=%%A"
REM Create The OutFile with changes
SETLOCAL EnableDelayedExpansion
<"!InFile!" (
FOR /L %%a IN (1 1 0) DO SET /p "="
FOR /L %%A IN (1 1 %Till%) DO (
SET "line="
SET /P "line="
IF "!line!x" == "x" ( Echo.
) ELSE ( Echo !line:%Replace%=%ReplaceWith%!)
)
)>>"%OutFile%"
ENDLOCAL
no need to count the lines, just process every line:
setlocal enabledelayedexpansion
for %%f in (*.txt) do (
( for /f "delims=" %%A in (%%f) do (
set line=%%A
set line=!line:%Replace%=%ReplaceWith%!
echo(!line!
)
)>%%~nf.out
)
Note: the ( after echo handles empty lines (an empty line is written with no "echo is off" if !line! is empty)
edited to process all .txt files in the Folder. Because a fixed %oufile% would overwrite the same outfile with every processed infile, I changed it to a new Extension: <SameNameAsInputFile>.out
Another way...
Your script is DOIT.bat.
Change DOIT.bat to use a command line parameter for the filename you want to process.
#Echo OFF
REM Set These Variables
SET "InFile=%~1"
SET "OutFile=NowLoad.txt"
...
Create another file named CALLIT.bat
#ECHO OFF
FOR /F "usebackq tokens=*" %%f IN (`dir /b *.txt`) DO (
CALL DOIT.bat "%%~f"
)
Running CALLIT.bat will process every .txt file in the directory.
Related
The script I had been working on reads all folders on the CD-ROM drive "i" and searches for mp3 files, copying them onto the destination "e:\MP3\new".
#ECHO off
setlocal enabledelayedexpansion
cd /d "i:\"
set count=1
for /r %%d in (*.mp3) do (
set /a count+=1
)
echo There were %count% files found
set countb=1
for /r %%g in (*.mp3) do (
set /a countb+=1
echo|set /p = File: !countb!/%count%
copy "%%g" "e:\MP3\new" > nul
)
endlocal
Let´s suppose in this example that 115 files were found.
What I get from the above code:
There were 115 files found
File: 1/115 File: 2/115 File: 3/115 File: 4/115 File: 5/115 (...) File: 115/115
What I want:
There were 115 files found
File: X/115 where X will be constantly updated on the screen each time an mp3 file is successfully copied
Any help to fix that?
Firstly, you are initialising your counters wrongly, they should be set to zero (like set count=0) rather than one to get the correct numbers.
Secondly, you should replace echo|set /p = by < nul set /P =, because the pipe (|) is slower than simple (input) redirection (<) since it creates new cmd instances for either side.
To move the cursor in the Command Prompt window back to the beginning of the current line, you need to write the carriage-return character first. However, you cannot use this character as the first one with set /P, because it is going to be removed, together with other leading white-space characters. So you need another invisible character preceding the carriage-return that is not going to be removed; let us choose the back-space character.
Therefore, the fixed code may look like this, for instance:
#echo off
setlocal EnableDelayedExpansion
rem // Gather the back-space character:
for /F %%B in ('prompt $H ^& for %%Z in ^(.^) do rem') do set "BS=%%B"
rem // Gather the carriage-return character:
for /F %%C in ('copy /Z "%~f0" nul') do set "CR=%%C"
cd /D "I:\"
set /A "count=0"
for /R %%d in ("*.mp3") do (
set /A "count+=1"
)
echo There were %count% files found.
set /A "index=0"
for /R %%g in ("*.mp3") do (
set /A "index+=1"
< nul set /P ="%BS%!CR!File: !index!/%count%"
copy "%%~g" "E:\MP3\new\" > nul
)
endlocal
I need a big help from the community, please if somebody can give me some hints. I have the following windows batch script which is supposed to read more than 10 million records as different CSV files and merge them all together. I am running the script on the server. So it's not very slow. But the problem is that the code doesn't handle duplicated records. I am not sure how to change the script in order to handle the duplication records and only passed unique records. I would be very very appreciated for your help.
rem Set current working directory to Task folder
set FilePath=%~dp0
set FolderPath=%FilePath:~0,-1%
rem Set Space environment variables
call "%FolderPath%"\..\SpaceEnv.bat
rem Set Task specific environment variables
set TaskName=MergeCSVfiles
set fileName=result.csv
set LogFile=%TaskName%_%LogDateTime%.log
:begin
cd ..
cd "Source Files\DCM_Source\Inbox"
echo Staring merge %fileName% at: %time%
setlocal enabledelayedexpansion
set "first=1"
>%fileName% (
for %%F in (msource*.csv) do (
if not "%%F"=="%fileName%" (
set /p "header="<"%%F"
if defined first (
type "%%F"
set "first="
) else (
type "%%F" |find /V "!header!"
)
)
)
)
endlocal
echo Finish merging %fileName% at: %time%
******UPDATED******
Example of CSV file
Sites|Level 2 sites|Date-time (visit start)|Visit ID|Unique visitor ID|Date-time (event)|Sources|Visitor categories|Visitor ID|Visits
SE Romania|PRM|2018-01-01T00:30:04|1|-6427177464|2018-01-01T00:30:04|Portal sites|-|0|2
SE Romania|PRM|2018-01-01T00:30:04|1|-6427177464|2018-01-01T00:30:04|Portal sites|-|0|2
This code will dedupe a file. In order to do that it must be sorted. This means any header record at the top of the file will be sorted into the file. This is code I received from dbenham. I can't remember if he originally posted it on StackOverflow or DosTips.com. If the file is very large it will more than likely crash with an out of memory error.
#echo off
:: Call function to dedupe file
CALL :DEDUPE "filename.txt"
goto :eof
:DEDUPE
:: DEDUPE file
setlocal disableDelayedExpansion
set "file=%~1"
set "sorted=%file%.sorted"
set "deduped=%file%.deduped"
::Define a variable containing a linefeed character
set LF=^
::The 2 blank lines above are critical, do not remove
sort "%file%" >"%sorted%"
>"%deduped%" (
set "prev="
for /f usebackq^ eol^=^%LF%%LF%^ delims^= %%A in ("%sorted%") do (
set "ln=%%A"
setlocal enableDelayedExpansion
if /i "!ln!" neq "!prev!" (
endlocal
(echo %%A)
set "prev=%%A"
) else endlocal
)
)
>nul move /y "%deduped%" "%file%"
del "%sorted%"
GOTO :EOF
#ECHO OFF
SETLOCAL
SETLOCAL ENABLEDELAYEDEXPANSION
SET "sourcedir=U:\sourcedir"
SET "filenamecommon=q49264647*.csv"
:: switch to required source directory
PUSHD "%sourcedir%"
:: get header line
FOR %%f IN (%filenamecommon%) DO FOR /f "delims=" %%h IN (%%f) DO SET "header=%%h"&goto gotheader
:gotheader
COPY %filenamecommon% atempfilename
SET "lastline="
>resultfilename (
ECHO %header%
SETLOCAL enabledelayedexpansion
FOR /f "delims=" %%d IN ('sort atempfilename' ) DO (
IF "%%d" neq "!lastline!" IF "%%d" neq "%header%" ECHO %%d
SET "lastline=%%d"
)
endlocal
)
DEL atempfilename
popd
GOTO :EOF
You would need to change the setting of sourcedir to suit your circumstances.
I used file/dirctorynames that suit my system for testing.
Note : datafiles containing the characters ! or ^ or unbalanced " will not be processed correctly.
First, find the header line by setting header from any matching filename. Once header is set, forcibly abort the for loops.
copy and concatenate all of the required files to a tempfile.
output the header line, then sort the tempfile to group identical lines. Read the result and output only those lines that differed from the previous and were not header lines.
Applying /i to the if statements will make the entire routine disregard character-case.
Sort the tempfile
Ok. Give this code a try. I think this code would generate the result file with not duplicated records not matters its size. However, the time the program will take depends on several factors, although IMHO it should not be excessive because the core part of the process is based on findstr.exe command.
#echo off
setlocal
del result.csv 2>NUL
rem Process all input files
for /F "delims=" %%f in ('dir /B /O:-S msource*.csv') do (
echo Merging file: %%f
if not exist result.csv (
rem Initialize output file with first input file
copy "%%f" result.csv > NUL
) else (
rem Get records in this file that are not in result file
findstr /V /G:result.csv "%%f" > newRecords.csv
rem and add they to the result file
type newRecords.csv >> result.csv
)
)
del newRecords.csv
You may also try to eliminate the dash in /O:-S switch of dir command; perhaps this change will speed up the process a little...
I have 50 text files and each text file has around 6000 lines. I am looking to make a batch job that will append the filename to each line of the text file.
I have a batch job to append data to the each line but cant wrap my head around getting the filename
This is what I have so far
#echo off
setLocal EnableDelayedExpansion
for /f "tokens=* delims= " %%a in (input.txt) do (
set /a N+=1
echo ^"(my filename)%%a^",>>output.txt
)
--- Example
filename 3315.txt
124123541234
1234123
2345623462356234
12341234562356245
Desired end result
3315124123541234
33151234123
33152345623462356234
331512341234562356245
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "destdir=U:\destdir"
FOR /f "delims=" %%a IN (
'dir /b /a-d "%sourcedir%\R*.txt" '
) DO (
(
FOR /f "usebackqdelims=" %%q IN ("%sourcedir%\%%a") DO (
ECHO (%%a^)%%q
)
)>"%destdir%\%%~na.out"
)
GOTO :EOF
You would need to change the settings of sourcedir and destdir to suit your circumstances.
I used a filemask of R*.txt to restrict the number of files processed on my test system.
Essentially, build a list of filenames using the dir command, and assign each to %%a in turn.
read each line of %%a into %%q and output to a new file in the destination directory made up of the name part of %%a (%%~na) and .out
--- later
To prepend the name part of the file to each line of the file, change
ECHO (%%a^)%%q
to
ECHO %%~na%%q
%%~na selects the Name part of %%a (see for /f|more from the prompt for more info)
I need to rename all of these files to the 6 character Part Number(306391) on the 3rd line.
Currently i have:
setlocal enabledelayedexpansion
set first=1
for /f "skip=3 delims= " %%a in (Name.txt) do (
if !first! ==1 (
set first=0
echo %%a > out.txt
ren Name.txt %%a.txt
)
)
Which finds the 6 digit part number and renames the file to the correct name. But breaks if i use *.txt instead of the actual name of the .txt file. I need it to work for all .txt files in the directory.
Surround your for /f loop with another for loop, then reference the outer loop variable in your ren command. You can also eliminate the need for delayed expansion by using if defined for boolean checks. I put in other tweaks here and there. Just ask if you want details.
#echo off
setlocal
for %%I in (*.txt) do (
set first=
for /f "usebackq skip=3" %%a in ("%%~fI") do (
if not defined first (
set first=1
echo %%~nxI ^> %%a.txt
rem // uncomment this when you're satisfied that it works correctly
rem // ren "%%~fI" "%%a.txt"
)
)
)
This method should run faster, specially if the files are large, because just 4 lines of each file are read (instead of the whole file):
#echo off
setlocal EnableDelayedExpansion
rem Process all .txt files
for %%f in (*.txt) do (
rem Read the 4th line
(for /L %%i in (1,1,4) do set /P "line4=") < "%%f"
rem Rename the file
for /F %%a in ("!line4!") do ECHO ren "%%f" "%%a.txt"
)
On windows 7 x64 with a batch files (*.bat):
I need to check in a folder the last file (less than 5 days) and path to add in Mycommand as argument.
I need i think :
"FORFILES -5"
perhaps "Call"
i begin with :
#echo off
REM
set folder="D:\temp"
set vararg="-a"
cd /d %folder%
for /F "delims=" %%i in ('dir /b') do (D:\stockage\Mycommand %vararg% %folder%\\"%%i" -v > c:\temp\test_log.txt)
How to output like that :
D:\stockage\Mycommand -a D:\temp\file.jpg -a D:\temp\file2.jpg -a D:\temp\file3.jpg etc...
don't put non existing file or blank argument, i think i must Loop the argument "-a D:\temp\filexxx.jpg" but i don't know how to do it.
And if possible escape file name probleme more than x chars or space inside.
Can you give me an example ?
Thank.
echo produces a linefeed and you can not stop it from doing so. As an alternative, first assemble the complete line:
setlocal enabledelayedexpansion
set commandline="D:\stockage\Mycommand"
set maxfiles=5
for /f "delims=" %%i in ('dir /b /o-d') do (
set /a maxfiles-=1
set commandline=!commandline! -a "%%i"
if !maxfiles! leq 0 goto :enough
)
:enough
echo %commandline%
This gives you the latest 5 files (or fewer, if the folder has not enough files)
Edited to reverse the order of the Parameters:
setlocal enabledelayedexpansion
set "commandline="
set maxfiles=5
for /f "delims=" %%i in ('dir /b /o-d') do (
set /a maxfiles-=1
set commandline=-a "%%i" !commandline!
if !maxfiles! leq 0 goto :enough
)
:enough
set commandline="D:\stockage\Mycommand" %commandline%
echo %commandline%