I have a .txt file that updates every five seconds.
22:37:03
Some text 1
22:37:08
Some text 2
Some text 3
22:37:13
Some text 4
I have a program that reads this data from the Start of the file, which I'd like to be as up to date as possible (to within 5 seconds if possible).
If I read the file directly with the program it starts from the earliest entry, which may be up to two hours behind (depending on when I started writing the file).
Is there a command I could add to a batch file that would delete any data that was say 8 seconds older than the current system time?
I already run a command that deletes the first 2 lines every 5 seconds, but the number of lines under the time stamp vary.
#ECHO Off
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "destdir=U:\destdir"
SET "filename1=%sourcedir%\q42253025.txt"
SET "outfile=%destdir%\outfile.txt"
FOR /f "usebackqdelims=" %%a IN ("%filename1%") DO (
FOR /f "tokens=1-3delims=:" %%p IN ("%%a") DO (
IF "%%r"=="" (>>"%outfile%" ECHO %%a) ELSE (>"%outfile%" ECHO %%a)
)
)
GOTO :EOF
You would need to change the settings of sourcedir and destdir to suit your circumstances.
I used a file named q42253025.txt containing your data for my testing.
Assuming filename1 is your logfile, then this code will read that file and create a new file containing that section of the logfile from the final timestamp onwards.
I'm making a few assumptions. First, you don't want empty lines and second that any lines that contain two or more colons are timestamp lines. I haven't tested for the effects of "poison characters" (those symbols with a special meaning to cmd like !%^&><|)
If the procedure takes too long (such that it still has hold of the file when the generator program wants to perform the next write) then perhaps copying your log to a completely separate file then using this procedure on the copy may work for you.
if the file gets updated every 5 seconds and you want to show anything younger than 8 seconds, then you obviously want the last timestamp and following line(s):
#echo off
REM get LineNo of last TimeStamp (subtract one for future use)
for /f "delims=:" %%a in ('findstr /n /r "^[ 012][0-9]:[0-5][0-9]:[0-5][0-9]$" test.txt') do set /a LastStart=%%a-1
REM show, starting with the found LineNo
more +%LastStart% test.txt
(preserves any empty line(s), works even with "poison characters")
Related
I have a batch file that gets run by the user typing:
usercompile filename
usercompile is a batch file that does this:
copy /y %1.txt lib\incoming_file.txt
and then starts the compiler:
compiler.exe
The compiler has the "incoming_file" name hard-coded into linked source (this can't be chaged), so the current method is simply to copy the user file in and rename it to the known name and run the compiler.
I'd like to present the user with a list of files that are generated when a batch file is run, then the batch file would copy the selected file in, rename it (just like is done now).
So it would look like this:
Please choose a file to compile:
1) matthews_build
2) marks_build
3) lukes_build
and then the user would type 1 or 2 or 3 (in this case) and press enter. The batch file would copy that file to the known file name and launch the compiler. The one good thing is that the files that need to be listed all have a unique extension (.jal).
Any ideas?
I changed my approach and consider my previous answer a bad practice: re-listing the files with a second dir command unnecessarily reads the disk again, not to mention the rare but possible case if a file is added/removed between the 2 dir's and makes the whole thing unreliable.
Based on this brilliant solution I did a possible implementation with dynamic array:
#echo off
set /a counter=0
setlocal enabledelayedexpansion
FOR /f "delims=|" %%i IN ('dir /b /on "yourpath*.jal"') DO (
set /a counter+=1
rem echo !counter!^) %%~ni
set FileList[!counter!]=%%~ni & rem This is an array element, a dinamically created variable
)
rem Iterate through variables:
FOR /l %%i IN (1,1,!counter!) DO (
echo %%i^) !FileList[%%i]!
)
set /p option="Choose an option: "
echo !FileList[%option%]!
endlocal
This makes the file list available for any number of following commands.
One possible solution is to list all .jal files and give them an option number, store the result, and based on user input, look up the file based on the option number. As I know no way of storing such a result in memory (no array/hash table data type), only in a file, if a file can not be used, then the listing should be repeated in a deterministic way so that if we re-assign the option numbers, we get the same result. We can do it ensuring alphabetical ordering.
Here is one implementation:
BLOCK 1
setlocal enabledelayedexpansion
FOR /f "delims=|" %%i IN ('dir /b /on "yourpath\*.jal"') DO (
set /a counter+=1
echo !counter!^) %%~ni
)
endlocal
The nested dir command ensures alphabetical ordering (reference.)
A remark why I put a pipe (|) as a delimiter: if you don't define a delimiter, the default space will be used. If your file name contains space then it would be truncated. So I picked a character that is not valid in file names ensuring the whole file name is returned.
Now if you get a number from the user by this:
set /p option=Choose your option:
after this command (evaluating and possibly re-requesting the input) to do a lookup for the file you can repeat BLOCK 1 but replace the echo line with examining the option like this:
if !counter! == %option%
and put those commands in the if block to do whatever you want to do with the file (for debugging, put back the echo command).
I have a file that look like below.
File Date Source Target
HD|Field1|Field2|Field3
ITEM1|Other fields1
ITEM2|Other fields2
HD|Field1|Field2|Field3
ITEM1|Other fields
ITEM2|Other fields
ITEM3|Other fields
I need to create separate files based on the occurrence of HD. First file will contain lines starting from HD and will have everything till the next HD segment starts.
There can be N number of HD segments. The files also need to be renamed based on Field1 value of HD segment.
So file 1 will be as File-Field1 and would contain
HD|Field1|Field2|Field3
ITEM1|Other fields1
ITEM2|Other fields2
File 2 will be File-Field1(of 2nd HD segment) and would contain
HD|Field1|Field2|Field3
ITEM1|Other fields
ITEM2|Other fields
ITEM3|Other fields
I need some help in getting the batch script. I have done some basic code and it looks like below.
setLocal EnableDelayedExpansion
set limit=1
set file=Sample.txt
set lineCounter=1
set filenameCounter=1
set name=
set extension=
for %%a in (%file%) do ( set "name=%%~na" set "extension=%%~xa" )
for /f "skip=1 delims=," %%a in (%file%) do ( set
splitFile=Load-!name!!filenameCounter!!extension!
if "%%a"=="HD|" ( set /a filenameCounter=!filenameCounter! + 1 set
lineCounter=1 echo Created !splitFile!. ) echo %%a>> !splitFile! set /a
lineCounter=!lineCounter! + 1 )
With this I get only 1 file and the line with HD| but the name is fine as Load-Sample1.txt. However there is huge loss of data. What I tried is to do a loop that will skip the first line and then in the for loop create a new file everytime a HD| is encountered.
Here is a brittle pure batch solution (lots of ways the code can break depending on the content of the source file)
#echo off
setlocal enableDelayedExpansion
set "outfile="
for /f "delims=" %%A in (sample.txt) do (
for /f "delims=| tokens=1,2" %%a in ("%%A") do if "%%a"=="HD" set "outfile=%%b"
if defined outfile echo(%%A>>"!outfile!"
)
Here are some of the ways the above code could fail:
Empty lines will be stripped
Lines beginning with ; will be stripped
Lines containing ! will be corrupted
The code could be made more robust, but it will become significantly more complicated. I would not bother. Pure batch is a terrible language for text file manipulation, except for the simplest of tasks. It is slow, and requires loads of arcane knowledge.
I have added a new feature (v6.8) to my JREPL.BAT regular expression text processor that makes it trivial to create a fast and robust solution for this problem.
JREPL.BAT is pure script (hybrid JScript/batch) that runs natively on any Windows machine from XP onward - no 3rd party exe file required.
I use a regular expression to locate HD lines and extract the file name. I use custom JScript to open a new output file at each HD line.
jrepl "^HD\|([^|]+)" "openOutput($1);$txt=$0" /jq /f "sample.txt" >nul
Be sure to use CALL JREPL if you use the command within another batch script. However, CALL will double the quoted caret, and a caret could technically be part of a file name. So you should also use another new feature of version 6.8 - the new \c caret escape sequence. This will hide the caret from CALL so it does not get doubled.
call jrepl "\cHD\|([\c|]+)" "openOutput($1);$txt=$0" /x /jq /f "sample.txt" >nul
I have a photobooth that will dump all the pictures as they're taken onto a helper PC on the same network.
On the helper PC, I had planned to have a scheduled task run every x minutes to perform some edits on the images using ImageMagick (eg montage, label, etc).
The files are named via timestamp and it takes 4 at a time, so if a few people back to back used the photobooth, then it would transfer the following files to the helper PC:
2016-01-29-22-05-01.jpg
2016-01-29-22-05-02.jpg
2016-01-29-22-05-03.jpg
2016-01-29-22-05-04.jpg
2016-01-29-22-07-01.jpg (note the timestamp change)
2016-01-29-22-07-02.jpg
2016-01-29-22-07-03.jpg
2016-01-29-22-07-04.jpg
I was initially using a batch file to just serve *.jpg into ImageMagick, but now I realise if two people use the photobooth in < x minutes, then there will be two sets of images (like the above). Handing 8 pictures instead of 4 to ImageMagick will result in a mess.
My question is - how can I identify files which a similarly named (eg, first 16 characters are the same) and just pass these to ImageMagick?
#ECHO OFF
SETLOCAL
SET "sourcedir=U:\sourcedir"
SET "filename1=%sourcedir%\transferred.log"
IF NOT EXIST "%filename1%" ECHO xxx>>"%filename1%"
:again
SET "transfer="
FOR /f "tokens=1-6delims=-" %%a IN (
'dir /b /a-d /on "%sourcedir%\*.jpg"^|findstr /b /v /g:"%filename1%"'
) DO (
SET "transfer=%%a-%%b-%%c-%%d-%%e"
GOTO movefiles
)
ECHO no new files to MOVE
GOTO :EOF
:movefiles
>>"%filename1%" ECHO %transfer%
:: don't know what you want from here
ECHO transfer %transfer%* ??
SET "transfer=%transfer%-01.jpg %transfer%-02.jpg %transfer%-03.jpg %transfer%-04.jpg"
ECHO transfer %transfer% ??
:: perhaps a delay here for helper PC to process??
timeout /t 5 >nul
GOTO again
You would need to change the settings of sourcedir and filename1 to suit your circumstances.
You haven't told us what happens to the files you want grouped, so I've assumed they remain.
The meat of the matter is the for - grab the first 6 tokens using - as separators and apply to a directory-list in basic form without directories sorted in name order and filtered for lines that begin with a string which does not appear in filename1
If any such line is found, gather the name into transfer, and do something with that value (not specified), recording the value in the log so that the next loop includes the value found in its exclude-list.
I don't have time to code this, but you could do the following
Get current date and time to the second
Format it like the filenames (set mydatetime=2016-01-29-22-07)
for /r %i in (%mydatetime%*) do move %i %targetpath%
wait 1 minute PING 127.0.0.1 -n 1 -w 60000 >NUL
loop
(this assumes only one batch per minute as you indicated in the comments above)
However, if you have the ability to modify the output (as indicated by adding "a couple of 1 second sleeps") I wonder like everyone else why you don't just make the output more robust? Separate folders, batch names, etc.
I've seen plenty of posts on similar requests but I can't quite find one that suits what I am trying to do. It is fairly simple but I cannot seem to get it.
ren FILE??.txt FILE%Year%%Month%%Day%??.txt
copy FILE%Year%%Month%%Day%??.txt C:\Users\me\Desktop\Batch\renamed\achod%Year%%Month%%Day%??.txt
I cannot get the script to keep the '??' which represents random characters the first file may have.
Any help is appreciated.
You won't be able to rename files directly using a wildcard character. Instead you need to locate all the applicable files and then rename each.
The script below works under the assumptions of your question/comments:
File name is 6 chars long.
Only the last 2 chars are interchangeable.
Of course, the script could be very easily adapted to accomodate other settings but this does just as you requested.
SETLOCAL EnableDelayedExpansion
REM Set your Year, Month, Day variable values here.
REM They will be used for file renaming.
...
CD "C:\Path\To\Files"
FOR /F "usebackq tokens=* delims=" %%A IN (`DIR "File??.txt" /B /A:-D`) DO (
REM Extract the last 2 chars of the file name.
SET FileName=%%~nA
SET First4=!FileName:~0,4!
SET Last2=!FileName:~-2!
REM Rename the file, inserting the new data.
RENAME "%%A" "!First4!%Year%%Month%%Day%!Last2!%%~xA"
)
ENDLOCAL
After fiddling around for the first time with the command-line and ImageMagic, I have been able to do the following:
c:\test\paper.pdf contains a pdf file
c:\test>convert paper.pdf output-%d.tiff
The pdf file contains five pages, and the output is as expected 5 tiff files :-) Now I want to put multiple files in c:\test and loop through them, creating pages based on the original filename. So assume the following files in c:\test:
paper.pdf (5 pages)
example.pdf (2 pages)
new.pdf (1 page)
The output of a batch script should be 8 tiff files in the c:\test\tiffs\ folder:
paper-0.tiff paper-1.tiff paper-2.tiff paper-3.tiff paper-4.tiff
example-0.tiff example-1.tiff
new-0.tiff
and I would very much like the command line return the time it took converting the pdf's to tiffs:
c:\test>convert.bat
c:\test>This action took 120 seconds
I'm trying to write this batch file but since it's my first try I'm having trouble. I am first trying to loop through the test folder and create all the tiffs (leaving timing the process for now):
for %%f in (C:\test\) do convert %%f.pdf %%f-%%d.tiff
I would have expected this to work but got an error:
C:\Users\Me>convert c:\test\c:\test\.pdf c:\test\c:\test\-%d.tiff convert.exe:
unable to open image `c:\test\c:\test\.pdf': Invalid argument #
error/blob.c/OpenBlob/2646. convert.exe: no decode delegate for this image
format `c:\test\c:\test\.pdf' # error/constitute.c/ReadImage/552. convert.exe:
no images defined `c:\test\c:\test\-%d.tiff' #
error/convert.c/ConvertImageCommand/3106.
What am I doing wrong?
Timing
This is one of those jobs that it is just wrong to do (for real—it's OK to do it to show off) with batch/CMD. Nearly every programming language I've ever encountered would be better at doing this than CMD.
This particular (hackish) implementation will not work if the elapsed time goes past midnight of any given day, and depending on how you have dates configured in your locale, it might break if the run starts before 1 P.M. and finishes after 1 P.M.
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
FOR /F "tokens=5 delims= " %%t IN ('^<NUL time') DO SET START_TIME=%%t
:: Stuff that you're timing.
FOR /F "tokens=5 delims= " %%t IN ('^<NUL time') DO SET END_TIME=%%t
SET ELAPSED=
FOR /F "delims=:. tokens=1,2,3,4" %%a IN ("%START_TIME%") DO (
FOR /F "delims=:. tokens=1,2,3,4" %%n IN ("%END_TIME%") DO (
SET /A HOURS=%%n-%%a
SET /A MINUTES=%%o-%%b
SET /A SECONDS=%%p-%%c
SET /A HUND=%%q-%%d
SET /A TOTAL=!HOURS!*3600 + !MINUTES!*60 + !SECONDS!
IF !HUND! LSS 0 (
SET /A TOTAL=!TOTAL!-1
SET /A HUND=!HUND!+100
)
IF !HUND! LSS 10 SET HUND=0!HUND!
SET ELAPSED=!TOTAL!.!HUND!
)
)
#ECHO.
#ECHO.
#ECHO This action took %ELAPSED% seconds.
ImageMagick
The problem I see with your convert command is that you've not told CMD what pattern to search for. It sounds like you want *.pdf. Doing this will mean that you don't need to append ".pdf" to the end of the input file name and need to remove the extension when specifying the output file name. The %%~dpnf produces the entire path to the file without the extension.
FOR %%f IN (C:\temp\*.pdf) do convert "%%~f" "%%~dpnf-%%d.tiff"
EDIT: To put the PDF's in a different location, change the %%f substitution:
FOR %%f IN (C:\temp\*.pdf) do convert "%%~f" "\path\to\tiffs\%%~nf-%%d.tiff"
Better Method?
I guess I should at least put in a plug for doing it the Right™ way. If you didn't want to invoke your command from a script in a language that supported timing (like Python), you could just retrieve the system time as an integer. This strategy will work until roughly 2038. I used Perl in this example.
#ECHO OFF
FOR /F %%t IN ('perl -e "print(time());"') DO SET START_TIME=%%t
:: stuff to time
FOR /F %%t IN ('perl -e "print(time());"') DO SET END_TIME=%%t
SET /A ELAPSED=%END_TIME%-%START_TIME%
#ECHO This step took %ELAPSED% seconds.