Running a command on every file in a folder, then saving output - windows

I am new to writing batch scripts and was hoping for some help.
I have a folder of images. I want to cURL each of these images, one by one, to an image classifier I made on the web. This will then send me a result.
I then want to save the response of the server into a file - let's keep it simple a say a CSV, in this format:
File, Response
So I know to run the Batch on each folder, I can do the following -
for /f %%f in ('dir /b c:\') do echo curl -X POST -F "images_file=#%%f" "https://gateway-a.watsonplatform.net/visual-recognition/api/v3/classify?api_key=abcdefgh&version=2016-05-20"*
I know, I can somehow pipe the results into an output file using ">>".
I don't know how to put this together, and how to create the CSV as I am running the batch script. Can anyone help? I think I am 90% there just new to batch scripting.

You can do something like this in powershell :
Change the placeholders as per your requirement.
$dir_files=Get-ChildItem "C:\"
foreach($file in $dir_files)
{
$result=Invoke-WebRequest 'https://gateway-a.watsonplatform.net/visual-recognition/api/v3/classify?api_key=abcdefgh&version=2016-05-20' -Method Post -Headers 'images_file=#%%f'
$result | ConvertTo-Csv | Out-File "D:\outputfile.csv" -Append -Force
}
I just gave you the way to set off. I didn't go through the complete batch code.
Hope it helps you.

If you want to use batch, you can do:
#echo
SetLocal EnableDelayedExpansion
set imagedir=C:\
set csvfile="C:\results.csv"
FOR %%G IN ("%imagedir%*") DO (
set resultfile=%%G.out
echo curl -X POST -F "images_file=#%%G" "https://gateway-a.watsonplatform.net/visual-recognition/api/v3/classify?api_key=abcdefgh&version=2016-05-20"* -o "!resultfile!"
echo %%G, !resultfile! >> %csvfile%
)
EndLocal
exit /b 0
I am not that experienced with curl so I'm not able to verify the correctness of your curl command and let the echo before it. But I do know that you can actually specify a file to write the output to in Windows with the -o flag.
The FOR command (without any flag/option) is another way to loop through files in a directory. If you still want to use the for /f %%f in ('dir /b ...') do-style (perhaps because of the order in which you want to treat your images), you can simply replace the line FOR %%G IN ("%imagedir%*") DO ( with for /f %%f in ('dir /b "%imagedir%"') DO (. I saved the directory in the variable imagedir, it's then easier to change. But if you change it, don't forget the \ at the end (if you use my for-command) and don't add double quotes (I add them in the FOR command). Be aware that this will treat all FILES in the directory. You can specify extensions if you want by using "%imagedir%\*.jpg" "%imagedir%\*.gif" for example for only GIF and JPEG images.
As last I'll say that I added SetLocal EnableDelayedExpansion to use delayed expansion. It's actually only needed because I saved the name of the file where the result of the curl operation will be saved, in the variable resultfile. In batch, a FOR -block (FOR ... ( ... )), and other blocks delimited with ( ) (as IF-blocks), are parsed as one single command (as if they were written on line). It is not really possible to change the value of a variable and read that new value in the same command (or same line) without using some special "tricks". Delayed expansion is one of them. When you use delayed expansion on a variable, you surround it with ! instead of %. As resultfile will change in each iteration, I use !resultfile! instead of %resultfile%.
If you don't want to save the name of the outputfile in a variable, you can remove the use of the variable and the delayed expansion (remove the EndLocal at the end too).
Good luck!

Related

Windows batch add suffixes to filenames with EXIF values in current folder with FOR loop

I'm trying to get some EXIF data added to the filenames in a folder. So far, I've been able to get the smaller pieces to work, but I can't get the name and extension separated so the can be combined with the EXIF values via the FOR loop.
I saw that I need to use ! to delineate the variable in the loop, but I can't get this to work with the filename.
SETLOCAL ENABLEDELAYEDEXPANSION
for /f "tokens=* delims=" %%a in ('dir /b *') do (
set file=%%~a
set _fname=%~n1
set _ext=%~x1
for /f %%i in ('exiftool.exe -s -S -SourceImageWidth %file%') do set WIDTH=%%i
for /f %%i in ('exiftool.exe -s -S -SourceImageHeight %file%') do set HEIGHT=%%i
for /f %%i in ('exiftool.exe -s -S -CompressorID %file%') do set CODEC=%%i
rename %file% "%_fname% [%WIDTH%x%HEIGHT% %CODEC%]%_ext%"
)
The process is: get filename -> get name only -> get extension -> get EXIF values -> rename file with values added to name and reuse extension -> repeat on next file in current directory
I apologize for the sloppy batch syntax as I'm sure there's a much more optimal way of coding.
Thank you!
You are making Exiftool Common Mistake #3. Exiftool's biggest performance hit is its startup time and using it in a loop like this will take a significantly longer time, especially since exiftool has very powerful batch and renaming abilities on its own.
Try this as a single command
exiftool "-Filename<%f [${SourceImageWidth}x$SourceImageHeight $CompressorID].%e" /path/to/files
Add -r to recurse into subdirectories. Remember to double any percent signs if used in a .BAT file.
The %f token stands for the base filename without the extention. The tags names are inserted by adding a dollar sign in front of them. Because SourceImageWidth is directly adjacent to the x, braces are used to separate the two so exiftool doesn't go looking for a tag named "SourceImageWidthx". The %e stands for the file extension.

Batch file variable filenames: make timestamp filename within a for loop?

I am trying to run a batch of Lighthouse audits from the command line and output the results to JSON files. To ensure they don't overwrite one another, each file name needs to be unique. I would like to use the timestamp as the filename ideally. I have been through many other Stack questions on a similar topic, but I cannot get this to work. I am currently using the below code, written within a batch file and called from Windows command line:
SETLOCAL ENABLEDELAYEDEXPANSION
for /f "delims=" %%a in (C:\Users\User\Desktop\Folder\Lighthouse_Project\urls.txt) DO (
set dte=!DATE:/=-%!
set tme=!TIME::=-%!
set tme=!%tme:.=-%!
ECHO Line is: %%a
lighthouse %%a --quiet --chrome-flags=" --headless" --output=json --output-path=C:\Users\User\Desktop\Folder\Lighthouse_Project\json_logs\!dte!!tme!.json
)
The first file produced has the correct name all subsequent JSONs have the literal name: '!dte!!tme!', and do overwrite one another. If you run the Lighthouse command outside of a for loop it executes perfectly, I'm aware that variables don't update within a for loop but I thought !var! could be used to counter this. The for loop iterates through a list of URLs stored in urls.txt, auditing each of them in turn and returning a JSON file.
Current outcome: !dte!!tme!.json
Desired oucome: 08-10-201913-55-12-07.json, 08-10-201913-56-14-00.json etc.
If anybody knows how to amend this to achieve the correct outcome I would be very grateful.
The urls.txt contains a list of URLs, each on a new line:
https://www.bbc.co.uk/weather/2643743
https://www.bbc.com/mundo
https://regex101.com/
https://www.bbc.co.uk/news
Example:
#Echo Off
SETLOCAL ENABLEDELAYEDEXPANSION
for /f "delims=" %%a in (C:\Users\Owner\Desktop\Folder\Lighthouse_Project\urls.txt) DO (
set dte=!DATE:/=-!
set tme=!TIME::=-!
set tme=!tme:.=-!
ECHO Line is: %%a
echo lighthouse "%%a" --quiet --chrome-flags="--headless" --output=json --output-path="C:\Users\Owner\Desktop\Folder\Lighthouse_Project\json_logs\!dte!!tme!.json"
Timeout 2 >NUL
)
Timeout -1 >NUL
This uses the exact content from your urls.txt example. I have added #Echo Off at the top to reduce screen clutter, and an echo in front of your lighthouse command so that it displays instead of runs. Additionally I included a couple of timeouts to both simulate a short timespan, (as the command is being echo'ed not ran), and to give you an opportunity to read the output at the end.
Here's the output from the above to prove my case:
Line is: https://www.bbc.co.uk/weather/2643743
lighthouse "https://www.bbc.co.uk/weather/2643743" --quiet --chrome-flags="--hea
dless" --output=json --output-path="C:\Users\Owner\Desktop\Folder\Lighthouse_Pro
ject\json_logs\08-10-201916-12-42-93.json"
Line is: https://www.bbc.com/mundo
lighthouse "https://www.bbc.com/mundo" --quiet --chrome-flags="--headless" --out
put=json --output-path="C:\Users\Owner\Desktop\Folder\Lighthouse_Project\json_lo
gs\08-10-201916-12-44-14.json"
Line is: https://regex101.com/
lighthouse "https://regex101.com/" --quiet --chrome-flags="--headless" --output=
json --output-path="C:\Users\Owner\Desktop\Folder\Lighthouse_Project\json_logs\0
8-10-201916-12-46-12.json"
Line is: https://www.bbc.co.uk/news
lighthouse "https://www.bbc.co.uk/news" --quiet --chrome-flags="--headless" --ou
tput=json --output-path="C:\Users\Owner\Desktop\Folder\Lighthouse_Project\json_l
ogs\08-10-201916-12-48-21.json"
Press any key to continue . . .
Your issue, as originally stated, was the inclusion of % characters in your initial set syntax, as you can see from the provided output.
By keeping the expansion without exiting the loop as you intended.. Note I have added a timeout of millisecond here to ensure we have a different time each time. ping 127.0.0.1 -n 1 -w 500> nul:
#echo off
setlocal enabledelayedexpansion
for /f "delims=" %%a in (%userprofile%\Desktop\Croud_DS_1\Lighthouse_Project\urls.txt) DO (
set dte=!date:/=-!
set tme=!time::=-!
set tme=!tme:.=-!
set tme=!tme:,=-!
echo Line is: %%a
lighthouse "%%a" --quiet --chrome-flags=" --headless" --output=json --output-path=%userprofile%\Desktop\Folder\Lighthouse_Project\json_logs\!dte!_!tme!.json
ping 127.0.0.1 -n 1 -w 500> nul
)
Please take note, I took the liberty of adding an underscore to make sure the date and time format is readable, so you'll have YYYY-MM-dd_HH-mm-ss-ms

wget: batch download with increment

I have this situation: I want to download a bunch of files named like this:
683482, 684483, 685484, 686485, 687486, 688487, 689488, 690489, 691490, 692491, ...
As you can see, the files are numbered with an increment of 1001. So, what's the easiest way to do a batch download?
Please try this:
#!/bin/bash
for i in {683482..1000000..1001}
do
wget $i
done
ECHO OFF
CLS
SETLOCAL ENABLEDELAYEDEXPANSION
FOR /L %%A IN (683482,1001, 692491) DO (
SET stringWget=wget %%A
!stringWget!
)
I'll go step by step:
ECHO OFF prevents windows command line from displaying every command on the command-prompt (this is optional; But, looks clean).
CLS clear screen; This clears command prompt's console display. It does not clear the temporary environment variables or command history.
SETLOCAL ENABLEDELAYEDEXPANSION is used for non-blocking statements and is Windows specific; When we use %ENVIRONMENTVARIABLE% , we are implying blocking statement and when we use !ENVIRONMENTVARIABLE! we are implying non-blocking statement (meaning, in a loop, we see updated values of %A instead of repeating what %A had when entering loop). We use %% instead of %in batch files.
FOR loop's syntax can be found in reference.
Reference: https://stackoverflow.com/a/3541415

Save command output to variable

In a batch file, I am trying to fetch the output of a command and save it to a variable.
The goal of my command is to count the number of folders in a certain folder.
I can't use the trick provided in this accepted answer because I would have to do cd path\to\my\folder to get to the current directory. Unfortunately, I can't do this command because path\to\my\folder is in fact a UNC path (\\path\to\my\folder), and cd \\some\UNC\path is not supported by the windows cmd.
I am aware of this answer but I don't want to use a temporary file.
So I tried to do the following:
To obtain the number of folders, I use:
dir \\path\to\my\folder | find /c "<REP>"
This works fine and returning me a number as I would expect.
To retrieve the output of this command in a batch variable, I tried:
FOR /F "TOKENS=*" %%i IN ('\\path\to\my\folder | find /c "<REP>"') DO
SET value = %%i
But without success, the error message being...
| was unexpected.
...when I execute the batch file and...
%%i was unexpected.
when I try to execute the command directly in a command window. I tried to escape the quotes around the <REP> string (...find /c ""<REP>""') DO...), got me the same error.
Am I on the right path to retrieve the output in a variable? What should I do to resolve the error message?
Or maybe there is a simpler way to set a command output in a variable?
You can use the answer you first mentioned. You don't have to cd there, but you can use pushd which will allocate a temporary drive letter for UNC paths which will be released when you do a popd.
So in essence:
pushd \\server\path
set count=0
for /d %%x in (*) do set /a count+=1
popd

How to blank files in batch

How would I go about reading an entire directory and blanking files with a specific extension? I have an application that reads the content of a specific folder and returns an error if a file is missing, however it does't check to see if the files are valid, so I want to make them NULL to get around the checks.
if by 'blanking' you mean truncating them, you could use the following:
for /f %%a in ('dir *.[my ext]') do (echo . > %%a)
note that the double % is for use within a batch file. if you are running this from a command line, use a single %.
EDIT:
to incorporate #Loadmaster's improvement:
for /f %%a in ('dir *.[my ext]') do (type nul > %%a)
First, create an empty file. Call it "blank". You can use Notepad to just save an empty file, for example.
Let's suppose the specific extension is ".xyz". Run this:
for %f in (*.xyz) do copy /y blank %f
The for loop sets variable "%f" to each file name in turn, and runs the copy command. The copy command copies the blank file on top of each matching file.
By the way, you can find out more about the for command using the help command:
help for
You could make the batch file take the filter and then it's much more useful.
#for %%i in (%1) do cd.>%%i
Usage (if you call it empty.bat):
empty *.c

Resources