I have a script that allow me to extract a specific columns from a csv file.
My aim subject is to apply my script to all csv files of my directory.
#echo off>fourcol.csv
setlocal
for /f "tokens=1-22* delims=," %%1 in (ManyColumns.csv) do (
echo %%4,%%5,%%6>>fourcol.csv
)
type fourcol.csv
Apply the script to all the csv file in my directory
Remove duplicates
FileA.csv ...FileZ.csv
server1,Dell,1.0.1,server1,Dell,28/06/2016,...
server2,Hp,1.0.2,server2,Hp,29/06/2016,...
server3,Dell,1.2.1,server3,Dell,30/06/2016,...
server4,Hp,1.3.1,server4,Hp,27/06/2016,...
server3,Dell,1.2.1,server3,Dell,30/06/2016,...
server4,Hp,1.3.1,server4,Hp,27/06/2016,...
My CSV files have the same header and the same data.
Output after applying the script (without duplicates):
server1,Dell,1.0.1
server2,Hp,1.0.2
server3,Dell,1.2.1
server4,Hp,1.3.1
How do I apply the script to all the csv files in my directory?
A couple of small changes should do it:
Don't use numbers as for loop variables. They are reserved for command line parameters.
Use dir /b *.csv to get the list of files to process.
Start from token 1 not token 4 to extract the first three columns.
Use the first for loop to process each file, and a second (nested) for loop to process the lines within the file.
Note:
delims=, may not do what you want (if there are values in the file that contain commas ,).
Try the following batch file (test.cmd):
#echo off
setlocal
rem remove output file if it already exists
if exist fourcol.csv del fourcol.csv
rem loop through the csv files
for /f "usebackq" %%a in (`dir /b *.csv`) do (
rem loop through the lines in each file
for /f "usebackq tokens=1,2,3 delims=," %%b in (%%a) do (
echo %%b,%%c,%%d>>fourcol.csv
)
)
rem add code here to strip duplicates
endlocal
test.csv:
server1,Dell,1.0.1,server1,Dell,28/06/2016,...
server2,Hp,1.0.2,server2,Hp,29/06/2016,...
server3,Dell,1.2.1,server3,Dell,30/06/2016,...
server4,Hp,1.3.1,server4,Hp,27/06/2016,...
server3,Dell,1.2.1,server3,Dell,30/06/2016,...
server4,Hp,1.3.1,server4,Hp,27/06/2016,...
fourcol.csv:
server1,Dell,1.0.1
server2,Hp,1.0.2
server3,Dell,1.2.1
server4,Hp,1.3.1
server3,Dell,1.2.1
server4,Hp,1.3.1
Further Reading
An A-Z Index of the Windows CMD command line - An excellent reference for all things Windows cmd line related.
dir - Display a list of files and subfolders.
for /f - Loop command against the results of another command.
Related
I have a batch file which copies some local files up to a google storage area using the gsutil tool. The gsutil tool produces a nice log file showing the details of the files that were uploaded and if it was OK or not.
Source,Destination,Start,End,Md5,UploadId,Source Size,Bytes Transferred,Result,Description
file://C:\TEMP\file_1.xlsx,gs://app1/backups/file_1.xlsx,2018-12-04T15:25:48.428000Z,2018-12-04T15:25:48.804000Z,CPHHZfdlt6AePAPz6JO2KQ==,,18753,18753,OK,
file://C:\TEMP\file_2.xlsx,gs://app1/backups/file_2.xlsx,2018-12-04T15:25:48.428000Z,2018-12-04T15:25:48.813000Z,aTKCOQSPVwDycM9+NGO28Q==,,18753,18753,OK,
What I would like to do is to
check the status result in column 8 (OK or FAIL)
If the status is OK then move the source file to another folder (so that it is not uploaded again).
The problem is that the source filename is appended with "file://" which I can't seem to remove, example
file://C:\TEMP\file_1.xlsx
needs to be changed into this
C:\TEMP\file_1.xlsx
I am using a for /f loop and I am not sure if the manipulation of the variables %%A is different within a for /f loop.
#echo off
rem copy the gsutil log file into a temp file and remove the header row using the 'more' command.
more +1 raw_results.log > .\upload_results.log
rem get the source file name (column 1) and the upload result (OK) from column 8
for /f "tokens=1,8 delims=," %%A in (.\upload_results.log) do (
echo The source file is %%A , the upload status was %%B
set line=%%A
set line=!line:file://:=! >> output2.txt echo !line!
echo !line!
)
The output is like this.
The source file is file://C:\TEMP\file_1.xlsx , the upload status was OK
The source file is file://C:\TEMP\file_2.xlsx , the upload status was OK
I'm expecting it to dump the altered values out into a new file but it is not producing anything at the moment.
Normally I would extract from a specific character to the end of the string with something like this but it doesn't work with my For/f loop.
%var:~7%
Any pointers or a different way of doing it greatly appreciated.
Since the part to remove seems fixed it is easier to use substrings.
Also using for /f "skip=1" evades he neccessity of the external command more +1 and another intermediate file.
#echo off & setlocal EnableDelayedExpansion
type NUL>output2.txt
for /f "skip=1 eol=| tokens=1,8 delims=," %%A in (.\upload_results.log) do (
echo The source file is %%A , the upload status was %%B
set "line=%%A"
set "line=!line:~7!"
echo(!line!>>output2.txt
echo(!line!
)
File names and paths can contain also one or more exclamation marks. The line set line=%%A is parsed by Windows command processor a second time before execution with enabled delayed expansion. See How does the Windows Command Interpreter (CMD.EXE) parse scripts? Every ! inside the string assigned to loop variable A is on this line interpreted as begin or end of a delayed expanded environment variable reference. So the string of loop variable A is assigned to environment variable line with an unwanted modification if file path/name contains one or more exclamation marks.
For that reason it is best to avoid usage of delayed expansion. The fastest solution is for this task using a second FOR to get file:// removed from string assigned to loop variable A.
#echo off
del output2.txt 2>nul
for /F "skip=1 tokens=1,8 delims=," %%A in (upload_results.log) do (
echo The source file is %%A , the upload status was %%B.
for /F "tokens=1* delims=/" %%C in ("%%~A") do echo %%D>>output2.txt
)
Even faster would be without the first echo command line inside the loop:
#echo off
(for /F "skip=1 delims=," %%A in (upload_results.log) do (
for /F "tokens=1* delims=/" %%B in ("%%~A") do echo %%C
))>output2.txt
The second solution can be written also as single command line:
#(for /F "skip=1 delims=," %%A in (upload_results.log) do #for /F "tokens=1* delims=/" %%B in ("%%~A") do #echo %%C)>output2.txt
All solutions do following:
The outer FOR processes ANSI (fixed one byte per character) or UTF-8 (one to four bytes per character) encoded text file upload_results.log line by line with skipping the first line and ignoring always empty lines and lines starting with a semicolon which do not occur here.
The line is split up on every occurrence of one or more commas into substrings (tokens) with assigning first comma delimited string to specified loop variable A. The first solution additionally assigns eighth comma delimited string to next loop variable B according to ASCII table.
The inner FOR processes the string assigned to loop variable A with using / as string delimiter to get assigned to specified loop variable file: and to next loop variable according to ASCII table the rest of the string after first sequence of forward slashes which is the full qualified file name.
The full qualified file name is output with command echo and appended either directly to file output2.txt (first solution) or first to a memory buffer which is finally at once written into file output2.txt overwriting a perhaps already existing file with that file name in current directory.
For understanding the used commands and how they work, open a command prompt window, execute there the following commands, and read entirely all help pages displayed for each command very carefully.
del /?
echo /?
for /?
See also the Microsoft article about Using command redirection operators for an explanation of the redirections >, >> and 2>nul
I need to split some text in lines and concatenate with a sufix using Windows cmd .bat.
I recieve lists that came like:
9448
9453
9463
9464
9474
9477
or like:
9448, 9453, 9463, 9464, 9474, 9477
So I need to put every number of these added with .jpg, like:
9448.jpg
9453.jpg
9463.jpg
them the program would run the way I need.
here goes the code I'm working on:
echo off
for %%a in (.) do set currentfolder=%%~na
set src_folder= %CD%
set dst_folder= "%currentfolder%_SELECTED/%date:/=%%"
md %dst_folder%
for /f %%i in (list.txt) DO copy %%i %dst_folder%\%%i
run two nested for loops: one to split into lines and another to split a line into separate tokens. So you don't have to care, which of the two formats the file has.
#echo off
for /f "delims=" %%a in (list.txt) do (
for %%b in (%%a) do (
ECHO copy "%%b.jpg" "%dst_folder%\%%b"
)
)
Note: it isn't clear to me, what you exactly try to do. Adapt the ECHO line until the output is what you want, then remove the ECHO.
I was inspired on works of our colleagues from portal
Have a batch create text files based on a list
Batch script to read input text file and make text file for each line of input text file
to develop creation from a text file for example
"comps.txt" ( comp1,comp2....
for each PC a batch script read from a list of PCs in a text file and create a text file for each line of text as local.
Later in code are created folders files named - the same names ( comp1,comp2....
at the end we have text files : comp1.txt ,comp2.txt ... and folders: comp1 , comp2.... till 400 computers.
Any idea how to add in code or write another separate batch code to move for each coresponding folder text file for text file
comp1.txt->comp1 folder
comp2.txt->comp2.txt
....
We have more than 400 lines!!
I am very begginers for any scrips working very hard every single day and this is my first question. My code in Windows Batch is below
#echo off
setlocal
for /f "tokens=*" %%a in (comps.txt) do (type nul>"%%a.txt")
for /f "tokens=*" %%a in (comps.txt) do (
echo This is line 1 of text>"%%a.txt"
)
for /f %%i in (comps.txt) do mkdir %%i
do (
echo "%%i.txt"
)
endlocal
[]
It doesn't make any sense to generate text files and then folders to move the files to. Create the folders first place and create the files into the folders. Using a (code block) only one for loop is needed.
#echo off
setlocal
for /f "tokens=*" %%a in (comps.txt) do (
if not exist "%%a" mkdir "%%a"
echo This is line 1 of text>"%%a\%%a.txt"
)
endlocal
I'm trying to get a side-by-side file path and file name in a text file so I can make inserting into a database easier. I've taken a look at other examples around SO, but I haven't been able to understand what is going on. For instance, I saw this batch file to append file names to end of lines but figured that I shouldn't ask for clarification because it's 1.5 years old.
What I have is a text file of file paths. They look like this:
\\proe\igi_files\TIFFS\AD\1_SIZE_AD\1AD0019.tif
What I want it to look like is this:
1AD0019.tif \\proe\igi_files\TIFFS\AD\1_SIZE_AD\1AD0019.tif
so that I can insert it into a database. Is there an easy way to do this on Windows via Batch files?
No batch file required. From the command line:
>"outputFile.txt" (for /f "usebackq eol=: delims=" %F in ("inputFile.txt") do #echo %~nxF %~dpF)
But that output format is risky because file and folder names can contain spaces, so it may be difficult to determine where the file name ends and the path begins. Better to enclose the file and path within quotes.
>"outputFile.txt" (for /f "usebackq eol=: delims=" %F in ("inputFile.txt") do echo "%~nxF" "%~dpF")
if done within a batch file, then percents must be doubled.
#echo off
>"outputFile.txt" (
for /f "usebackq eol=: delims=" %%F in ("inputFile.txt") do echo "%%~nxF" "%%~dpF"
)
You should read the built in help for the FOR command. Type help for or for /? from a command prompt to get help. That strategy works for pretty much for all commands.
In powershell, this little script should do the trick. In the first line, just specify the name of the text file that contains all the file paths.
$filelist="c:\temp\filelist.txt"
foreach($L in Get-Content $filelist) {
$i = $L.length - $L.lastindexof('\') -1
$fname=$L.substring($L.length - $i, $i)
echo ($fname + ' ' + $L)
}
If you don't have powershell installed on your machine, check out http://technet.microsoft.com/en-us/library/hh847837.aspx.
#ECHO OFF
SETLOCAL
(
FOR /f "delims=" %%i IN (yourfile.txt) DO ECHO %%~nxi %%i
)>newfile.txt
GOTO :EOF
No big drama - all on one active line, but spaced for clarity
I want to write a Windows batch file script that will loop through a text file of FILE PATHS, do some work using data from each file path, then ultimately delete the file.
I started by running the FORFILES command and sending its output (the #PATH parameter is the full path of any file it matches) to a text file (results.txt).
I end up with a results.txt file like this:
"C:/Windows/Dir1/fileA.log"
"C:/Windows/Dir1/fileA.log"
"C:/Windows/Dir2/fileC.log"
"C:/Windows/Dir3/fileB.log"
What I want to do is:
Use a FOR loop and read each line in the results.txt file
For each line (file path), strip out the directory name that the log file is sitting in (ie: Dir1, Dir2, etc..) and create a directory with that SAME name in a different location (ie. D:/Archive/Backups/Dir1, D:/Archive/Backups/Dir2, etc..) -- assuming the directory doesn't exist.
Move the actual .log file to a zip file in that directory [I have code to do this].
Delete the .log file from its original location. [Pretty straightforward]
I'm having trouble figuring out the best way to accomplish the first 2 steps. My FOR loop seems to stop after reading the very first line:
FOR /F "tokens=1,2,3,4,5,6,7,8,9,10 delims=\" %%G in ("results.txt") DO (
...
)
You don't want to parse the path with the tokens/delims options because you don't know how many directory levels you are dealing with. You want to preserve each line in its entirety. TO parse the path you want to use the FOR variable modifiers. (type HELP FOR from the command line and look at the last section of the output)
%%~pG gives the path (without the drive or file name). If we then strip off the last \, we can go through another FOR iteration and get the name (and possible extension) of the file's directory by using %%~nxA.
The toggling of delayed expansion is just to protect against a possible ! in the path. If you know that no path contains ! then you can simply enable delayed expansion at the top of the script and be done with it.
EDIT - this code has been modified significantly since Aacini pointed out that I misread the requirements. It should satisfy the requirements now.
for /f "usebackq delims=" %%G in ("results.txt") do (
set "myPath=%~pG"
setlocal enableDelayedExpansion
for /f "eol=: delims=" %%A in ("!myPath:~0,-1!") do (
endlocal
if not exist d:\Archive\Backups\%%~nxA md d:\Archive\Backups\%%~nxA
rem ::zip %%G into zip file in the D: location
rem ::you should be able to create the zip with the move option
rem ::so you don't have to del the file
)
)
I wrote this to timestamp files before offloading to SFTP.
Hope you find it useful.
The timestamp coding may seem irrelevant to your issue, but I left it because it's a good example of dissecting the filename itself.
I suggest you put an ECHO in front of the REN command for testing. Different shells may have different results.
In the end, the delayedexpansion command wasn't necessary. It was the sub-routine that fixed my issues with variables inside the loop. That could possibly be because of my OS ver. (Win 8.1) - It wouldn't hurt to leave it.
#echo off
cls
setlocal enabledelayedexpansion
if %time:~0,2% geq 10 set TIMESTAMP=%date:~10,4%%date:~4,2%%date:~7,2%_%time:~0,2%%time:~3,2%%time:~6,2%
if %time:~0,2% leq 9 set TIMESTAMP=%date:~10,4%%date:~4,2%%date:~7,2%_0%time:~1,1%%time:~3,2%%time:~6,2%
echo TimeStamp=%TIMESTAMP%
echo.
for %%G in (*.txt) do (
set OLDNAME=%%G
call :MXYZPTLK
)
dir *.txt
goto :EOF
:MXYZPTLK
echo OldName=%OLDNAME%
ren %OLDNAME% %OLDNAME:~0,-4%_%TIMESTAMP%%OLDNAME:~-4,4%
echo.
:END
You have two minor problems:
The path separator in the file is '/' but you use '\' in the for loop.
The quotes around "results.txt" stop it working.
This works. Don't write quotes to results.txt and you won't get a quote at the end of the filename.
#echo off
FOR /F "tokens=3,4 delims=/" %%I in (results.txt) DO (
REM Directory
echo %%I
REM File
echo %%J
)