Create subfolders/zip every 20 files automatically - windows

I have a huge folder that I would like to transform into several small ones of 20 files each and compressed. I would like to do this automatically with a batch file and 7zip.
To achieve this I thought about 2 steps:
1. Create a sub-folder for each 20 files (source) :
#echo off
set /a counter=1
set /a filesperfolder=20
cd "C:\Users\Desktop\dir\"
:loopstart
set dirname=dir_%counter%
md %dirname%
echo %dirname%
dir /b | findstr /v /i "dir_*"> %temp%\temp.txt && for /l %%l in (1,1,%filesperfolder%) do #for /f "tokens=1,2* delims=:" %%a in ('findstr /n /r "^" %temp%\temp.txt ^| findstr /r "^%%l:"') do #move %%b %dirname%\%%b >nul
set /a counter=%counter%+1
for /f "tokens=*" %%a in ('type %temp%\temp.txt ^| find /c /v ""') do set _filesmoved=%%a
del %temp%\temp.txt
IF %_filesmoved% LSS 20 goto done
goto loopstart
:done
cls
echo All files were moved!!
pause
exit
Unfortunately this does not work : The syntax of the command is incorrect.. I've tried debugging the script by removing the #echo off and it tells me that the dir /b | findstr /v /i "dir_*"> %temp%\temp.txt && for /l %%l in (1,1,%filesperfolder%) do #for /f "tokens=1,2* delims=:" %%a in ('findstr /n /r "^" %temp%\temp.txt ^| findstr /r "^%%l:"') do #move %%b %dirname%\%%b >nul part is not working (same error). This part is quite fuzzy for me and a little help would be welcome.
I specify that the file temp.txt contains my complete list of files, without any separator (one file per line).
--- UPDATE: filenames must not contain spaces ---
2. Compress all these subfolders one by one with (source) :
for /D %d in (*.*) do 7z a -tzip "%d.zip" ".\%d\*"
Do you have an idea for (I summarize) : create, for a large number of files, subfolders of 20 files in order to compress each subfolder one by one.
If you have any idea on how to compress each 20 files directly (without going through the creation of subfolders) I also agree!
Thank you in advance for your help!

Resolved!
I found my mistake. In reality my files included spaces that the script could not fit.
So, to answer my basic question which was: in a big folder filled with thousands of files, how to compress each 20 files in subfolders? it is necessary (to my knowledge) to:
Create a sub-folder for each 20 files whose name does not have spaces (code in my question)
Compress all these subfolders one by one (code in my question)
And it's done!
Do not hesitate to suggest a solution if you know of a "faster" one.

No need to create folders. Just iterate over the files and add every single file to it's designated zip file (instead of copying every 20 files into a folder and zip that folder)
Use two counters to keep track of (source)files and zip files. One counter for the files (use the Modulo-operator to check for each 20) and one for the zip filenames. I started the zip filenames with 10001 instead of 1 and took the last four chars of it to get sortable file names (0001.zip etc).
#echo off
setlocal enabledelayedexpansion
set filesperfolder=20
cd /d "%cd%"
set "zipDest=C:\temp"
set filecounter=0
set foldercounter=10001
for %%a in (*) do (
set /a filecounter+=1
set /a check=filecounter %% filesperfolder
if !check! == 0 set /a foldercounter+=1
ECHO 7z a -tzip "%zipDest%\!foldercounter:~-4!.zip" "%%a"
)
Obviously adapt the cd /d line (source folder), destination folder (Attention: don't use the source folder, or your zip files might be zipped again into other zip files), and maybe the zip-filename (like "%zipDest%\MyZips-!foldercounter:~-4!.zip")
NOTE: I disarmed the 7zip command with ECHO for testing and troubleshooting. Remove the ECHO when the output satisfies you.
To delete each file after successfully zipping them, change the zip line to:
7z a -tzip "%zipDest%\!foldercounter:~-4!.zip" "%%a" && del "%%a"
The quotes should take care of any spaces.

Related

How to copy newest 100 files from thousands of subfolders to another folder?

I have to copy the first 100 files with a specific file extension to another folder daily.
The source folder looks like this:
sourcefolder\ParentFolderA
├───folder1
│ └──────file_a.dat
├───folder2
│ └──────file_b.dat
└───folder3
└──────file_c.dat
I need to grab the newest 100 .dat files – there is only one .dat file in each folder UNDER the ParentFolderA – and copy them to a new folder.
This is what I have so far:
#echo off
setlocal enableextensions enabledelayedexpansion
set /a "index = 0"
set /a "count = 99"
set "source=sourcefolder\ParentFolderA\"
set "destination=destinationfolder\somerandomFoldername"
:whileloop
if %index% leq %count% (
echo %index%
for /R "%source%" %%f in (*.dat) do copy %%f "%destination%"
set /a "index = index + 1"
goto :whileloop
)
endlocal
timeout 10
Some of it will be taken out as I just have it in there to help me while I am writing it. The end behavior is to get those 100 newest .dat files. Currently this is grabbing ALL the .dat files in every subfolder, but it never terminates because it doesn´t have a chance to get to the incrementing portion of code. (There are thousands of folders it would have to go through before it got to that step).
Can someone give me some tips or an advice on where/how I can achieve the desired effect?
You need a list of all your files which is sortable by date/time. There isn't a command capable to do this recursive in cmd. But it's possible with a trick, I used in this answer: Temporarily set the date format to a sortable format, get the list and set it back to the original format:
#echo off
setlocal EnableDelayedExpansion
set /a "count=100"
set "source=sourcefolder\ParentFolderA\"
set "destination=destinationfolder\somerandomFoldername"
REM get current short date format:
for /f "tokens=2,*" %%a in ('reg query "HKCU\Control Panel\International" /v sShortDate') do set orig-format=%%b
REM set short format to yyyy.MM.dd:
reg add "HKCU\Control Panel\International" /v sShortDate /d "yyyy.MM.dd" /f >nul
REM get a recursive listing with format "YYYY.MM.DD hh:mm <full qualified file name>":
(for /F "delims=" %%a in ('dir /a-d /T:W /S /B "%source%\*"') do #echo %%~Ta "%%a")|sort /r>report.csv
REM set short date format back to original settings:
reg add "HKCU\Control Panel\International" /v sShortDate /d "%orig-format%" /f >nul
REM copy the first %count% files:
set n=0
for /f "tokens=2,*" %%a in (report.csv) do (
set /a n+=1
if !n! gtr %count% goto :done
ECHO copy %%b "%destination%\"
)
:done
echo done.
Adapt /T:W to your needs.
Note: I "disarmed" the copy command for security reasons. When it works as you want it to, just remove the ECHO.
(sorry to quote the filename within the for loop. I know it's bad habit, but it
is necessary because sort manages to add a space to the end of each line.)
#ECHO OFF
SETLOCAL
rem The following settings for the source directory, destination directory, target directory,
rem batch directory, filenames, output filename and temporary filename [if shown] are names
rem that I use for testing and deliberately include names which include spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
SET "destdir=u:\your results"
:: I used 20 files for my test
SET /a maxfiles=20
:: I chose to process all the .T* files
FOR /r "%sourcedir%" %%b IN (*.t*) DO COPY "%%b" "%destdir%\" >NUL
FOR /f "skip=%maxfiles%delims=" %%b IN ('DIR /b /a-d /o-d "%destdir%\*.t*" ') DO ECHO DEL "%destdir%\%%b"
GOTO :EOF
Since you seem to have no objections to building a complete copy of your target files, all that is needed is to delete all of the files from a directory listing in reverse-date order, skipping the required number of retentions.
The echo del is present for verification, to make sure the directory is correct. Change echo del to del to execute the deletions.

Batch Script to delete only files with .zip extension

I want to skip the latest 3 files created and delete the rest of the files. What I need is that if there is text, xml and zip files we need to delete only the zip files and leave the text and xml files behind and the total files left should be 3.If there are 3 or more files other than .zip files, delete all .zip files; if there are less, keep the newest .zip files so that there are 3 files left in total. Can anyone help. I am stuck with this
For example(inside bracket created date of files):
Folder A contains - aa.txt(2/1/18), bb.xml(3/1/18), cc.zip(4/1/18), dd.zip(2/1/18),ee.zip(5/1/18)
What I need after deleting is
aa.txt, bb.xml, ee.zip
This is what I have written
#ECHO OFF
SETLOCAL
SET "targetdir=C:\source"
SET /a retain=3
FOR /f "skip=%retain%delims=" %%a IN (
'dir /b /a-d /o-d "%targetdir%\*.zip" '
) DO DEL (DEL "%targetdir%\%%a.zip"
GOTO :EOF
Given the parameters of your question, a very simple method, although not the most efficient or speedy, would be to do this:
#Echo Off
Set "targetdir=C:\source"
Set "retain=3"
CD /D "%targetdir%" 2>Nul || Exit /B
For /F "Skip=%retain% Delims=" %%A In ('Dir /B/A-D/O-D/TC'
) Do If /I "%%~xA"==".zip" Del "%%A"

Search for files based on a list of partial names and copy them to a destination folder using windows shell

I am an absolute newbie to batch programming. I have posted this question after some amount of searching. Kindly guide me.
I have a folder containing a thousand images:
000001_x_abc1.jpg
000001_x_efg1.jpg
000001_x_hij1.jpg
000002_x_abc1.jpg
000002_x_efg1.jpg
000002_x_hij1.jpg
.
.
.
.
234562_x_abc2.jpg
234562_x_efg2.jpg
234562_x_hij2.jpg
Of theses files I have generated a 'list of files' that I need to pull out based on partial names i.e the numeric ID - first 6 numeric values in the file name e.g 234562*.jpg and copy them to a destination folder.
Note: Every numeric ID based search should give me 3 files and I need to copy all three. Any help would be appreciated.
I have tried the following code based on my search:
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET "DEST_DIR=my_desination"
SET "SEARCH_DIR=my_source"
FOR /F "tokens=*" %%a IN (%~dp0my_list.txt%) DO (
FOR /R "%SEARCH_DIR%" %%f IN (*%%a*) DO (
SET "SRC=%%~dpf"
SET DEST=!SRC:%SEARCH_DIR%=%DEST_DIR%!
xcopy /S /I "%%~f" "!DEST!"
)
)
And my list file is as below:
002631_*.jpg
054741_*.jpg
054992_*.jpg
055053_*.jpg
055054_*.jpg
055118_*.jpg
055267_*.jpg
055294_*.jpg
055382_*.jpg
055415_*.jpg
055466_*.jpg
055546_*.jpg
This is an example of copying specific files to a folder.
#ECHO OFF
set "SOURCE_DIR=%userprofile%\Desktop\Source"
set "DEST_DIR=%userprofile%\Desktop\Output"
set "FILENAMES_TO_COPY=test.txt test1.txt test2.txt"
pushd "%SOURCE_DIR%"
for %%F IN (%FILENAMES_TO_COPY%) do (
echo file "%%F"
xcopy /Y "%%F" "%DEST_DIR%\"
)
popd
pause
The script copies test.txt test1.txt and test2.txt from the folder Source, to the folder Output
Check out this page
Supposing the list file contains full file patterns one per line, the following should work for you:
#echo off
setlocal EnableExtensions DisableDelayedExpansion
rem Define constants here:
set "SOURCEDIR=D:\Data"
set "TARGETDIR=D:\BackUp"
set "LISTFILE=D:\files.lst"
cd /D "%SOURCEDIR%" || exit /B 1
for /F "usebackq delims= eol=|" %%L in ("%LISTFILE%") do (
for /F "delims= eol=|" %%F in ('dir /B "%%L"') do (
copy "%%~F" "%TARGETDIR%\%%~nxF"
)
)
endlocal
exit /B
(edited to reflect additional information in the question)
for /f %%a in (partial.txt) do copy %%a "x:\destination folder\"
for every entry in the textfile copy <entry from textfile> to the destination.
see for /? for more details

Searching for string in multiple zip archives

Here's what I have;
On a windows-based web server there are roughly 1,000 zip files, each with dozens of log files inside. I already have a script that goes through each archive and deletes all but one specific file type (in an attempt to save diskspace and delete things I don't need). Then, the script unzips each archive to their own folder. And I know how to code the reverse of that to zip them back when I'm done.
Here's what I need to figure out;
Once I run the previously mentioned script (we call it garbageman because it cleans out the garbage in the zip files) I need to go through the remaining 5 or 6 files in each of the newly created unzipped folders, and look for a specific string in each file. If I find the string, I delete everything that is not that string, and save it to a file called "export.txt" in that folder. Then, I move to the next unzipped file, and so on. Once completed, I need re-zip everything back together into their own archives
Here's what I have for code so far. Any help is extremely appreciated.
cd "C:\Program Files\7-Zip"
FOR %%c in (C:\Users\xxxxxx\Desktop\LogQueue\*.*) DO 7z d %%c "-x!xstore*" -r
FOR /R "C:\Users\xxxxxx\Desktop\LogQueue" %%I in ("*.zip") do (
"%ProgramFiles%\7-Zip\7z.exe" x -y -o"%%~dpnI" "%%~fI"
)
cd "C:\Users\xxxxxxx\Desktop\LogQueue"
FOR /R "C:\Users\xxxxxx\Desktop\LogQueue" %%I in ("*.*") do (
findstr "xxxxxxxx_eReceipt" %%~fI > %%~dpnI\export.txt
pause
)
for /d %%X in (*) do "c:\Program Files\7-Zip\7z.exe" a "%%X.zip" "%%X\"
This is an edit:
This script should do what you want.
Only Change sourcedir and mystring variables.
export.txt will be inside a file called Storage in the root directory of the batch.
:ScriptA
#ECHO ON
MKDIR "%CD%\Storage"
MKDIR "%USERPROFILE%\Desktop\Outx"
GOTO :ScriptB
:ScriptB
::REM ONLY CHANGE
SET "sourcedir=%USERPROFILE%\Desktop\Test"
FOR /R "%sourcedir%\" %%a in (*.txt) do copy "%%a" "%CD%\Storage"
:ScriptC
:ScriptC
#ECHO OFF
SETLOCAL
SET "VARA=%CD%\Storage"
SET "VARB=%USERPROFILE%\Desktop\Out"
::REM ONLY CHANGE
SET "mystring=PUT_STRING_HERE"
FOR %%a IN ("%VARA%\*.txt") DO FINDSTR "%mystring%" "%%a">nul&IF NOT ERRORLEVEL 1 FINDSTR "%mystring%" "%%a">"%VARB%\%%~nxa"
DEL /F "%CD%\Storage\*.txt"
GOTO :ScriptD
:ScriptD
#ECHO ON
COPY /B "%USERPROFILE%\Desktop\Outx\*.txt" "%CD%\Storage\export.txt"
RD /S /Q "%USERPROFILE%\Desktop\Outx"
goto :eof

Copying files with name containing just numbers using a batchfile

I have a Directory with a deep Directory->Sub-directory tree structure. I need to write a batch file to copy all the numbered files (files with names as digits and no alphabetic characters) from all the sub-directories.
For example, a sub-directory might contain the following files:
WR10091.txt
AX10091.htm
10091.txt
AX10091.xml
10091.xml
I need to copy 10091.txt and 10091.xml to another location. I can copy files like AX10091.xml and AX10091.htm by specifying AX*.*. But I cannot figure out how to copy just numbered files with no alphabetic characters. There are thousands of directories and the directory structure does not have any pattern (the depth of a tree branch can vary considerably).
Any help will be appreciated.
#echo off
setlocal enableextensions disabledelayedexpansion
set "source=%cd%"
set "target=x:\target\folder"
for /r "%source%" %%a in (*) do (
(for /f "delims=0123456789" %%b in ("%%~na") do (
break
)) || echo copy "%%~fa" "%target%"
)
In this code the for %%a will iterate over all the files under the indicated folder. For each of them, the for /f %%b will try to tokenize the file name (%%~na) using numbers as delimiters. If the file name only contains numbers, there will be nothing to process (only delimiters) and the inner for raises errorlevel. This is checked with conditional execution (the code after the || is executed if the previous command fails) and if errorlevel was raised the copy operation is echoed to console.
If the output is correct, remove the echo to perform the copy.
note: the break in the inner for loop is included just to have a command that does nothing when files with non numeric names are found.
#echo off
for /f "tokens=* delims=" %%a in ('dir /b /s /a:-d "*"') do (
echo %%~na|findstr /e /b /r "[1-9]*" >nul 2>nul && (
copy %%~fa c:\somewhere\
)
)
should be executed in the same directory as the files.
for /f "delims=" %%a in ('dir /b/s/a-d ^| findstr /reic:"\\[0-9][0-9]*\..*" /c:"\\[0-9][0-9]*"') do copy "%%~a" "targetDir"
This might not work with XP and/or Vista, but this can be fixed if needed (see What are the undocumented features and limitations of the Windows FINDSTR command).

Resources