Bash robocopy from text file containing path to folders and files to copy with special characters - bash

I had a bash xcopy script, but got a lot of issues. After a lot of searching and reading I guess robocopy was a better tool to do the copy.
(The script must run on windows 10 computers without installing anything else and without internet access)
I'm trying to make a bash script that copy (with robocopy) some local network folders and files to a local custom directory. The aim is to be able to access to the files off from the local network.
The path to folders and files are stored inside a txt file (each line = a path)
I want to keep the structure of folder I save locally.
For example the folder X:\Path\to some\local\network\folder\with\some & characters\ will result in C:\PathTolocalFolder\Path\to some\local\network\folder\with\some & characters\ (without the X:\ letter)
Based on many similar questions (but not all at the same time) I have done this :
#echo off
SETLOCAL EnableDelayedExpansion
cls
cd C:
chcp 28591 > nul
for /f "delims=*" %%a in ('type "X:\path with spaces & specials characters\List.txt"') do (
REM echo %%a
REM echo !%%a!
echo %%a to C:\PathTolocalFolder!%%a!
ROBOCOPY "%%~a" "C:\PathTolocalFolder!%%a!" /S /A+:RA /R:1 /W:5
)
It is partially a success, but :
As there are special characters everywhere in paths and files names, I got some issues. Specially with & characters. My double quotes doesn't solve the problem. How could I go better?
For some cases, I want to save some files but not the whole directory where they are. The full path to the file is inside the text file. But as robocopy needs to add a space between folder path and file filter I have do some manipulation. How can I detect and extract the file name when there is one to adapt the robocopy command?
I want to use an exclusion list like I was doing before with xcopy. But robocopy doesn't accept a file in input for exclusions. I tried this to extract the exclusion file:
for /f "usebackq tokens*" %%D in ("C:\path to exclusion file\exclusions.txt") do (
if NOT "!dirs!"=="" (
Set dirs=!dirs! "%%D"
else (
Set dirs ="%%D"
)
)
But doesn't really know what I am doing and how to combine with the first part.
Bonus questions I'm using the robocopy log file functionality (removed from below) is there a way to archive (by adding the date in the name for example) previous log file before creating the new one? Is it possible to remove the progress percents in the log file but to display it in the terminal instead? How to use the "/np" option for log file but not for terminal display?
It's hard to me to understand how the delayed variables are working in batch files and how the different methods to read a file or variable are working.
Any help is welcome :)
Sorry for my bad English skills
thank for having read

Related

How can I remove a specific part of many folders names using .bat files?

My university is using a proprietary system that outputs data in a very specific way in which each "module" is output into folders following the structure (4 digit numeral) - (name of module)
ex: 5574-CHEM104
I need to remove the name and hyphen so that only the numeral remains:
5574-CHEM104 > 5574
The problem is that there's thousands of these folders and there's no way I could do it by hand. I'm having difficulty trying to automate the process, so if anyone could at least point out a command I could look into it would help immensely
I've tried the REN command, putting "REN 5574-CHEM104 5574", but it only works for one folder. There's thousands of folders, each with different numerals, under "CHEM104", for example, and I need for the program to rename the folder no matter the original name into the first 4 original numerals, which I can't figure out. Thanks!
#ECHO OFF
SETLOCAL
rem The following setting for the directory is a name
rem that I use for testing and deliberately includes spaces to make sure
rem that the process works using such names. These will need to be changed to suit your situation.
SET "sourcedir=u:\your files"
FOR /f "delims=" %%e IN (
'dir /b /ad "%sourcedir%\*" '
) DO FOR /f "delims=-" %%o IN ("%%e") DO ECHO REN "%sourcedir%\%%e" "%%o"
)
GOTO :EOF
Always verify against a test directory before applying to real data.
The required REN commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO REN to REN to actually rename the directories.
Using a list of the directory-names, names only (/b) and directories only (/ad), tokenise the name using - as a delimiter and execute the rename using the token assigned to %%o (default is first token)

Parsing every file in submap recursively to output folder whilst maintaining relativity

I'm using the following batch code to convert all files in a certain directory if the target file doesn't already exist however I'm stuck at getting this to run through every submap and file within that (and keep the output relative with that submap)
So I currently use this:
for %%f in (input/textures/*.*) do ( IF NOT EXIST "ouput/textures/%%~nf.dds" (
"bin/ThempImageParser.exe" "input/textures/%%f" "ouput/textures/%%~nf.dds"
)
)
This works perfectly for a single folder (as was intended), it takes all the files in that specific folder, and passes them as arguments to my executable, which then outputs the file on the path of the second argument.
However this also contains a flaw (this is an additional problem though..) as it does not work if the output -folder- does not exist, so if possible I'd also want it to create the folder if need be.
I've found some batch documentation (I really don't have much experience with Batch) showing me a command called FORFILES and the /R parameter, however I couldn't adjust this so it'd keep the relative paths for the output too, it'd require string manipulation and I have no clue on how to do that.
So the result I'm after is something like this, it takes any file deeper than "input/textures/ for example:
input/textures/some/very/deep/submap/why/does/it/go/on/myfile.anyExtension
it should then take that file (and relative path) and basically change "input" with "output" and replace the file extension with .dds like this:
ouput/textures/some/very/deep/submap/why/does/it/go/on/myfile.dds
and pass those two strings to my executable.
#ECHO Off
SETLOCAL ENABLEDELAYEDEXPANSION
SET "sourcedir=U:\sourcedir\t w o"
SET "destdir=U:\destdir\wherever\something"
FOR /f "delims=" %%a IN ('xcopy /y /L /s "%sourcedir%\*"') DO (
SET "destfile=%%a"
SET "destfile=!destfile:*%sourcedir%=%destdir%!"
IF /i "%%a" neq "!destfile!" (
FOR %%m IN ("!destfile!") DO IF NOT EXIST "%%~dpm%%~na.dds" (
ECHO MD "%%~dpm"
ECHO "bin\ThempImageParser.exe" "%%a" "%%~dpm%%~na.dds"
)
)
)
GOTO :EOF
You would need to change the settings of sourcedir and destdir to suit your circumstances.
First, perform an xcopy with the /L option to list-only the individual fullnames of files that would be copied by the xcopy.
Assign each name found from %%a to destfile, then remove all characters before the source-directoryname from that filename, and replace that string with the destination directoryname.
This will yield the destination name for the file (with the original extension). The only exception will be the very last output line, which is a count-of-files report. Since this line will not contain the source directoryname, the replacement will not take place, so %%a will be the same as !destfile! - so we eliminate that.
Now assign the destination filename to a metavariable so we can select its various parts, and if the filename made from the destination drive and pathname, the name part of the original file and .dds does not exist, then make the destination directoryname and execute the imageparser, providing the desired output filename.
Note that these last two are ECHOed instead of being executed for testing purposes. Remove the ECHOes to actually perform the command.
Note that / is a switch-indicator, \ is a directory-separator.
Note that MD will report an error if the directory already exists. Append 2>nul to the end of the md command to suppress that error message.

Need a BAT file to copy & rename all files in specific tree

I need to create a .bat that runs through a multilayered directory... copying certain files that contain the following suffix: '.full.jpg' to save as '.jpg'
What I've tried:
copy /y "C:\Users\myname\Desktop\maindir\*.full.jpg" "C:\Users\myname\Desktop\maindir\*.jpg"
However, I cannot get it to work.
The .bat is located in the 'maindir' directory and ran from the terminal (cmd).
Here's an example scenario that maps closely to mine:
Existing Files:
C:\Users\myname\Desktop\maindir\a\a\picture1.full.jpg
C:\Users\myname\Desktop\maindir\a\a\picture3.full.jpg
C:\Users\myname\Desktop\maindir\a\b\picturea.full.jpg
C:\Users\myname\Desktop\maindir\a\b\pic1.full.jpg
C:\Users\myname\Desktop\maindir\b\a\foto.full.jpg
C:\Users\myname\Desktop\maindir\b\a\photo.full.jpg
C:\Users\myname\Desktop\maindir\b\b\pic1.full.jpg
C:\Users\myname\Desktop\maindir\b\c\pi2.full.jpg
Example Output Wanted:
C:\Users\myname\Desktop\maindir\a\a\picture1.full.jpg
C:\Users\myname\Desktop\maindir\a\a\picture1.jpg
C:\Users\myname\Desktop\maindir\a\a\picture3.full.jpg
C:\Users\myname\Desktop\maindir\a\a\picture3.jpg
C:\Users\myname\Desktop\maindir\a\b\picturea.full.jpg
C:\Users\myname\Desktop\maindir\a\b\picturea.jpg
C:\Users\myname\Desktop\maindir\a\b\pic1.full.jpg
C:\Users\myname\Desktop\maindir\a\b\pic1.jpg
C:\Users\myname\Desktop\maindir\b\a\foto.full.jpg
C:\Users\myname\Desktop\maindir\b\a\foto.jpg
C:\Users\myname\Desktop\maindir\b\a\photo.full.jpg
C:\Users\myname\Desktop\maindir\b\a\photo.jpg
C:\Users\myname\Desktop\maindir\b\b\pic1.full.jpg
C:\Users\myname\Desktop\maindir\b\b\pic1.jpg
C:\Users\myname\Desktop\maindir\b\c\pi2.full.jpg
C:\Users\myname\Desktop\maindir\b\c\pi2.jpg
I'd appreciate any help towards this as I haven't been able to do it yet. I will run across a directory structure whereby the top level directory will contain 15+ directories and each containing 20+ directories with 100+ files in each lowest directory.
Thanks.
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET "sourcedir=u:\Users\myname\Desktop\maindir"
FOR /r "%sourcedir%" %%a IN (*.full.jpg) DO (
FOR %%b IN ("%%~dpna") DO ECHO(COPY "%%a" "%%~dpnb.jpg"
)
GOTO :EOF
The inner for examines the drive-path-name only of the complete filename in %%a (ie. it drops the .jpg) and delivers the drive-path-name of that name (ie. drops the .full) to which you add .jpg and job done.
You would need to change the setting of sourcedir to suit your circumstances.
The required COPY commands are merely ECHOed for testing purposes. After you've verified that the commands are correct, change ECHO(COPY to COPY to actually copy the files.

How to write a batch file to crop all the pdfs in a directory and store them in a new directory?

I'm totally unfamiliar with batch files but I'm pretty sure I need one for the task at hand:
I want to run pdfcrop for all the files in a particular directory and store the cropped files in a new directory. New directory is called 'croppedfiles' and if it doesn't already exist in the location where the pdfs are stored then such a directory is created and the output files are stored there.
I'd like the output files to have the same name as old files with the addition of '_cpp' at the end.
syntax for pdfcrop is just pdfcrop input.pdf output.pdf
Referring to the poor material you provided for this question - I mean only your for cycle quotation - I can only suggest a pair of tips: first you can narrow the group of files on which the for cycle is going to work, using a piped find ".pdf" command. Then you can use another nested for cycle to obtain the name of the file to be processed by pdfcrop and to set it as a variable to be used for the output path. Here is the example script:
for /f "delims=" %%g in ('dir ^"[set your desired path]^" ^| find
^".pdf^"') do ( set VAR=%%g for /f "tokens=1,2 delims=." %%m in
('!VAR!') do (mkdir "[chose your subdirectory name]" pdfcrop "!VAR!" "[subdirectory]\%%m[set
your additional characters].%%n" ) )
I hope it works, because I have been not able to test it.

Bulk renaming files in relation to the folder names

I am very new to coding and bulk processes but i am looking for a command line SPECIFICALLY for windows command prompt and i am wondering if such a thing exists. So I have a folder containing 111 subfolders, with each subfolder containing between 20 and 40 png image files. Each subfolder is named 001-111 accordingly and the png files are ordered how i want them, however i am looking for a command line that would be able to quickly and efficiently name all the pngs in the folders to the name of the folder followed by the png number in brackets
e.g. for folder 037, i would want the png's to be renamed to: 037(1), 037(2), 037(3) etc...
I am hoping for the best although i am unsure such a code may not be possible or be simply done.
Also if you come up with a code that achieves this process, it would be great if you could reply with the simple command line that i could use rather than a full explanation because i am new to coding and far from fluent with the language or terms or how things work. I know this same process can be achieved by going select all>rename (ctrl a>f2) and renaming to the folder name however i need to use this process frequently and dont want to have to open each folder, i would rather have a command line for cmd that would do it swiftly
Thank you and a simple answer would be greatly appreciated
#ECHO OFF
SETLOCAL ENABLEDELAYEDEXPANSION
SET "parentdir=u:\parent"
FOR /l %%a IN (1001,1,1111) DO (
SET dir=%%a&SET "dir=!dir:~1!"
FOR /f "delims=" %%i IN ('dir /a-d /b "%parentdir%\!dir!\*.png" 2^>nul') DO (
ECHO REN "%parentdir%\!dir!\%%~nxi" "!dir!(%%~ni)%%~xi"
)
)
GOTO :EOF
Test results:
Starting directory :
u:\parent\001\1.png
u:\parent\037\1.png
u:\parent\037\2.png
u:\parent\111\999 with spaces in name.png
Script response
REN "u:\parent\001\1.png" "001(1).png"
REN "u:\parent\037\1.png" "037(1).png"
REN "u:\parent\037\2.png" "037(2).png"
REN "u:\parent\111\999 with spaces in name.png" "111(999 with spaces in name).png"
Obviously, you'd need to replace the value assigned to parentdir with your actual target directory name.
The script will report the renames it proposes to do. To actually invoke the rename remove the ECHO keyword.
I would create a batch file like so:
renamepng.bat:
cd %%1
if ERRORLEVEL 1 goto end
for %f in *.png do mv "%f" "%%1(%f).png"
cd ..
:end
This will attempt to cd to the directory name provided on the command line, abort if that fails, then rename all the .png files and return to the previous directory
then call it like so:
for %d in ??? do call renamepng.bat %d
which will loop through all 3-character file and directory names in the current directory, can call the batch file on each one. Using call instead of just the batch file name causes execution to return to the loop when the batch finishes.

Resources