I'm trying to create a script that will automatically backup a complete directory tree. I also want it to work for incremental backups. Basically, it wil work like this:
If file is in both source and destination and they are different, source file will be copied
If file is in both source and destination and they are the same, nothing will be copied
If file is only in the source, source file will be copied
If file is only in the destination, destination file will be deleted.
I'm still new to shell scripting and I'm not sure how I could implement this. Any ideas? Windows batch scripts would be better, but shell scripts that run on Cygwin are also fine.
Take a look at rsync tool - it was designed to minimize traffic during files synchronization.
Also, probably "cp" with "-u" argument will be useful:
-u, --update
copy only when the SOURCE file is newer than the destination file or when the destination file is missing
For windows batch scripting xcopy (native to windows) or robocopy (a free download in windows) both work extremely well.
An example xcopy script (in a .bat file):
#echo off
:: /L = list only.
:: /v=Verify, /y=No prompting /s=subdirs /z=Network mode (supports bad)
:: /i=Tells xcopy dest is folder /f=Display names /d=Copy only changed
echo Backing up projects...
xcopy e:\projects h:\projects /V /Y /S /Z /I /F /D
It will even support orphaned files (if you delete something from your source you no longer need a copy in the backup). Xcopy is typically fine for your needs until you deal with sync between NTFS and Fat32 file systems - the later only has a 2 second resolution and problems with daily savings time so you occasionally run into issues: (a) on time change day you might not get a backup of a changed file - depends on change-regularity of course or you might get a backup of all files even though none have changed (b) because of time resolution some files may backup even though they haven't changed.
Related
How can I copy all files/folders in a directory using a batch file but omit one? I know I could always delete it after the fact but it is an extremely large folder and I don't want to waste the time copying it. There are numerous files in the location I want to copy so doing it one individually would be unrealistic. Thank yoU!
robocopy "source" "destination" files_to_copy.ext /e /xd "folder_to_exclude"
robocopy is an upgraded version of xcopy, thus, is more useful. You can check the complete usage by typing robocopy /?, or from the microsoft docs, or from ss64.
EDIT: Almost forgot, robocopy accepts wildcards, so you can type *.* to copy every files. The line I suggest already have the option /e, which will copy all subfolders, even the empty ones. /xf:file_to_exclude.ext can also be used to skip a specific file type, eg. /xf:"*.txt" to exclude all the .txt files.
xf is used to exclude files, xd to exclude folders.
I'd like to move my program files and user documents from C:\Windows.old to C:\, completely replacing what's currently in C:. The old and new OSs on my computer are both windows 10, I just had to make a clean install because I couldn't get it to start after a restoration that went wrong. How can I do this operation using the command prompt at windows startup? Is there a command to do this : move, merge folders, replace files with the same name?
Thank you!
First, don't move program files just like that. Programs don't store data only here. Absence of this data might prevent some programs from running, especially licensed ones because they will think it's an attempt of running the executable directly on another computer. A better solution is to reinstall the programs, as it will put everything in the good place to make it is running OK.
Here's the command:
xcopy <Source> <Destination> /s /y
Replace <Source> and <Destination> with their appropriated values. Example:
xcopy C:\Windows.old\Users\David C:\Users\David /s /y
xcopy will ask if it is a file or folder destination. Answer that it is a folder by typing the appropriate letter.
Explanations:The /s command line switch means to copy the entire file and folder structure, while /y means to not ask when overriding files. If you want to look even more geeky, add /f.
Morning All,
We're currently running some software for our users which is failing to copy their PST's on to our servers, it's too intrusive, requires client side software and configuration and its paid for!
I'm used to free/open source software and love command line batch files compared, as I find them easier to automate and also add to scheduled tasks without worrying about users input.
I have found hobocopy which works great! - but only if you list : source folder, dest folder and then file type... my script searches the C:\ drive, finds PST files and lists the full file path. Hobocopy doesn't seem to handle this.
Below is my script:
#echo off
REM ### COPY HOBOCOPY TO WINDOWS DIR #####
if not exist C:\windows\Hobocopy.exe xcopy \\icao-supp-01\support\hobocopy\hobocopy.exe C:\windows
REM ### SCAN SYSTEM FOR LOCAL PST FILES ####
dir *.pst /s /b > C:\temp\pst.txt
REM ### RUN HOBOCOPY TO COPY PST FILES ####
For /f %f in (C:\temp\pst.txt) do hobocopy /y %f P:\
**HERE IS THE OUTPUT OF C:\TEMP\PST.TXT
C:\Jdeane.pst
C:\Games\IGNORE1.pst
C:\Windows\ModemLogs\fake2.pst**
It wont copy the file paths e.g. :
hobocopy /y C:\Jdeane.pst P: wont work. However, hobocopy /y C:\ P:\ *.pst would work.
My goal:
Search the C:\ drive for PST files and then have them backed up on a schedule to the servers.
Thanks in advance!
(PS. Were running Windows 7 x64 and outlook 2010 if it makes a difference, and the users will NOT save their PST's to our servers).
I'm the author of hobocopy. It is written to expect source folder, destination folder, and a file selector. So you're not going to be able to use a full path. That said, you can use the flags that can be found by running "help for" at the command prompt to break apart the path you find into directory and file components. Something like %~nf in your case, I believe.
We have a file share that has a directory containing all of our build version direcotries named like this
WebApp_20140702.1 first number being date second number being the build count for that date
these are then contained in the following directory
\\server\share\product\
What i need to do from a virtual machine is create a batch file that can check the target location on my vm ie. c:\product\ see if it has the latest version from the network share either by comparing the file names or dates, then copy the new version and delete the old if necessary.
So far i can copy the folder over using xcopy but that's about the extent of my dos/batch file knowledge iv had a look around for a while but haven't been able to see anything that i can use
This is what i have so far, as you can see though i dont know how to do the comparison between the two directories as explained above.
xcopy "\\server\share\webapp" "c:\users\username\desktop\webapp" /E /K
I did try to use just /D at the end and just copying the directories from \\server\share\product\ that had a later date than the target but it ended up just copying the whole directory.
EDIT : to make my self clear
i need to find out if i have the latest sub directory but no matter what i do it always copies all the sub directories from \\server\share\product\
ie. the \\server\share\WebApp directory will have the following sub dirs
..\WebApp_20140628.1\
..\WebApp_20140628.2\
..\WebApp_20140703.1\
and my vm will have the directory
c:\product\WebApp_20140628.2\
Now i need to be able to go into the file share see that it has a more up to date subdirectory i need to copy that directory to my vm and then delete the older one from my vm so i would then have
c:\product\WebApp_20140703.1\
OK i eventually found another question that wanted to do a similar thing and the answer worked exactly as i wanted it
Question can be found here: How to get the most recent file using a batch script in windows
i had to use xcopy instead of the copy used in the answer for the above question. here is my solution as well in case anyone needs something similar
(z is the mapped version of the network share i talk about in my question)
pushd "z:\WebApp\"
for /f "tokens=*" %%a in ('dir/b /od') do set newest=%%a
xcopy /e /k "%newest%" "c:\product\"
popd
im not sure if i actually need the popd command as i believe it just goes back to the directory set in the pushd command
Anyone know of a command line utility (or one that can run as a command line) that will collect all the .jpg files in a directory tree to a single folder, only copying files that change?
I started with Renamer, which is great for renaming files in their current directories, but fell short when I tried to mangle the path. This is probably because I don't know Renamer that well. I ended up creating a text file directory dump, then using a REGEX find / replace to create a batch file, but this is hardly efficient nor automated.
The REGEX:
(G:\DIR\DIR\)([0-9]+\)([0-9]+\)([0-9]+\)([0-9]+\)(p[0-9]+.jpg)
changed this
G:\DIR\DIR\00\00\00\00\p0000000000.jpg
to this
G:\DIR\DIR\p0000000000.jpg
(copy \1\2\3\4\5\6 \1\6) in the batch file.
I need to run the whole thing as a scheduled task without a real person logging in. Not really looking for a Zip file because I don't want to disturb the system processor, plus most of the files will not change from day to day. This is more of a file sync.
In a Windows command line you can do this:
for /R A %i IN (*.jpg) DO xcopy %i B /M /Y
Where A is the source directory and B is the destination directory. You need to have command extensions enabled, which I believe is the default.
A couple of notes from the comments:
If any of your paths could have spaces in you will need to add quotes around the second %i. This prevents the string being interpreted by the xcopy command as two separate parameters. You may need to do the same around A and B paths. Like this:
for /R "A" %%i IN (*.jpg) DO xcopy "%%i" "B" /M /Y
If you are putting this inside a .bat or .cmd file you will need to double the percentage like this.
for /R A %%i IN (*.jpg) DO xcopy %%i B /M /Y
The /M option on xcopy will only copy files with the Archive bit set and then unset this bit. This prevents the files being copied twice. If you have other processes that also alter this bit it may cause issues. It is also possible to use the /D option which compares the file's last modified time with that in the destination and only copies newer files.
I'm guessing you're on Windows from the path format.
I've not read the whole thing, but http://www.infionline.net/~wtnewton/batch/batguide.html#6a might help you.
The same page has dizzy.bat, (http://www.infionline.net/~wtnewton/batch/dizzy.bat) which should be trivial to edit to do what you want.
In a Unix environment I would use find or rsync (and maybe some features of the shell). Cygwin and MinGW come with find, maybe with rsync. You can also probably get a standalone port of find for Windows somewhere.
If the SOURCE shell variable is the directory containing subdirectories with files to copy, and the DEST shell variable is the directory to copy them to:
find $SOURCE -name \*.jpg -exec cp --update \{\} $DEST/ \;
find is by nature recursive. "-name \*.jpg" selects files that match that pattern. You can add additional conditions with -and. The --update option to the cp command (or -u) only bothers copying the file if changed or not yet copied. There are other options to cp that might be useful too.
If $SOURCE is the same as $DEST as in your DIR/DIR/ example, then find will also find the destination files (already copied), though this will be ok, cp will recognize that you are trying to copy the same file to itself and skip it, but if you want to avoid that wasted work you can use 'for' and 'if' (or something) to only run find on the subdirectories of DIR/DIR/.
You can also use rsync, which has options that can delete files from the destination directory if they have also been deleted from the source directory, and many other such variations.