I have this script that I use for backups. The problem is that it is kind of slow. I want to know if there is a diff command that stops when finds the first difference.
DocumentsFiles=("Books" "Comics" "Distros" "Emulators" "Facturas" "Facultad" "Laboral" "Mods" "Music" "Paintings" "Projects" "Scripts" "Tesis" "Torrents" "Utilities")
OriginDocumentsFile="E:\Documents\\"
DestinationDocumentsFile="F:\Files\Documents\\"
## loop file to file and copy in backup
for directory in "${DocumentsFiles[#]}"
do
RealOrigin="${OriginDocumentsFile}${directory}"
RealDestination="${DestinationDocumentsFile}${directory}"
echo $directory
if [ -a "$RealDestination" ]; then
echo ok
if diff -r $RealOrigin $RealDestination; then
echo "${directory} are equal!"
else
rm -rfv $RealDestination
cp -ruv $RealOrigin "${DestinationDocumentsFile}"
fi
else
cp -ruv $RealOrigin "${DestinationDocumentsFile}"
fi
done
diff -q reports "only when files differ" (per man diff), so I believe it'll stop after the first difference.
But this is a bit of an XY problem. Really you need a better backup program like rsync:
It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination.
From man rsync
Related
I have small script to copy the all the files from one directory (SRC) to another directory (DES). This below script is running perfectly.
#!/bin/bash
SRC="/home/user/dir1/*"
DES="/home/user/dir2/"
for file in "$SRC"
do
if [ -f "$file" ]
then
cp "$file" "$DES"
echo "$file -----> file copied"
fi
done
Now what i am thinking while copying files from one directory to another directory, how to skip the copying file if that file has already exist in (DES) directory with same name of (SRC) directory and continue the remaining file as usual from source to destination?
Here how to i use break and continue looping to perform this action?
Thanks,
I recommend to use rsync:
src="/home/user/dir1/"
dst="/home/user/dir2/"
rsync -rav --ignore-existing "${src}" "${dst}"
The switch --ignore-existing tells rsync to skip files which exist at the destination.
Why not just reduce the entire script to a oneliner?
cp -n /home/user/dir1/* /home/user/dir2/
The -n flag (--no-clobber) prevents cp from overwriting existing files.
If your real situation is more complicated, you can also take a look at rsync.
Decades ago I was a programmer (IBM assembly, Fortran, COBOL, MS DOS scripting, a bit of Visual Basic.) Thus I'm familiar with the generalities of IF-Then-Else, For loops, etc.
However, I'm now needing to delve into Bash for my current job, and I'm having a difficult time with syntax and appropriate commands for what I need.
I'm in need of a trivial (concept-wise) script, which will:
Determine if a specific folder (e.g., ~/Desktop/Archive Folder) exists on the user Desktop
If not, create it ("Archive")
Move all files/folders on desktop - except for ~/Desktop/Archive, into "Archive Folder" - AND appending a timestamp onto the end of the filenames being moved.
It is this very last piece - the timestamp addition - which is holding me up.
I'm hoping a clear and simple solution can be sent my way. Here is what I've come up with so far:
#!/bin/bash
shopt -s extglob
FOLDERARCH="Archive Folder"
cd ~/Desktop
if [ ! -d $"FOLDERARCH" ]; then
mkdir "$FOLDERARCH"
echo "$FOLDERARCH did not exist, was created"
fi
mv !(-d "$FOLDERARCH") "$FOLDERARCH"
One final note: the script above works (without the timestamp piece) yet also ends with the message
mv: rename Archive Folder to Folder/Archive Folder: Invalid argument
Why?
Any help will be deeply, deeply appreciated. Please assume I know essentially zilch about the BASH environment, cmds and their arguments - this first request for assistance marks my first step into the journey of becoming at least proficient.
Update
First: much gratitude for the replies I've gotten; they've been very useful.
I've now got was it essentially a working version, but with some oddities I do not understand and, after hours of attempted research, have yet to understand/solve.
I'm hoping for some insight; I feel I'm on the verge of making some real headway in comprehending, but these anomalies are hindering my progress. Here's my (working, with "issues") code so far:
shopt -s extglob
FOLDERARCH="Archives"
NEWARCH=$(date +%F_%T)
cd ~/Desktop
if [ ! -d $"FOLDERARCH" ]; then
mkdir "$FOLDERARCH"
echo "$FOLDERARCH did not exist, was created"
fi
mkdir "$FOLDERARCH/$NEWARCH"
mv !(-d "$FOLDERARCH") $FOLDERARCH/$NEWARCH
This in fact largely accomplishes my goal, but:
In the case where the desktop Archives folder already exists, I'm expecting the if-then construct to simply follow through (with no echo msg) to the following mkdir command, but instead the msg "Archives not exist, was created" msg is output anyway (erroneously). Any answers as to why?
The script completes with the following msg:
mv: rename Archives to Archives/2016-01-10_00:06:54/Archives: Invalid argument
I don't understand this at all; what should be happening is that all files/folders on the desktop EXCEPT the /Desktop/Archives folder should be moved into a newly created "subfolder" of /Desktop/Archives, e.g., /Desktop/Archives/2016-01-10_00:06:54. In fact, the move accomplishes my goal, but that the message arises makes no sense to me. What is the invalid argument?
One last note: at this point in my newbie-status I'm looking for code which is clear and easy to read, versus much more elegant/sophisticated one-line piped-command solutions. I look forward to working my way up to those in due time.
You have several options. One of the simplest is to loop over the directories below ~/Desktop and if they are not "$FOLDERARCH", move them to "$FOLDERARCH", e.g.:
for i in */; do
[ "$i" != "$FOLDERARCH"/ ] && mv "$i" "$FOLDERARCH"
done
I haven't run a test case, but something similar to the following should work.
#!/bin/bash
shopt -s extglob
FOLDERARCH="Archive Folder"
cd ~/Desktop || { printf "failed to change to '~/Destop'\n"; exit 1; }
if [ ! -d "$FOLDERARCH" ]; then
if mkdir "$FOLDERARCH" , then
echo "$FOLDERARCH did not exist, was created"
else
echo "error: failed to create '$FOLDERARCH'"
exit 1
fi
fi
for i in */; do
[ "$i" != "$FOLDERARCH"/ ] && mv "$i" "$FOLDERARCH"
done
I apologize, I forgot the datestamp portion. As pointed out in the comments, you can include the datestamp (set the format to your taste) with something similar to the following:
tstamp=$(date +%s)
for i in */; do
[ "$i" != "$FOLDERARCH"/ ] && mv "$i" "$FOLDERARCH/${i}_${tstamp}"
done
src_dir="/export/home/destination"
list_file="client_list_file.txt"
file=".csv"
echo "src directory="$src_dir
echo "list_file="$list_file
echo "file="$file
cd /export/home/destination
touch $list_file
x=`ls *$file | sort >$list_file`
if [ -s $list_file ]
then
echo "List File is available, archiving now"
y=`tar -cvf mystuff.tar $list_file`
else
echo "List File is not available"
fi
The above script is working fine and it's supposed to create a list file of all .csv files and tar's it.
However I am trying to do it from a different directory while running the script, so it should go to the destination directory and makes a list file with all the .csv in destination directory and make a .tar from the list file(i.e archive the list file)
So i am not sure what to change
there are a lot of tricks in filename handling. the one thing you should know is file naming under POSIX sucks. commands like ls or find may not return the expected result(but 99% of the time they will). so here is what you have to do to get the list of files truely:
for file in $src_dir/*.csv; do
echo `basename $file` >> $src_dir/$list_file
done
tar cvf $src_dir/mystuff.tar $src_dir/$list_file
maybe you should learn bash in a serious manner and try to google first before you asking question in SO next time.
http://www.gnu.org/software/bash/manual/html_node/index.html#SEC_Contents
http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO.html
I'm trying to create a script on my Raspberry's Xbian that will "watch" a folder and compress any folders with images I saved there from Google Image Search.
What I want: The script will move all folders with " - Google Search" in their name to a temp folder, rename them removing the " - Google Search" part, leaving only the subject of the search query. Then, it will sequentially number the files in each folder using the folder name / the search query as their new name. So, "Random file.jpg" and "anoth3r_rand0m_file.png" will become "search_topic_01.jpg" and "search_topic_02.jpg".
Then, they'll be all moved to another folder, an "Incoming Images" one, where ImageMagick will do its magic on them, at the same time moving them to a "Ready Images" folder.
Still with me?
Here's what I got so far, from bundling stuff I found online together with my limited knowledge of Bash scripting:
echo "making temp"
mkdir /media/dBox/downloads/Xyma/temp
wait
echo "moving files"
mv /media/dBox/downloads/Xyma/*Google\ Search /media/dBox/downloads/Xyma/temp
wait
echo "renaming folders"
rename s/\ -\ Google\ Search// /media/dBox/downloads/Xyma/temp/*
wait
echo "renaming files"
for dir in /media/dBox/downloads/Xyma/temp/; do
if test -d "$dir"; then
(
cd $dir
for file in *; do
newfile=$dir.${file#*.}
mv "$file" "$newfile"
done
)
fi
done
wait
echo "making ready subfolder"
mkdir /media/dBox/downloads/Xyma/temp/00_Unreg_Ready_Image_Folders
wait
echo "moving folders to ready folder"
mv /media/dBox/downloads/Xyma/temp/* /media/dBox/downloads/Xyma/00_Unreg_Ready_$
wait
echo "removing temp folder"
rmdir /media/dBox/downloads/Xyma/temp
...and let's just say "AAARGHRGHRGHHhh".
I'm sure that there must be an even simpler way, with, say, a five-word command and maybe two parameters, that will auto magically do everything and sprinkle it with stardust, or generally "a simpler and better way to do it", but it's currently slipping my mind.
So, I'm open to ideas and suggestions.
Any help? Anyone?
Help!
I need to make a recycle bin code using bash. Here is what I have done so far. My problem is that when I move a file with the same name into the trash folder it just overwrites the previous file. Can you give me any suggestions on how to approach this problem?
#!/bin/bash
mkdir -p "$HOME/Trash"
if [ $1 = -restore ]; then
while read file; do
mv $HOME/Trash/$2 /$file
done < try.txt
else
if [ $1 = -restoreall ]; then
mv $HOME/Trash/* /$PWD
else
if [ $1 = -empty ]; then
rm -rfv /$HOME/Trash/*
else
mv $PWD/"$1"/$HOME/Trash
echo -n "$PWD" >> /$HOME/Bash/try
fi
fi
fi
You could append the timestamp of the time of deletion to the filename in your Trash folder. Upon restore, you could strip this off again.
To add a timestamp to your file, use something like this:
DT=$(date +'%Y%m%d-%H%M%S')
mv $PWD/"$1" "/$HOME/Trash/${1}.${DT}"
This will, e.g., create a file like initrd.img-2.6.28-11-generic.20110615-140159 when moving initrd.img-2.6.28-11-generic.
To get the original filename, strip everything starting from the last dot, like with:
NAME_WITHOUT_TIMESTAMP=${file%.*-*}
The pattern is on the right side after the percentage char. (.* would also be enough to match.)
Take a look how trash-cli does it. It's written in Python and uses the same trash bin as desktop environments. Trash-cli is available at least in the big Linux distributions.
http://code.google.com/p/trash-cli/
Probably the easiest thing to do is simply add -i to the invocation of mv. That will prompt the user whether or not to replace. If you happen to have access to gnu cp (eg, on Linux), you could use cp --backup instead of mv.