Simple BASH script needed: moving and renaming files - bash

Decades ago I was a programmer (IBM assembly, Fortran, COBOL, MS DOS scripting, a bit of Visual Basic.) Thus I'm familiar with the generalities of IF-Then-Else, For loops, etc.
However, I'm now needing to delve into Bash for my current job, and I'm having a difficult time with syntax and appropriate commands for what I need.
I'm in need of a trivial (concept-wise) script, which will:
Determine if a specific folder (e.g., ~/Desktop/Archive Folder) exists on the user Desktop
If not, create it ("Archive")
Move all files/folders on desktop - except for ~/Desktop/Archive, into "Archive Folder" - AND appending a timestamp onto the end of the filenames being moved.
It is this very last piece - the timestamp addition - which is holding me up.
I'm hoping a clear and simple solution can be sent my way. Here is what I've come up with so far:
#!/bin/bash
shopt -s extglob
FOLDERARCH="Archive Folder"
cd ~/Desktop
if [ ! -d $"FOLDERARCH" ]; then
mkdir "$FOLDERARCH"
echo "$FOLDERARCH did not exist, was created"
fi
mv !(-d "$FOLDERARCH") "$FOLDERARCH"
One final note: the script above works (without the timestamp piece) yet also ends with the message
mv: rename Archive Folder to Folder/Archive Folder: Invalid argument
Why?
Any help will be deeply, deeply appreciated. Please assume I know essentially zilch about the BASH environment, cmds and their arguments - this first request for assistance marks my first step into the journey of becoming at least proficient.
Update
First: much gratitude for the replies I've gotten; they've been very useful.
I've now got was it essentially a working version, but with some oddities I do not understand and, after hours of attempted research, have yet to understand/solve.
I'm hoping for some insight; I feel I'm on the verge of making some real headway in comprehending, but these anomalies are hindering my progress. Here's my (working, with "issues") code so far:
shopt -s extglob
FOLDERARCH="Archives"
NEWARCH=$(date +%F_%T)
cd ~/Desktop
if [ ! -d $"FOLDERARCH" ]; then
mkdir "$FOLDERARCH"
echo "$FOLDERARCH did not exist, was created"
fi
mkdir "$FOLDERARCH/$NEWARCH"
mv !(-d "$FOLDERARCH") $FOLDERARCH/$NEWARCH
This in fact largely accomplishes my goal, but:
In the case where the desktop Archives folder already exists, I'm expecting the if-then construct to simply follow through (with no echo msg) to the following mkdir command, but instead the msg "Archives not exist, was created" msg is output anyway (erroneously). Any answers as to why?
The script completes with the following msg:
mv: rename Archives to Archives/2016-01-10_00:06:54/Archives: Invalid argument
I don't understand this at all; what should be happening is that all files/folders on the desktop EXCEPT the /Desktop/Archives folder should be moved into a newly created "subfolder" of /Desktop/Archives, e.g., /Desktop/Archives/2016-01-10_00:06:54. In fact, the move accomplishes my goal, but that the message arises makes no sense to me. What is the invalid argument?
One last note: at this point in my newbie-status I'm looking for code which is clear and easy to read, versus much more elegant/sophisticated one-line piped-command solutions. I look forward to working my way up to those in due time.

You have several options. One of the simplest is to loop over the directories below ~/Desktop and if they are not "$FOLDERARCH", move them to "$FOLDERARCH", e.g.:
for i in */; do
[ "$i" != "$FOLDERARCH"/ ] && mv "$i" "$FOLDERARCH"
done
I haven't run a test case, but something similar to the following should work.
#!/bin/bash
shopt -s extglob
FOLDERARCH="Archive Folder"
cd ~/Desktop || { printf "failed to change to '~/Destop'\n"; exit 1; }
if [ ! -d "$FOLDERARCH" ]; then
if mkdir "$FOLDERARCH" , then
echo "$FOLDERARCH did not exist, was created"
else
echo "error: failed to create '$FOLDERARCH'"
exit 1
fi
fi
for i in */; do
[ "$i" != "$FOLDERARCH"/ ] && mv "$i" "$FOLDERARCH"
done
I apologize, I forgot the datestamp portion. As pointed out in the comments, you can include the datestamp (set the format to your taste) with something similar to the following:
tstamp=$(date +%s)
for i in */; do
[ "$i" != "$FOLDERARCH"/ ] && mv "$i" "$FOLDERARCH/${i}_${tstamp}"
done

Related

Bash diff that stops when it finds the first difference

I have this script that I use for backups. The problem is that it is kind of slow. I want to know if there is a diff command that stops when finds the first difference.
DocumentsFiles=("Books" "Comics" "Distros" "Emulators" "Facturas" "Facultad" "Laboral" "Mods" "Music" "Paintings" "Projects" "Scripts" "Tesis" "Torrents" "Utilities")
OriginDocumentsFile="E:\Documents\\"
DestinationDocumentsFile="F:\Files\Documents\\"
## loop file to file and copy in backup
for directory in "${DocumentsFiles[#]}"
do
RealOrigin="${OriginDocumentsFile}${directory}"
RealDestination="${DestinationDocumentsFile}${directory}"
echo $directory
if [ -a "$RealDestination" ]; then
echo ok
if diff -r $RealOrigin $RealDestination; then
echo "${directory} are equal!"
else
rm -rfv $RealDestination
cp -ruv $RealOrigin "${DestinationDocumentsFile}"
fi
else
cp -ruv $RealOrigin "${DestinationDocumentsFile}"
fi
done
diff -q reports "only when files differ" (per man diff), so I believe it'll stop after the first difference.
But this is a bit of an XY problem. Really you need a better backup program like rsync:
It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination.
From man rsync

searching file existence at any depth in bash

I need to check if a file exists in directory using bash. I have tried below method but it needs complete path as input.
if [ -e /*/my_file.txt ] ;
then
echo "file found"
else
echo "not found"
fi
Is there any way that I can check if the file exists at any depth dynamically.
NOTE: I don't want to use "find" as it takes lot of time to execute.
If you are using bash 4, you can write patterns that recursively descend a hierarchy:
shopt -s globstar
for f in /**/myfile.txt; do
if [[ -e $f ]]; then
found=1
echo "File found"
break
fi
done
if [[ $found -ne 1 ]]; then
echo "File not found"
fi
Using find:
found=$( find / -name myfile.txt )
if [[ -n $found ]]; then
echo "File found"
else
echo "File not found"
fi
If really speed is your concern, file globbing like ls * */* */*/* is not helping you that much. And it has its limit with this error: Argument list too long. find is a useful tool for finding stuff. It is very flexible. But like using file globbing, it has to scan the directory tree with every invocation. For occasional searches, like for maintenance, this is totally acceptable. But if this is part processing pipeline, the speed is not acceptable. You need an optimised database for that.
The simplistic way
Most every UNIX I know is shipped with locate.
If it is preinstall you can search like this:
$ locate -b '\my_file.txt'
The backslash in front of my_file.txt is intended. It switches off wildcard search. Adding -i gives case insensitive search.
If the command is not available it should be installable from your OS repository. For Debian/Ubuntu: apt install locate. For first init run /etc/cron.daily/locate as root or with sudo.
The database is updated on a daily basis. For some applications, this interval is probably too long. By moving the cronjob from daily to like every 3 hours, you get more recent results.
The realtime way ...
This is a bit out of the scope of this answer. But you would need some kind of deamon, that would watch kernel inotify events for directory changes. These in turn would be reflected in a database, that can be queried through some API. Like Spotlight from MacOS or Tracker from Gnome.
find is the proper solution.
however you can use bash expansion feature
if ls */* | grep -q my_file.txt
then echo file found
else echo file not found
fi
note
that above solution will not find my_file.txt if a top level.
if my_file.txt is part of a directory name you might get a wrong result.
if there are many (thousands) directories and many files / expansion might get paste bash limit (arg list too long)
you can ls * */* */*/* | grep with limit state above.

writing a shell script if statement to check for directory

I need to write a script that will recreate my opt folder if it gets deleted when I remove a package from it. Here's a link to my previous post: dpkg remove to stop processes
Now, the issue I'm running into could be better described here: http://lists.debian.org/debian-devel/2006/03/msg00242.html
I was thinking of just adding a postrem script which checks if an opt directory exists, and if not, creates one. My experience with shell scripts is pretty limited though..
[ -d "$dir" ] || mkdir -p "$dir"
This could be written more verbosely / clearly as:
if ! test -d "$dir"; then
mkdir -p "$dir"
fi
See help test for more information.

Is this a valid self-update approach for a bash script?

I'm working on a script that has gotten so complex I want to include an easy option to update it to the most recent version. This is my approach:
set -o errexit
SELF=$(basename $0)
UPDATE_BASE=http://something
runSelfUpdate() {
echo "Performing self-update..."
# Download new version
wget --quiet --output-document=$0.tmp $UPDATE_BASE/$SELF
# Copy over modes from old version
OCTAL_MODE=$(stat -c '%a' $0)
chmod $OCTAL_MODE $0.tmp
# Overwrite old file with new
mv $0.tmp $0
exit 0
}
The script seems to work as intended, but I'm wondering if there might be caveats with this kind of approach. I just have a hard time believing that a script can overwrite itself without any repercussions.
To be more clear, I'm wondering, if, maybe, bash would read and execute the script line-by-line and after the mv, the exit 0 could be something else from the new script. I think I remember Windows behaving like that with .bat files.
Update: My original snippet did not include set -o errexit. To my understanding, that should keep me safe from issues caused by wget.
Also, in this case, UPDATE_BASE points to a location under version control (to ease concerns).
Result: Based on the input from these answers, I constructed this revised approach:
runSelfUpdate() {
echo "Performing self-update..."
# Download new version
echo -n "Downloading latest version..."
if ! wget --quiet --output-document="$0.tmp" $UPDATE_BASE/$SELF ; then
echo "Failed: Error while trying to wget new version!"
echo "File requested: $UPDATE_BASE/$SELF"
exit 1
fi
echo "Done."
# Copy over modes from old version
OCTAL_MODE=$(stat -c '%a' $SELF)
if ! chmod $OCTAL_MODE "$0.tmp" ; then
echo "Failed: Error while trying to set mode on $0.tmp."
exit 1
fi
# Spawn update script
cat > updateScript.sh << EOF
#!/bin/bash
# Overwrite old file with new
if mv "$0.tmp" "$0"; then
echo "Done. Update complete."
rm \$0
else
echo "Failed!"
fi
EOF
echo -n "Inserting update process..."
exec /bin/bash updateScript.sh
}
(At least it doesn't try to continue running after updating itself!)
The thing that makes me nervous about your approach is that you're overwriting the current script (mv $0.tmp $0) as it's running. There are a number of reasons why this will probably work, but I wouldn't bet large amounts that it's guaranteed to work in all circumstances. I don't know of anything in POSIX or any other standard that specifies how the shell processes a file that it's executing as a script.
Here's what's probably going to happen:
You execute the script. The kernel sees the #!/bin/sh line (you didn't show it, but I presume it's there) and invokes /bin/sh with the name of your script as an argument. The shell then uses fopen(), or perhaps open() to open your script, reads from it, and starts interpreting its contents as shell commands.
For a sufficiently small script, the shell probably just reads the whole thing into memory, either explicitly or as part of the buffering done by normal file I/O. For a larger script, it might read it in chunks as it's executing. But either way, it probably only opens the file once, and keeps it open as long as it's executing.
If you remove or rename a file, the actual file is not necessarily immediately erased from disk. If there's another hard link to it, or if some process has it open, the file continues to exist, even though it may no longer be possible for another process to open it under the same name, or at all. The file is not physically deleted until the last link (directory entry) that refers to it has been removed, and no processes have it open. (Even then, its contents won't immediately be erased, but that's going beyond what's relevant here.)
And furthermore, the mv command that clobbers the script file is immediately followed by exit 0.
BUT it's at least conceivable that the shell could close the file and then re-open it by name. I can't think of any good reason for it to do so, but I know of no absolute guarantee that it won't.
And some systems tend to do stricter file locking that most Unix systems do. On Windows, for example, I suspect that the mv command would fail because a process (the shell) has the file open. Your script might fail on Cygwin. (I haven't tried it.)
So what makes me nervous is not so much the small possibility that it could fail, but the long and tenuous line of reasoning that seems to demonstrate that it will probably succeed, and the very real possibility that there's something else I haven't thought of.
My suggestion: write a second script whose one and only job is to update the first. Put the runSelfUpdate() function, or equivalent code, into that script. In your original script, use exec to invoke the update script, so that the original script is no longer running when you update it. If you want to avoid the hassle of maintaining, distributing, and installing two separate scripts. you could have the original script create the update script with a unique in /tmp; that would also solve the problem of updating the update script. (I wouldn't worry about cleaning up the autogenerated update script in /tmp; that would just reopen the same can of worms.)
Yes, but ... I would recommend you keep a more layered version of your script's history, unless the remote host can also perform version-control with histories. That being said, to respond directly to the code you have posted, see the following comments ;-)
What happens to your system when wget has a hiccup, quietly overwrites part of your working script with only a partial or otherwise corrupt copy? Your next step does a mv $0.tmp $0 so you've lost your working version. (I hope you have it in version control on the remote!)
You can check to see if wget returns any error messages
if ! wget --quiet --output-document=$0.tmp $UPDATE_BASE/$SELF ; then
echo "error on wget on $UPDATE_BASE/$SELF"
exit 1
fi
Also, Rule-of-thumb tests will help, i.e.
if (( $(wc -c < $0.tmp) >= $(wc -c < $0) )); then
mv $0.tmp $0
fi
but are hardly foolproof.
If your $0 could windup with spaces in it, better to surround all references like "$0".
To be super-bullet proof, consider checking all command returns AND that Octal_Mode has a reasonable value
OCTAL_MODE=$(stat -c '%a' $0)
case ${OCTAL_MODE:--1} in
-[1] )
printf "Error : OCTAL_MODE was empty\n"
exit 1
;;
777|775|755 ) : nothing ;;
* )
printf "Error in OCTAL_MODEs, found value=${OCTAL_MODE}\n"
exit 1
;;
esac
if ! chmod $OCTAL_MODE $0.tmp ; then
echo "error on chmod $OCTAL_MODE %0.tmp from $UPDATE_BASE/$SELF, can't continue"
exit 1
fi
I hope this helps.
Very late answer here, but as I just solved this too, I thought it might help someone to post the approach:
#!/usr/bin/env bash
#
set -fb
readonly THISDIR=$(cd "$(dirname "$0")" ; pwd)
readonly MY_NAME=$(basename "$0")
readonly FILE_TO_FETCH_URL="https://your_url_to_downloadable_file_here"
readonly EXISTING_SHELL_SCRIPT="${THISDIR}/somescript.sh"
readonly EXECUTABLE_SHELL_SCRIPT="${THISDIR}/.somescript.sh"
function get_remote_file() {
readonly REQUEST_URL=$1
readonly OUTPUT_FILENAME=$2
readonly TEMP_FILE="${THISDIR}/tmp.file"
if [ -n "$(which wget)" ]; then
$(wget -O "${TEMP_FILE}" "$REQUEST_URL" 2>&1)
if [[ $? -eq 0 ]]; then
mv "${TEMP_FILE}" "${OUTPUT_FILENAME}"
chmod 755 "${OUTPUT_FILENAME}"
else
return 1
fi
fi
}
function clean_up() {
# clean up code (if required) that has to execute every time here
}
function self_clean_up() {
rm -f "${EXECUTABLE_SHELL_SCRIPT}"
}
function update_self_and_invoke() {
get_remote_file "${FILE_TO_FETCH_URL}" "${EXECUTABLE_SHELL_SCRIPT}"
if [ $? -ne 0 ]; then
cp "${EXISTING_SHELL_SCRIPT}" "${EXECUTABLE_SHELL_SCRIPT}"
fi
exec "${EXECUTABLE_SHELL_SCRIPT}" "$#"
}
function main() {
cp "${EXECUTABLE_SHELL_SCRIPT}" "${EXISTING_SHELL_SCRIPT}"
# your code here
}
if [[ $MY_NAME = \.* ]]; then
# invoke real main program
trap "clean_up; self_clean_up" EXIT
main "$#"
else
# update myself and invoke updated version
trap clean_up EXIT
update_self_and_invoke "$#"
fi

Recycle bin in bash problem

I need to make a recycle bin code using bash. Here is what I have done so far. My problem is that when I move a file with the same name into the trash folder it just overwrites the previous file. Can you give me any suggestions on how to approach this problem?
#!/bin/bash
mkdir -p "$HOME/Trash"
if [ $1 = -restore ]; then
while read file; do
mv $HOME/Trash/$2 /$file
done < try.txt
else
if [ $1 = -restoreall ]; then
mv $HOME/Trash/* /$PWD
else
if [ $1 = -empty ]; then
rm -rfv /$HOME/Trash/*
else
mv $PWD/"$1"/$HOME/Trash
echo -n "$PWD" >> /$HOME/Bash/try
fi
fi
fi
You could append the timestamp of the time of deletion to the filename in your Trash folder. Upon restore, you could strip this off again.
To add a timestamp to your file, use something like this:
DT=$(date +'%Y%m%d-%H%M%S')
mv $PWD/"$1" "/$HOME/Trash/${1}.${DT}"
This will, e.g., create a file like initrd.img-2.6.28-11-generic.20110615-140159 when moving initrd.img-2.6.28-11-generic.
To get the original filename, strip everything starting from the last dot, like with:
NAME_WITHOUT_TIMESTAMP=${file%.*-*}
The pattern is on the right side after the percentage char. (.* would also be enough to match.)
Take a look how trash-cli does it. It's written in Python and uses the same trash bin as desktop environments. Trash-cli is available at least in the big Linux distributions.
http://code.google.com/p/trash-cli/
Probably the easiest thing to do is simply add -i to the invocation of mv. That will prompt the user whether or not to replace. If you happen to have access to gnu cp (eg, on Linux), you could use cp --backup instead of mv.

Resources