I use this bash command often
find ~ -type f -name \*.smt -exec grep something {} /dev/null \;
so I am trying to turn it into a simple bash script that I would invoke like this
findgrep ~ something --mtime -12 --name \*.smt
However I get stuck with processing the command line options with GNU getopt like this:
if ! options=$(getopt -o abc: -l name:,blong,mtime: -- "$#")
then
# something went wrong, getopt will put out an error message for us
exit 1
fi
set -- $options
while [ $# -gt 0 ]
do
case $1 in
-t|--mtime) mtime=${2} ; shift;;
-n|--name|--iname) name="$2" ; shift;;
(--) shift; break;;
(-*) echo "$0: error - unrecognized option $1" 1>&2; exit 1;;
(*) break;;
esac
shift
done
echo "done"
echo $#
if [ $# -eq 2 ]
then
echo "2 args"
dir="$1"
str="$1"
elif [ $# -eq 1 ]
then
dir="."
str="$1"
echo "1 arg"
else
echo "need a search string"
fi
echo $dir
echo $str
echo $mtime
echo "${mtime%\'}"
echo "${mtime%\'}"
echo '--------------------'
mtime="${mtime%\'}"
mtime="${mtime#\'}"
dir="${dir%\'}"
dir="${dir#\'}"
echo $dir $mtime $name
# grep part not in yet
find $dir -type f -mtime $mtime -name $name
which does not seem to work - I suspect because the $name variable gets passed in quotes to find.
How do I fix that?
set -- $options
Is invalid (and it's not quoted). It's eval "set -- $options". Linux getopt outputs properly quoted string to be eval-ed.
mtime="${mtime%\'}"
mtime="${mtime#\'}"
dir="${dir%\'}"
dir="${dir#\'}"
Remove it. That's not how expansions work.
-name $name
It's not quoted. You have to quote it upon use.
-name "$name"
Check your scripts with shellcheck.
Related
I use this bash command often
find ~ -type f -name \*.smt -exec grep something {} /dev/null \;
so I am trying to turn it into a simple bash script that I would invoke like this
findgrep ~ something --mtime -12 --name \*.smt
Thanks to this answer I managed to make it work like this:
if ! options=$(getopt -o abc: -l name:,blong,mtime: -- "$#")
then
exit 1
fi
eval "set -- $options"
while [ $# -gt 0 ]
do
case $1 in
-t|--mtime) mtime=${2} ; shift;;
-n|--name|--iname) name="$2" ; shift;;
(--) shift; break;;
(-*) echo "$0: error - unrecognized option $1" 1>&2; exit 1;;
(*) break;;
esac
shift
done
if [ $# -eq 2 ]
then
dir="$1"
str="$2"
elif [ $# -eq 1 ]
then
dir="."
str="$1"
else
echo "Need a search string"
exit
fi
echo "find $dir -type f -mtime $mtime -name $name -exec grep -iln \"$str\" {} /dev/null \;"
echo "find $dir -type f -mtime $mtime -name $name -exec grep -iln \"$str\" {} /dev/null \;" | bash
but the last line - echo'ing a command into bash - seems outright barbaric, but it works.
Is there a better way to do that? somehow trying to execute the find command directly gives no output, while running the one echo'ed out in bash works ok.
ame $name -e
It's still not quoted. Check your script with shellcheck.
find "$dir" -type f -mtype "$mtime" -name "$name" -exec grep -iln "$str" {} ';'
You might want to take a few steps back and do some research about quoting and expansions in shel, find and glob. find program expects literal glob pattern, and unquoted variable expansions undergo filename expansion, changing *.smt into the list of words representing filenames, while find wants the pattern not the result of expansions.
I can throw: man find, man 7 glob, https://www.gnu.org/software/bash/manual/html_node/Quoting.html https://mywiki.wooledge.org/BashFAQ/050
https://mywiki.wooledge.org/BashGuide/Parameters#Parameter_Expansion
Before you start deciding how to pass variable number of arguments to find, I encourage to research Bash arrays. I would do:
#!/bin/bash
fatal() {
echo "$0: ERROR: $*" >&2
exit 1
}
args=$(getopt -o abc: -l name:,iname:,mtime: -- "$#") || exit 1
eval "set -- $args"
findargs=() # bash array
while (($#)); do
case $1 in
-t|--mtime) findargs+=(-mtime "$2"); shift; ;;
-n|--name) findargs+=(-name "$2"); shift; ;;
--iname) findargs+=(-iname "$2"); shift; ;;
--) shift; break; ;;
-*) fatal "unrecognized option $1"; ;;
*) break; ;;
esac
shift
done
if (($# == 2)); then
dir="$1"
str="$2"
elif (($# == 1)); then
dir="."
str="$1"
else
fatal "Need a search string"
fi
set -x
find "$dir" -type f "${findargs[#]}" -exec grep -iln "$str" /dev/null {} +
I was fiddling around with bash last month and am trying to create a script.
I want the script to search through the folders for files with some kind of extension defined by the argument -e. The folders are defined without -option. The output is 2 columns where in the first it prints the found files, and in the second the respective folders.
Is this the most efficient and/or flexible way to go?
I also can't manage to let the -l command work. Any idea what's wrong? When I enter -name \${CHAR}*, it simply doesn't work. Also, how can I make it recognize a range being used? With an if-function looking for the "-" character or something?
I think I managed to mount a block device, but how can I add the path as a parameter so it can be used as a folder? Setting a number as a var doesn't work, it tells me it doesn't recognize the command.
For some reason the 'no recursion' tag works, but the 'no numbers' doesn't. I have no idea why this would be different.
When using the 'no recursion' (nn) and 'no numbers' (nr) tags I use a long tag --tag for the arguments. Is it possible to use only 1 -tag? This is possible with get opts, but then I can't manage to use the other tags after the get opts has been used. Someone a solution?
Finally, is it possible, when finding 2 files with the same file name, instead of printing the file twice, can it just show the file once. But for every file with the same name keep a white space, so it can still show all the folders in the second column?
#!/bin/bash
#FUNCTIONS
#Error
#Also written to stderr
err() {
echo 1>&2;
echo "Error, not enough arguments" 1>&2;
echo "Usage: $0 [-e <file extension>] [<folder>]";
echo "Please enter the argument -e and at least 1 folder.";
echo "More: Please chek Help by using -h or --help.";
echo 1>&2;
exit
}
#Help
help() {
echo
echo "--- Help ---"
echo
echo "This script will look for file extentions in 1 or more directories. The output shows the found files with the according folder where it's located."
echo
echo "Argument -e <ext> is required."
echo "Other arguments the to-look-trough folders."
echo
echo "These are also usable options:"
echo "-h or --help shows this."
echo "-l <character> looks for files starting with the character."
echo "-l <character1>-<character2> does the same, but looks trough a range of characters."
echo "-b <block-device> mounts a partition to /mnt and let it search through."
echo "--nn (no numbers) makes sure there are no numbers in the file name."
echo "--nr (no recursion) doesn't look trough subdirectories."
echo "-r of –-err <file> writes the errors (f.e. corrupted directory) to <file>."
echo "-s <word> searches the word through the files and only shows the files having that word."
echo
exit
}
#VARS
#execute getopt
OPTS=$(getopt -o e:hl:b:r:s: -l "help,nn,nr,err" -n "FileExtensionScript" -- "$#");
#Bad arguments
if [ $? -ne 0 ];
then
err;
exit
fi
#Rearrange arguments
eval set -- "$OPTS";
#echo "AFTER SET -- \$OPTS: $#";
while true; do
case "$1" in
-e)
shift;
if [ -n "$1" ]; then
EXT=$1;
shift;
fi
;;
-h|--help)
shift;
help;
;;
-l)
shift;
if [ -n "$1" ]; then
CHAR=$1;
shift;
fi
;;
-b)
shift;
if [ -n "$1" ]; then
sudo mkdir /mnt/$1;
sudo echo -e "/dev/$1 /mnt/$1 vfat defaults 0 0 " >> /etc/fstab;
sudo mount -a;
999=/mnt/$1;
shift;
fi
;;
--nn)
shift;
NONUM=" ! -name '*[0-9]*'";
;;
--nr)
shift;
NOREC="-maxdepth 1";
;;
-f|--err)
shift;
if [ -n "$1" ]; then
ERROR="| 2>filename | tee -a $1";
shift;
fi
;;
-s)
shift;
if [ -n "$1" ]; then
SEARCH="-name '*$1*'";
shift;
fi
;;
--)
shift;
break;
;;
esac
done
#No folder or arguments given
if [ $# -lt 1 ];
then
err;
exit
fi
#Debug
echo "Folder argumenten: $#" >2;
echo \# $# >2;
#Create arrays with found files and according folders
FILES=( $(find $# $NOREC $SEARCH $NONUM -name \*.${EXT} $ERROR | rev | cut -d/ -f 1 | rev) )
FOLDERS=( $(find $# $NOREC $SEARCH $NONUM -name \*.${EXT} $ERROR | rev | cut -d/ -f 1 --complement | rev) )
#Show arrays in 2 columns
for ((i = 0; i <= ${#FILES[#]}; i++));
do
printf '%s %s\n' "${FILES[i]}" "${FOLDERS[i]}"
done | column -t | sort -k1 #Make columns cleaner + sort on filename
I am not native English speaker and am hoping to get some tips to finish my script :) Thanks in advance!
After several days of research, I still can't figure out the best method for parsing cmdline args in a .sh script. According to my references the getopts cmd is the way to go since it "extracts and checks switches without disturbing the positional parameter variables.Unexpected switches, or switches that are missing arguments, are recognized and reportedas errors."
Positional params(Ex. 2 - $#, $#, etc) apparently don't work well when spaces are involved but can recognize regular and long parameters(-p and --longparam). I noticed that both methods fail when passing parameters with nested quotes ("this is an Ex. of ""quotes""."). Which one of these three code samples best illustrates the way to deal with cmdline args? The getopt function is not recommended by gurus, so I'm trying to avoid it!
Example 1:
#!/bin/bash
for i in "$#"
do
case $i in
-p=*|--prefix=*)
PREFIX=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
;;
-s=*|--searchpath=*)
SEARCHPATH=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
;;
-l=*|--lib=*)
DIR=`echo $i | sed 's/[-a-zA-Z0-9]*=//'`
;;
--default)
DEFAULT=YES
;;
*)
# unknown option
;;
esac
done
exit 0
Example 2:
#!/bin/bash
echo ‘number of arguments’
echo "\$#: $#"
echo ”
echo ‘using $num’
echo "\$0: $0"
if [ $# -ge 1 ];then echo "\$1: $1"; fi
if [ $# -ge 2 ];then echo "\$2: $2"; fi
if [ $# -ge 3 ];then echo "\$3: $3"; fi
if [ $# -ge 4 ];then echo "\$4: $4"; fi
if [ $# -ge 5 ];then echo "\$5: $5"; fi
echo ”
echo ‘using $#’
let i=1
for x in $#; do
echo "$i: $x"
let i=$i+1
done
echo ”
echo ‘using $*’
let i=1
for x in $*; do
echo "$i: $x"
let i=$i+1
done
echo ”
let i=1
echo ‘using shift’
while [ $# -gt 0 ]
do
echo "$i: $1"
let i=$i+1
shift
done
[/bash]
output:
bash> commandLineArguments.bash
number of arguments
$#: 0
using $num
$0: ./commandLineArguments.bash
using $#
using $*
using shift
#bash> commandLineArguments.bash "abc def" g h i j*
Example 3:
#!/bin/bash
while getopts ":a:" opt; do
case $opt in
a)
echo "-a was triggered, Parameter: $OPTARG" >&2
;;
\?)
echo "Invalid option: -$OPTARG" >&2
exit 1
;;
:)
echo "Option -$OPTARG requires an argument." >&2
exit 1
;;
esac
done
exit 0
I find the use of getopt to be the easiest. It provides correct handling of arguments which is tricky otherwise. For example, getopt will know how to handle arguments to a long option specified on the command line as --arg=option or --arg option.
What is useful in parsing any input passed to a shell script is the use of the "$#" variables. See the bash man page for how this differs from $#. It ensures that you can process arguments that include spaces.
Here's an example of how I might write s script to parse some simple command line arguments:
#!/bin/bash
args=$(getopt -l "searchpath:" -o "s:h" -- "$#")
eval set -- "$args"
while [ $# -ge 1 ]; do
case "$1" in
--)
# No more options left.
shift
break
;;
-s|--searchpath)
searchpath="$2"
shift
;;
-h)
echo "Display some help"
exit 0
;;
esac
shift
done
echo "searchpath: $searchpath"
echo "remaining args: $*"
And used like this to show that spaces and quotes are preserved:
user#machine:~/bin$ ./getopt_test --searchpath "File with spaces and \"quotes\"."
searchpath: File with spaces and "quotes".
remaining args: other args
Some basic information about the use of getopt can be found here
If you want to avoid using getopt you can use this nice quick approach:
Defining help with all options as ## comments (customise as you wish).
Define for each option a function with same name.
Copy the last five lines of this script to your script (the magic).
Example script: log.sh
#!/bin/sh
## $PROG 1.0 - Print logs [2017-10-01]
## Compatible with bash and dash/POSIX
##
## Usage: $PROG [OPTION...] [COMMAND]...
## Options:
## -i, --log-info Set log level to info (default)
## -q, --log-quiet Set log level to quiet
## -l, --log MESSAGE Log a message
## Commands:
## -h, --help Displays this help and exists
## -v, --version Displays output version and exists
## Examples:
## $PROG -i myscrip-simple.sh > myscript-full.sh
## $PROG -r myscrip-full.sh > myscript-simple.sh
PROG=${0##*/}
LOG=info
die() { echo $# >&2; exit 2; }
log_info() {
LOG=info
}
log_quiet() {
LOG=quiet
}
log() {
[ $LOG = info ] && echo "$1"; return 1 ## number of args used
}
help() {
grep "^##" "$0" | sed -e "s/^...//" -e "s/\$PROG/$PROG/g"; exit 0
}
version() {
help | head -1
}
[ $# = 0 ] && help
while [ $# -gt 0 ]; do
CMD=$(grep -m 1 -Po "^## *$1, --\K[^= ]*|^##.* --\K${1#--}(?:[= ])" log.sh | sed -e "s/-/_/g")
if [ -z "$CMD" ]; then echo "ERROR: Command '$1' not supported"; exit 1; fi
shift; eval "$CMD" $# || shift $? 2> /dev/null
done
Testing
Running this command:
./log.sh --log yep --log-quiet -l nop -i -l yes
Produces this output:
yep
yes
By the way: It's compatible with posix!
I saw a question on stackflow about parsing arguments. I tried to write this, but it's not working and now it's getting on my nerves.
The usual way of running a script on the terminal is ./scriptname, but I later introduced the argument -d. So, if I put ./scriptname it will not run. If I put ./scriptname -d it will.
Now I want to put another argument for the path (where the files are moving, in this case "/home/elg19/documents") such that when I do not include the path, it won't run. But, if I put ./scriptname -d path I want to replace $To in the existing script with the command argument after -d.
#!/bin/bash
From="/home/mark/doc"
To=$2
if [ $1 = -d ]; then
cd "$From"
for i in pdf txt doc; do
find . -type f -name "*.${i}" -exec mv "{}" "$To" \;
done
fi
Your desired usage isn't completely clear, but it seems to be:
scriptname -d path
So, you can do it the extensible way, or the brute force way. Since you're changing directories willy-nilly, you also need to ensure that the paths are absolute, not relative.
Brute force
#!/bin/bash
From="/home/mark/doc"
if [ $# = 2 ] && [ "$1" = '-d' ] && [ -d $2 ]
then
case "$2" in
(/*) cd "$From" &&
for extn in pdf txt doc
do find . -type f -name "*.$extn" -exec mv {} "$To" \;
done;;
(*) echo "$0: path name must be absolute ($2 is not)" 1>&2; exit 1;;
esac
else
echo "Usage: $0 -d /absolute/dirname" 1>&2; exit 1
fi
Extensible
#!/bin/bash
From="/home/mark/doc"
To=""
usage()
{
echo "Usage: $(basename $0 .sh) -d /absolute/dirname" 1>&2
exit 1
}
while getopts d: opt
do
case "$opt" in
(d) if [ ! -d "$OPTARG" ]
then echo "$0: $OPTARG is not a directory" 1>&2; exit 1
else
case "$OPTARG" in
(/*) To="$OPTARG";;
(*) echo "$0: path name must be absolute ($2 is not)" 1>&2; exit 1;;
esac
fi;;
(*) usage;;
esac
done
shift $(($OPTIND - 1))
if [ $# != 0 ] || [ -z "$To" ]
then usage
fi
cd "$From" &&
for extn in pdf txt doc
do find . -type f -name "*.$extn" -exec mv {} "$To" \;
done
For example, it will be very easy to add a -f from option to deal with changing the source of the files.
Note that you could also use:
for extn in pdf txt doc
do find "$From" -type f -name "*.$extn" -exec mv {} "$To" \;
done
This would allow you to permit relative names for the 'from' and 'to' directories because it does not change directory.
I assume you want to do some input validation to your command line arguments. I guess the following would be somewhat useful:
#!/bin/bash
usage() {
echo "USAGE :"
echo "./move -d <to-directory>"
}
if [ $# -ne 2 ] ; then
usage
exit
fi
case $1 in
-d ) shift
To=$1
;;
* ) usage
exit
esac
From="/tmp/From/"
cd "$From"
for i in pdf txt doc; do
find . -type f -name "*.${i}" -exec mv "{}" "$To" \;
done
Moreover to debug your script, you may use the following command:
bash -x ./move.sh -d /tmp/To/
You may add more error checking (and informative echo's) for the following cases:
Source/destination directory does not exits
N files have been copied from the to
No files available at
You can take the type of files as arguments f.e. -t doc xls pdf
EDITED: this is more or less what I came up after #Mechanical's nice input. Any insight?
#!/bin/bash
path1="$(readlink -e "$1")"
path2="$(readlink -e "$2")"
EBADARGS=65
function checkArgsNumber()
{
if test "$#" -ne 2; then
echo "ERRORE: this script takes exactly 2 params."
exit $EBADARGS
fi
}
function checkExistence()
{
if [ ! -d $path1 ]; then
echo "ERROR: "$1" does not exist"
exit $EBADARGS
elif [ ! -d "$2" ]; then
echo "ERROR: "$2" does not exist"
exit $EBADARGS
elif [[ -L $path1 ]]; then
echo "ERROR: path1 can't be a symbolic link"
exit $EBADARGS
elif [[ -L $2 ]]; then
echo "ERROR: path2 can't be a symbolic link"
exit $EBADARGS
fi
}
function checkIfSame()
{
if [[ $path1 == $path2 ]]; then
echo "ERROR: path1 and path2 must be different directories"
exit $EBADARGS
fi
}
function checkIfSubdirectories()
{
if [[ $path1 = *$path2* ]]; then
echo "ERROR:"$1" is a $path2 subdirectory"
exit $EBADARGS
elif [[ $path2 = *$path1* ]]; then
echo "ERROR:"$2" is a $path1 subdirectory"
exit $EBADARGS
elif [[ -e "$(find $path1 -samefile $path2)" ]]; then
echo "ERROR:"$(find $path1 -samefile $path2 -print0)" and "$2" have the same inode, $path2 is a $path1 subdirectory"
exit $EBADARGS
elif [[ -e "$(find $path2 -samefile $path1)" ]]; then
echo "ERROR:"$(find $path2 -samefile $path1 -print0)" and "$2" have the same inode, $path1 is a $path2 subdirectory"
exit $EBADARGS
fi
}
checkArgsNumber "$#"
checkExistence "$#"
checkIfSame "$#"
checkIfSubdirectories "$#"
now.. this should work and I hope it is useful somehow.
Could someone explain me how the *$path2* part works? What is the name of this * * operator? Where should I go read about it?
Some problems:
Stylistic
You should probably quote the entire argument to echo, as
echo "ERROR: $1 is a subdirectory of $(readlink -e "$2")"
Without the quotes around the argument to echo, you are technically passing each word as its own parameter: echo "ERROR:somedir" "is" "a" "subdirectory".... Since echo prints its parameters in the order given, separated by spaces, the output is the same in your case. But semantically it's not what you want.
(An example where it would be different:
echo foo bar
would print foo bar.)
Error message doesn't work properly
If the arguments don't exist
$ ./check.sh nonexistent1 nonexistent2
ERROR:nonexistent1 is a subdirectory of
Obviously, this is irrelevant if you've already checked they exist.
You similarly need to check for corner cases such as where the parameters refer to the same directory:
$ mkdir a b
$ ln -s ../a b/c
$ ./check.sh a b/c
ERROR:a is a subdirectory of /dev/shm/a
Doesn't detect symbolic links
$ mkdir a b
$ ln -s ../a b/c
$ ./check.sh a b
gives no error message.
Doesn't detect mount --bind
$ mkdir a b b/c
$ sudo mount --bind a b/c
$ ./check.sh a b
gives no error message.