How to Use Bash getopts Command to Get All Arguments Together - bash

I have couple of files needed to be parsed here, called parse.sh. I have to enable optional arguments for it with -l for line, -f for field. So to run the program would be ./parse.sh -l 5 -f 14 foo. Without -l or -f arguments, I want the program to default parse all lines and all fields. If -l is specified, I want it to parse just that line of foo, and further if -f is specified too, I want it to parse just that field. I see getopts usually work like this:
while getopts "l:f:" opts; do
case $opts in
l) #code to parse that line;;
f) #code to parse that field;;
case
done
BUT this is not what I need, because I want -l and -f work together sometimes. I am thinking maybe I should do getopts to parse all options into array, then write code based on parsing that array? Any better choices?
This is my code:
while getopts "l:f:" opt;
do
options=${opt}${options}
case $opt in
l) lineNumber=$OPTARG ;;
f) fieldNumber=$OPTARG ;;
esac
done
case $options in
f) echo "Parse field $fieldNumber of each line" ;;
l) echo "Parse all fields of line number $lineNumber" ;;
lf | fl) echo "Parse field $fieldNumber of line $lineNumber" ;;
*) echo "Parse all fields of all lines" ;;
esac

#!/bin/bash
parse()
{
local lines=$1
local fields=$2
local file=$3
# logic goes here
echo "parsing line(s) ${lines} and field(s) ${fields} of file ${file}"
}
lines=all
fields=all
while getopts "l:f:" o; do
case $o in
l) lines=${OPTARG} ;;
f) fields=${OPTARG} ;;
esac
done
shift $((OPTIND-1))
for file; do
parse "${lines}" "${fields}" "${file}"
done
Example runs:
$ ./t.sh foo.txt bar.txt
parsing line(s) all and field(s) all of file foo.txt
parsing line(s) all and field(s) all of file bar.txt
$ ./t.sh -l 10 foo.txt bar.txt
parsing line(s) 10 and field(s) all of file foo.txt
parsing line(s) 10 and field(s) all of file bar.txt
$ ./t.sh -l 10 -f 5 foo.txt bar.txt
parsing line(s) 10 and field(s) 5 of file foo.txt
parsing line(s) 10 and field(s) 5 of file bar.txt
$ ./t.sh -f 5 foo.txt bar.txt
parsing line(s) all and field(s) 5 of file foo.txt
parsing line(s) all and field(s) 5 of file bar.txt

#!/bin/bash
while getopts "l:f:" opts; do
case $opts in
l)
lOn=1
;;
f)
fOn=1
;;
esac
done
if [[ -n $lOn && -n $fOn ]]; then
echo 'both l and f'
elif [[ -n $lOn ]]; then
echo 'just l'
elif [[ -n $fOn ]]; then
echo 'just f'
else
echo 'nothing'
fi
If statements give you more flexibility in situations when you want to check for other variables or do more complex stuff. This wont work in sh unless you change [[ ]] to [ ].

I made a concept script. Please try.
#!/bin/bash
PARSE_SPECIFIC_LINE=0
PARSE_SPECIFIC_FIELD=0
shopt -s extglob
while getopts "l:f:" opts; do
case $opts in
l)
if [[ $OPTARG != +([[:digit:]]) || OPTARG -lt 1 ]]; then
echo "Invalid argument to -l: $OPTARG"
exit 1
fi
PARSE_SPECIFIC_LINE=$OPTARG
;;
f)
if [[ $OPTARG != +([[:digit:]]) || OPTARG -lt 1 ]]; then
echo "Invalid argument to -f: $OPTARG"
exit 1
fi
PARSE_SPECIFIC_FIELD=$OPTARG
;;
esac
done
FILES=("${#:OPTIND}")
function parse_line {
local LINE=$1
if [[ -n $LINE ]]; then
if [[ PARSE_SPECIFIC_FIELD -gt 0 ]]; then
read -ra FIELDS <<< "$LINE"
echo "${FIELDS[PARSE_SPECIFIC_FIELD - 1]}"
else
echo "$LINE"
fi
fi
}
for F in "${FILES[#]}"; do
if [[ -e $F ]]; then
if [[ PARSE_SPECIFIC_LINE -gt 0 ]]; then
parse_line "$(sed -n "${PARSE_SPECIFIC_LINE}{p;q}" "$F")"
else
while read -r LINE; do
parse_line "$LINE"
done < "$F"
fi
else
echo "File does not exist: $F"
fi
done
I ran it with
bash script.sh -f 2 <(for i in {1..20}; do echo "$RANDOM $RANDOM $RANDOM $RANDOM $RANDOM"; done)
And I got
1031
1072
4350
12471
31129
32318
...
Adding -l 5 I got
11604

Related

Print bash script argument passed into a new script generated by your current script

Title might be confusing -- I'm writing up a bash script that creates another bash script in the same relative directory, and I want to print an argument passed into the second script.
Layout:
#!/bin/bash
set -e
while [[ $# -gt 1 ]]; do
key="$1"
if [[ ! "$2" =~ ^-[^-].* ]];
case $key in
--arg1 | -t)
arg1=$2
shift #shift past argument
;;
--arg2 | -k)
arg2=$2
shift #shift past argument
;;
--arg3 | -i)
arg3=$2
break
;;
*)
;;
esac
fi
done
cat <<'EOF' > secondScript.sh
#!/bin/bash
set -e
while [[ $# -gt 1 ]]; do
key="$1"
if [[ ! "$2" =~ ^-[^-].* ]]; then # don't gobble up next key if this key doesn't have a value
case $key in
--finalArg1 | -t)
finalArg1=$2
shift #shift past argument
;;
--finalArg2 | -k)
finalArg2=$2
shift #shift past argument
;;
--finalArg3 | -i)
finalArg3=$2
break
;;
*)
;;
esac
fi
done
echo ${finalArg1}
EOF
chmod +x secondScript.sh
./secondScript.sh --finalArg1 {arg1} --finalArg2 {arg2} --finalArg3 {arg3}
so the end result is arg1 printed to the console. Instead, this code just prints
{finalArg1}. Any way to do this?
Ah nvm this was just an oversight -- Forgot to add $ to substitute the variables on the second script call. Should look like this:
./secondScript.sh --finalArg1 ${arg1} --finalArg2 ${arg2} --finalArg3 ${arg3}

script to edit config files

I'm working on a script that would allow me to add, remove, or edit config files. I have tested it a little and it seems like I got it to work at least with a single file, but I would like to be able to just do .config or fi.config and have it perform the desired action.
I would appreciate any help.
Config file looks looks similar to this just bigger
-- Config File
-- Environment DEV7
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
-- General properties
-------------------------------------------------------------------------------
com.x.yy.zz.version=2.0.2
com.x.yy.zz.instanceRole.ServerA=PRIMARY
com.x.yy.zz.instanceRole.ServerB=SECONDARY
com.x.yy.zz.StopDelay=30
com.x.yy.zz.sourceType=t
com.x.yy.zz.sNumberInc=20000
com.x.yy.zz.sNumberMin=20000
com.x.yy.zz.sNumberMax=9980000
so -a would allow me to add a line after something
ex. -a StopDealy New
com.x.yy.zz.StopDelay=30
New
#!/bin/bash
i=1
usage()
{
echo "test usage"
}
if [[ $# -gt 4 ]]
then
i=2
fi
while [[ $# -gt $i ]]
do
key="$1"
case $key in
-f|--file)
file="$2"
shift
;;
-a|--after)
sed -i "/$2/a $3" $file
#shift # past argument
;;
-b|--before)
sed -i "/$2/i $3" $file
#shift
;;
-d|--delete)
sed -i "/$2/d" $file
#shift
;;
-e|--edit)
sed -ie "s/$2/$3/g" $file
shift
;;
*)
usage
;;
esac
shift # past argument or value
done
I didn't test it yet, but this is the closest version to what I understand you want to achieve.
#!/bin/bash
usage() {
echo "Usage: $0 -f file1 -f *.txt -[abde] ... -f file3 ... -[abde] ... "
}
# Do all the required action on one file
do_work() {
file="$1" # 1st argument must be the file to work on
shift
while [[ $# -gt 0 ]]; do
case "$1" in
-f|--file) while [[ ! "$2" = -* && $# -gt 0 ]]; do shift; done ;; # Ignore any "file" since we have one to work on.
-a|--after) sed -i "/$2/a $3" $file; shift 2 ;;
-b|--before) sed -i "/$2/i $3" $file; shift 2 ;;
-d|--delete) sed -i "/$2/d" $file; shift ;;
-e|--edit) sed -ie "s/$2/$3/g" $file; shift 2 ;;
esac
shift # past argument or value
done
}
# Check the arguments for files and print the valid ones
# Other arguments will just be skipped
# Invalid arguments will be displayed.
identify_files() {
while [[ $# -gt 0 ]]; do
case "$1" in
-f|--file) # Validate the the file exists (just in case)
# check each following argument until next option
# ... or end of arguments
while [[ ! "$2" = -* && $# -gt 0 ]]; do
if [[ -f "$2" ]]; then
echo "$2"
else
echo "Error: Invalid file '$2'." >&2
fi
shift
done ;;
-[abe]) shift 2 ;;
-d) shift ;;
-h) usage >&2; exit ;;
*) echo "Invalid otpion '$1'" >&2 ;;
esac
shift
done
}
# Do the required actions on all files received in the options
for File in $(identify_files "$#"); do
do_work "$File" "$#"
done
## Alternative on predefined files (not the ones received as arguments)
## Usage: $0 -[abde] ...
#for File in $(ls *.config); do
# do_work "$File" "$#"
#done

cut -d, ignore delimiter in argument

I have written a script that gets a variable number of arguments:
test.sh -i <input1> <input2> ... -o <output1> <output2> ...
I'm parsing the arguments as follows:
while [ $# -gt 1 ]; do
TMP=$(echo "$#" | cut -d '-' -f 2) #i <input1> <input2>
TMP1=$(echo "$TMP" | cut -d ' ' -f 1) #i
CNT=$(echo "$TMP" | wc -w) #3
set -x
case "$TMP1" in
i)
INPUTS=$(echo "$TMP" | cut -c 3-)
shift "$CNT"
;;
o)
OUTPUTS=$(echo "$TMP" | cut -c 3-)
shift "$CNT"
;;
esac
done
This works everytime, except for files that happen to have a '-' in their name.
Example:
./test.sh -i file1.txt file-2.txt -o out1.txt out-2.txt
Is there anyway I can force cut to ignore delimiters that occur within the file names?
You don't need all this string manipulation; each argument is already a separate word.
while (( $# > 0 )); do
case $1 in
-i) shift
while [[ $# -gt 0 && $1 != -* ]]; do
inputs+=( "$1" )
shift
done
;;
-o) shift
while [[ $# -gt 0 && $1 != -* ]]; do
outputs+=( "$1" )
shift
done
;;
*) echo "Unrecognized option $1"
exit 1
;;
esac
done
This can be refactored a little to avoid the repeated checks for running out of arguments.
for arg in "$#"; do
case $1 in
-i) mode=input; continue ;;
-o) mode=output; continue ;;
esac
case $mode in
input) input+=("$arg") ;;
output) output+=("$arg") ;;
*) echo "Unknown mode: $mode"
exit 1
;;
esac
done
Here's an alternative approach that may benefit someone.
The fact is, argument parsing is always a tradeoff, hence there's benefit in tailoring it to the application. Here's a pretty generic solution that allows a little bit of error checking and disorder in the arguments.
It's very simple, but I have added some example output and comments, and for the sake of readability and compatibility, stayed away from complex ways to save a line or two (especially on the if statements).
Sample Usage:
bash #> touch file-1 file3 file4 file-8 file7
bash #> argparse -i file-1 file3 file4 -c -k --q --j -r -t -o file-8 file7
Output:
Input files: file-1 file3 file4
Output files: file-8 file7
Args are: c k q j r t
Doing action for argument "c"
Doing action for argument "k"
Doing action for argument "j"
Script:
#!/bin/bash
#argparse
#Assign arrays
until [[ $# < 1 ]]; do
#ignore args "-i" and "-o", and tell the script to check for files following
if [ "$1" == "-i" ] ; then unset output ; input=1 ; shift
elif [ "$1" == "-o" ] ; then unset input ; output=1 ; shift
fi
#Add input and output files to respective arrays
if [ -f "$1" ] ; then
if [[ $input == 1 ]]; then
infiles+=($1)
elif [[ $output == 1 ]]; then
outfiles+=($1)
fi
else
#Add args to array
arg="$(echo "$1" | sed 's/-//g')"
args+=($arg)
fi
shift
done
#Some debug feedback
echo -e "Input files: ${infiles[#]}\nOutput files: ${outfiles[#]}\nArgs are: ${args[#]}\n"
#Simulate actually "doing" something with the args
for arg in "${args[#]}" ; do
case $arg in
"c") echo "Doing action for argument \"c\"" ;;
"k") echo "Doing action for argument \"k\"" ;;
"j") echo "Doing action for argument \"j\"" ;;
*) ;;
esac
done
Update/Edit: I've just realised, that the OP didn't have any requirement for parsing actual arguments other than -i and -o. Well regardless, this may still come in handy for someone at some point.

bash- reading file from stdin and arguments

So I have googled this and thought I found the answers, but it still doesnt work for me.
The program computes the average and median of rows and columns in a file of numbers...
Using the file name works:
./stats -columns test_file
Using cat does not work
cat test_file | ./stats -columns
I am not sure why it doesnt work
#file name was given
if [[ $# -eq 2 ]]
then
fileName=$2
#file name was not given
elif [[ $# -eq 1 ]]
then
#file name comes from the user
fileName=/dev/stdin
#incorrect number of arguments
else
echo "Usage: stats {-rows|-cols} [file]" 1>&2
exit 1
fi
A very simple program that accepts piped input:
#!/bin/sh
stdin(){
while IFS= read -r i
do printf "%s" "$i"
done
}
stdin
Test is as follows:
echo "This is piped output" | stdin
To put that into a script / utility similar to the one in the question you might do this:
#!/bin/sh
stdin(){
while IFS= read -r i
do printf "%s" "$i"
done
}
rowbool=0
colbool=0
for i in $#
do case "$i" in
-rows) echo "rows set"
rowbool=1
shift
;;
-cols) echo "cols set"
colbool=1
shift
;;
esac
done
if [[ $# -gt 0 ]]
then
fileName=$1
fi
if [[ $# -eq 0 ]]
then fileName=$(stdin)
fi
echo "$fileName"

Exclude string from wildcard in bash

I'm trying to adapt a bash script from "Sams' Teach Yourself Linux in 24 Hours" which is a safe delete command called rmv. The files are removed by calling rmv -d file1 file2 etc. In the original script a max of 4 files can by removed using the variables $1 $2 $3 $4.
I want to extend this to an unlimited number of files by using a wildcard.
So I do:
for i in $*
do
mv $i $HOME/.trash
done
The files are deleted okay but the option -d of the command rmv -d is also treated as an argument and bash objects that it cannot be found. Is there a better way to do this?
Thanks,
Peter
#!/bin/bash
# rmv - a safe delete program
# uses a trash directory under your home directory
mkdir $HOME/.trash 2>/dev/null
# four internal script variables are defined
cmdlnopts=false
delete=false
empty=false
list=false
# uses getopts command to look at command line for any options
while getopts "dehl" cmdlnopts; do
case "$cmdlnopts" in
d ) /bin/echo "deleting: \c" $2 $3 $4 $5 ; delete=true ;;
e ) /bin/echo "emptying the trash..." ; empty=true ;;
h ) /bin/echo "safe file delete v1.0"
/bin/echo "rmv -d[elete] -e[mpty] -h[elp] -l[ist] file1-4" ;;
l ) /bin/echo "your .trash directory contains:" ; list=true ;;
esac
done
if [ $delete = true ]
then
for i in $*
do
mv $i $HOME/.trash
done
/bin/echo "rmv finished."
fi
if [ $empty = true ]
then
/bin/echo "empty the trash? \c"
read answer
case "$answer" in
y) rm -i $HOME/.trash/* ;;
n) /bin/echo "trashcan delete aborted." ;;
esac
fi
if [ $list = true ]
then
ls -l $HOME/.trash
fi
You can make use of shift here.
Once you find -d is one of the options in the switch, you can shift and get rid of -d from the positional parameters. Next you can
until [ -z $1 ] ; do
mv $1 $HOME/.trash
shift
done
getopts sets OPTIND to the index of the first argument after the options. (#)
So after parsing the options you can do:
shift $OPTIND-1
to remove the options from the argument list.
Then use "$#" rather than $*, and you can handle files with spaces in them.
Thanks a lot!
I changed the code to read:
#!/bin/bash
# rmv - a safe delete program
# todo: add ability to handle wildcards
# uses a trash directory under your home directory
mkdir $HOME/.trash 2>/dev/null
# four internal script variables are defined
cmdlnopts=false
delete=false
empty=false
list=false
# uses getopts command to look at command line for any options
while getopts "dehl" cmdlnopts; do
case "$cmdlnopts" in
d ) echo -e "deleting: \n" "${#:2}" ; delete=true ;;
e ) echo -e "emptying the trash..." ; empty=true ;;
h ) echo -e "safe file delete v1.0"
echo -e "rmv -d[elete] -e[mpty] -h[elp] -l[ist] file [...]" ;;
l ) echo -e "your .trash directory contains:" ; list=true ;;
esac
done
shift $OPTIND-1
if [ $delete = true ]
then
for i in $#
do
mv $i $HOME/.trash
done
echo "rmv finished."
fi
then
/bin/echo "empty the trash? \c"
read answer
case "$answer" in
y) rm -i $HOME/.trash/* ;;
n) /bin/echo "trashcan delete aborted." ;;
esac
fi
if [ $list = true ]
then
ls -l $HOME/.trash
fi
This deletes the files as desired but I get this error:
/home/peter/rmv: line 21: shift: 2-1: numeric argument required
mv: invalid option -- 'd'
Try `mv --help' for more information.
You need to use
shift $(($OPTIND - 1))
to get red of the processed command line args. Try this version:
#!/bin/bash
# rmv - a safe delete program
# uses a trash directory under your home directory
mkdir -p $HOME/.trash
# uses getopts command to look at command line for any options
while getopts "dehl" cmdlnopts; do
case "$cmdlnopts" in
d ) delete=true;;
e ) echo "emptying the trash..." ; empty=true ;;
h ) echo "safe file delete v1.0"
echo "rmv -d[elete] -e[mpty] -h[elp] -l[ist] files" ;;
l ) echo "your .trash directory contains:" ; list=true ;;
esac
done
shift $(($OPTIND - 1))
if [ -n "${delete}" ]; then
echo "deleting: " "${#}"
mv ${#} $HOME/.trash
echo "rmv finished."
fi
if [ -n "${empty}" ]; then
read -p "empty the trash? " answer
case "$answer" in
y) rm -i $HOME/.trash/* ;;
n) echo "trashcan delete aborted." ;;
esac
fi
if [ -n "${list}" ]; then
ls -l $HOME/.trash
fi
Late to the party, but for Googlers, this will generate the error Peter describes:
shift $OPTIND-1
while the syntax in Jurgen's reply will not:
shift $(($OPTIND-1))
The problem is that $OPTIND-1 is interpreted as a string, and shift can't use a string as an argument. $(( )) is Bash's arithmetic expansion operator. You put a string inside it, the string is evaluated as an arithmetic expression, and the value returned.

Resources