script to edit config files - bash

I'm working on a script that would allow me to add, remove, or edit config files. I have tested it a little and it seems like I got it to work at least with a single file, but I would like to be able to just do .config or fi.config and have it perform the desired action.
I would appreciate any help.
Config file looks looks similar to this just bigger
-- Config File
-- Environment DEV7
-------------------------------------------------------------------------------
-------------------------------------------------------------------------------
-- General properties
-------------------------------------------------------------------------------
com.x.yy.zz.version=2.0.2
com.x.yy.zz.instanceRole.ServerA=PRIMARY
com.x.yy.zz.instanceRole.ServerB=SECONDARY
com.x.yy.zz.StopDelay=30
com.x.yy.zz.sourceType=t
com.x.yy.zz.sNumberInc=20000
com.x.yy.zz.sNumberMin=20000
com.x.yy.zz.sNumberMax=9980000
so -a would allow me to add a line after something
ex. -a StopDealy New
com.x.yy.zz.StopDelay=30
New
#!/bin/bash
i=1
usage()
{
echo "test usage"
}
if [[ $# -gt 4 ]]
then
i=2
fi
while [[ $# -gt $i ]]
do
key="$1"
case $key in
-f|--file)
file="$2"
shift
;;
-a|--after)
sed -i "/$2/a $3" $file
#shift # past argument
;;
-b|--before)
sed -i "/$2/i $3" $file
#shift
;;
-d|--delete)
sed -i "/$2/d" $file
#shift
;;
-e|--edit)
sed -ie "s/$2/$3/g" $file
shift
;;
*)
usage
;;
esac
shift # past argument or value
done

I didn't test it yet, but this is the closest version to what I understand you want to achieve.
#!/bin/bash
usage() {
echo "Usage: $0 -f file1 -f *.txt -[abde] ... -f file3 ... -[abde] ... "
}
# Do all the required action on one file
do_work() {
file="$1" # 1st argument must be the file to work on
shift
while [[ $# -gt 0 ]]; do
case "$1" in
-f|--file) while [[ ! "$2" = -* && $# -gt 0 ]]; do shift; done ;; # Ignore any "file" since we have one to work on.
-a|--after) sed -i "/$2/a $3" $file; shift 2 ;;
-b|--before) sed -i "/$2/i $3" $file; shift 2 ;;
-d|--delete) sed -i "/$2/d" $file; shift ;;
-e|--edit) sed -ie "s/$2/$3/g" $file; shift 2 ;;
esac
shift # past argument or value
done
}
# Check the arguments for files and print the valid ones
# Other arguments will just be skipped
# Invalid arguments will be displayed.
identify_files() {
while [[ $# -gt 0 ]]; do
case "$1" in
-f|--file) # Validate the the file exists (just in case)
# check each following argument until next option
# ... or end of arguments
while [[ ! "$2" = -* && $# -gt 0 ]]; do
if [[ -f "$2" ]]; then
echo "$2"
else
echo "Error: Invalid file '$2'." >&2
fi
shift
done ;;
-[abe]) shift 2 ;;
-d) shift ;;
-h) usage >&2; exit ;;
*) echo "Invalid otpion '$1'" >&2 ;;
esac
shift
done
}
# Do the required actions on all files received in the options
for File in $(identify_files "$#"); do
do_work "$File" "$#"
done
## Alternative on predefined files (not the ones received as arguments)
## Usage: $0 -[abde] ...
#for File in $(ls *.config); do
# do_work "$File" "$#"
#done

Related

Parsing optional and not optional arguments

I am new with bash and after reading and trying a lot about how to parse arguments I cannot what I really want to do I want to parse optional and not optional arguments. More specifically I want to parse 3 arguments, first (a fastaq file) second (a second optional fastaq file) a third argument that will be a directory.
my_script.sh -f1 file1.fasta --f2 file2.fasta -d/home/folder1/folder2
or
my_script.sh -f1 file1.fasta -d /home/folder1/folder2
I have tried to do this in many ways but I dont know how to let the program identifies when there are two fasta files and a directory and, when there is only one fasta file and a directory.
With this arguments I want to save them in variables because they will be used later by third parties.
I have tried this:
for i in "$#"; do
case $i in
-f1=|-fasta1=)
FASTA1="${i#=}"
shift # past argument=value
;;
-d) DIRECTORY=$2
shift 2
;;
-d=|-directory=) DIRECTORY="${i#=}"
shift # past argument=value
;;
--f2=|-fasta2=) FASTA2="${i#*=}"
shift # past argument=value
;;
*)
;;
esac
done
But I just got this
scripts_my_first_NGS]$ ./run.sh -f1 fasta.fasta -d /home/folder1
FASTA1 =
DIRECTORY =
FASTA2 =
Never parse command line options on your own!
Instead either use the Bash function getopts, if you do not need GNU style long options or use use the GNU program getopt otherwise.
The following examples uses an array for FASTA. FASTA1 is ${FASTA[0]} and FASTA2 is ${FASTA[1]}. In case of getopts this makes it possible to use just one option character (-f) multiple times.
Using getopts with only one-character options:
#! /bin/bash
FASTA=()
DIRECTORY=
while getopts 'f:d:' option; do
case "$option" in
f)
FASTA+=("$OPTARG")
;;
d)
DIRECTORY="$OPTARG"
;;
*)
printf 'ERROR: Invalid argument\n' >&2
exit 1
;;
esac
done
shift $((OPTIND-1))
if [[ -z ${FASTA[0]} ]]; then
printf 'ERROR: FASTA1 missing\n' >&2
exit 1
fi
if [[ -z $DIRECTORY ]]; then
printf 'ERROR: DIRECTORY missing\n' >&2
exit 1
fi
printf 'FASTA1 = %s\n' "${FASTA[0]}"
printf 'FASTA2 = %s\n' "${FASTA[1]}"
printf 'DIRECTORY = %s\n' "$DIRECTORY"
Usage:
run -f file1.fasta -f file2.fasta -d /home/folder1/folder2
Using getopt with one-character and GNU style long options mixed:
#! /bin/bash
FASTA=()
DIRECTORY=
options=$(getopt -o d: -l f1: -l f2: -- "$#") || {
printf 'ERROR: Invalid argument\n' >&2
exit 1
}
eval set -- "$options"
while true; do
case "$1" in
--f1)
FASTA[0]="$2"
shift 2;;
--f2)
FASTA[1]="$2"
shift 2;;
-d)
DIRECTORY="$2"
shift 2;;
--)
shift
break;;
*)
break;;
esac
done
if [[ -z ${FASTA[0]} ]]; then
printf 'ERROR: FASTA1 missing\n' >&2
exit 1
fi
if [[ -z $DIRECTORY ]]; then
printf 'ERROR: DIRECTORY missing\n' >&2
exit 1
fi
printf 'FASTA1 = %s\n' "${FASTA[0]}"
printf 'FASTA2 = %s\n' "${FASTA[1]}"
printf 'DIRECTORY = %s\n' "$DIRECTORY"
Usage:
run --f1 file1.fasta --f2 file2.fasta -d /home/folder1/folder2
Basically you need to add a separate parser for versions of the options where they aren't used with the equal sign.
Also your shift commands are useless since you're processing a for loop. So convert it to to a while [[ $# -gt 0 ]]; do loop instead.
I also added a few modifications which I suggest be added.
while [[ $# -gt 0 ]]; do
case $1 in
-f1|-fasta1)
FASTA1=$2
shift
;;
-f1=*|-fasta1=*)
FASTA1=${1#*=}
;;
-d|-directory)
DIRECTORY=$2
shift
;;
-d=*|-directory=*)
DIRECTORY=${1#*=}
;;
-f2|fasta2)
FASTA2=$2
shift
;;
-f2=*|-fasta2=*)
FASTA2=${1#*=}
;;
-*)
echo "Invalid option: $1" >&2
exit 1
;;
--)
# Do FILES+=("${#:2}") maybe
break
;;
*)
# TODO
# Do FILES+=("$1") maybe
;;
esac
shift
done
The "parser" for the with-equal and non-with-equal versions of the options can also be unified by
using a helper function:
function get_opt_arg {
if [[ $1 == *=* ]]; then
__=${1#*=}
return 1
elif [[ ${2+.} ]]; then
__=$2
return 0 # Tells that shift is needed
else
echo "No argument provided to option '$1'." >&2
exit 1
fi
}
while [[ $# -gt 0 ]]; do
case $1 in
-d|-directory|-d=*|-directory=*)
get_opt_arg "$#" && shift
DIRECTORY=$__
;;
-f1|-fasta1|-f1=*|-fasta1=*)
get_opt_arg "$#" && shift
FASTA1=$__
;;
-f2|fasta2|-f2=*|-fasta2=*)
get_opt_arg "$#" && shift
FASTA2=$__
;;
-*)
echo "Invalid option: $1" >&2
exit 1
;;
--)
# Do FILES+=("${#:2}") maybe
break
;;
*)
# TODO
# Do FILES+=("$1") maybe
;;
esac
shift
done
Update
I found a complete solution to command-line parsing without relying on getopt[s] and it does it even more consistentlty: https://konsolebox.io/blog/2022/05/14/general-command-line-parsing-solution-without-using-getopt-s.html

How to make it manditory for options to be spaced for bash scripts

I have the following script.
I would like to modify it so that if I were to call temp.sh with both the options, I would have to space them. Ie: A call to the script like temp.sh -fc30 should be invalid, rather it should be temp.sh -f -c 30
ARGS=$(getopt -o c:f -l "charlie:fox" -n "temp.sh" -- "$#");
#bad args
if [ $? -ne 0 ];
then
exit 1
fi
eval set --"$ARGS";
while true; do
case "$1" in
-c|--charlie)
shift;
if [ -n "$1" ]; then
echo "-c =: $1";
shift;
fi
;;
-f|--fox)
shift;
echo "fox used";
;;
--)
shift;
break;
;;
esac
done
Just don't use getopt.
#!/bin/bash
# parse options
while [[ $# -gt 0 ]]; do
case $1 in
-c|--charlie)
echo "$1 = $2"
shift
;;
-f|--fox)
echo "fox used"
;;
--)
shift
break
esac
shift
done
# do script

Print bash script argument passed into a new script generated by your current script

Title might be confusing -- I'm writing up a bash script that creates another bash script in the same relative directory, and I want to print an argument passed into the second script.
Layout:
#!/bin/bash
set -e
while [[ $# -gt 1 ]]; do
key="$1"
if [[ ! "$2" =~ ^-[^-].* ]];
case $key in
--arg1 | -t)
arg1=$2
shift #shift past argument
;;
--arg2 | -k)
arg2=$2
shift #shift past argument
;;
--arg3 | -i)
arg3=$2
break
;;
*)
;;
esac
fi
done
cat <<'EOF' > secondScript.sh
#!/bin/bash
set -e
while [[ $# -gt 1 ]]; do
key="$1"
if [[ ! "$2" =~ ^-[^-].* ]]; then # don't gobble up next key if this key doesn't have a value
case $key in
--finalArg1 | -t)
finalArg1=$2
shift #shift past argument
;;
--finalArg2 | -k)
finalArg2=$2
shift #shift past argument
;;
--finalArg3 | -i)
finalArg3=$2
break
;;
*)
;;
esac
fi
done
echo ${finalArg1}
EOF
chmod +x secondScript.sh
./secondScript.sh --finalArg1 {arg1} --finalArg2 {arg2} --finalArg3 {arg3}
so the end result is arg1 printed to the console. Instead, this code just prints
{finalArg1}. Any way to do this?
Ah nvm this was just an oversight -- Forgot to add $ to substitute the variables on the second script call. Should look like this:
./secondScript.sh --finalArg1 ${arg1} --finalArg2 ${arg2} --finalArg3 ${arg3}

cut -d, ignore delimiter in argument

I have written a script that gets a variable number of arguments:
test.sh -i <input1> <input2> ... -o <output1> <output2> ...
I'm parsing the arguments as follows:
while [ $# -gt 1 ]; do
TMP=$(echo "$#" | cut -d '-' -f 2) #i <input1> <input2>
TMP1=$(echo "$TMP" | cut -d ' ' -f 1) #i
CNT=$(echo "$TMP" | wc -w) #3
set -x
case "$TMP1" in
i)
INPUTS=$(echo "$TMP" | cut -c 3-)
shift "$CNT"
;;
o)
OUTPUTS=$(echo "$TMP" | cut -c 3-)
shift "$CNT"
;;
esac
done
This works everytime, except for files that happen to have a '-' in their name.
Example:
./test.sh -i file1.txt file-2.txt -o out1.txt out-2.txt
Is there anyway I can force cut to ignore delimiters that occur within the file names?
You don't need all this string manipulation; each argument is already a separate word.
while (( $# > 0 )); do
case $1 in
-i) shift
while [[ $# -gt 0 && $1 != -* ]]; do
inputs+=( "$1" )
shift
done
;;
-o) shift
while [[ $# -gt 0 && $1 != -* ]]; do
outputs+=( "$1" )
shift
done
;;
*) echo "Unrecognized option $1"
exit 1
;;
esac
done
This can be refactored a little to avoid the repeated checks for running out of arguments.
for arg in "$#"; do
case $1 in
-i) mode=input; continue ;;
-o) mode=output; continue ;;
esac
case $mode in
input) input+=("$arg") ;;
output) output+=("$arg") ;;
*) echo "Unknown mode: $mode"
exit 1
;;
esac
done
Here's an alternative approach that may benefit someone.
The fact is, argument parsing is always a tradeoff, hence there's benefit in tailoring it to the application. Here's a pretty generic solution that allows a little bit of error checking and disorder in the arguments.
It's very simple, but I have added some example output and comments, and for the sake of readability and compatibility, stayed away from complex ways to save a line or two (especially on the if statements).
Sample Usage:
bash #> touch file-1 file3 file4 file-8 file7
bash #> argparse -i file-1 file3 file4 -c -k --q --j -r -t -o file-8 file7
Output:
Input files: file-1 file3 file4
Output files: file-8 file7
Args are: c k q j r t
Doing action for argument "c"
Doing action for argument "k"
Doing action for argument "j"
Script:
#!/bin/bash
#argparse
#Assign arrays
until [[ $# < 1 ]]; do
#ignore args "-i" and "-o", and tell the script to check for files following
if [ "$1" == "-i" ] ; then unset output ; input=1 ; shift
elif [ "$1" == "-o" ] ; then unset input ; output=1 ; shift
fi
#Add input and output files to respective arrays
if [ -f "$1" ] ; then
if [[ $input == 1 ]]; then
infiles+=($1)
elif [[ $output == 1 ]]; then
outfiles+=($1)
fi
else
#Add args to array
arg="$(echo "$1" | sed 's/-//g')"
args+=($arg)
fi
shift
done
#Some debug feedback
echo -e "Input files: ${infiles[#]}\nOutput files: ${outfiles[#]}\nArgs are: ${args[#]}\n"
#Simulate actually "doing" something with the args
for arg in "${args[#]}" ; do
case $arg in
"c") echo "Doing action for argument \"c\"" ;;
"k") echo "Doing action for argument \"k\"" ;;
"j") echo "Doing action for argument \"j\"" ;;
*) ;;
esac
done
Update/Edit: I've just realised, that the OP didn't have any requirement for parsing actual arguments other than -i and -o. Well regardless, this may still come in handy for someone at some point.

How to Use Bash getopts Command to Get All Arguments Together

I have couple of files needed to be parsed here, called parse.sh. I have to enable optional arguments for it with -l for line, -f for field. So to run the program would be ./parse.sh -l 5 -f 14 foo. Without -l or -f arguments, I want the program to default parse all lines and all fields. If -l is specified, I want it to parse just that line of foo, and further if -f is specified too, I want it to parse just that field. I see getopts usually work like this:
while getopts "l:f:" opts; do
case $opts in
l) #code to parse that line;;
f) #code to parse that field;;
case
done
BUT this is not what I need, because I want -l and -f work together sometimes. I am thinking maybe I should do getopts to parse all options into array, then write code based on parsing that array? Any better choices?
This is my code:
while getopts "l:f:" opt;
do
options=${opt}${options}
case $opt in
l) lineNumber=$OPTARG ;;
f) fieldNumber=$OPTARG ;;
esac
done
case $options in
f) echo "Parse field $fieldNumber of each line" ;;
l) echo "Parse all fields of line number $lineNumber" ;;
lf | fl) echo "Parse field $fieldNumber of line $lineNumber" ;;
*) echo "Parse all fields of all lines" ;;
esac
#!/bin/bash
parse()
{
local lines=$1
local fields=$2
local file=$3
# logic goes here
echo "parsing line(s) ${lines} and field(s) ${fields} of file ${file}"
}
lines=all
fields=all
while getopts "l:f:" o; do
case $o in
l) lines=${OPTARG} ;;
f) fields=${OPTARG} ;;
esac
done
shift $((OPTIND-1))
for file; do
parse "${lines}" "${fields}" "${file}"
done
Example runs:
$ ./t.sh foo.txt bar.txt
parsing line(s) all and field(s) all of file foo.txt
parsing line(s) all and field(s) all of file bar.txt
$ ./t.sh -l 10 foo.txt bar.txt
parsing line(s) 10 and field(s) all of file foo.txt
parsing line(s) 10 and field(s) all of file bar.txt
$ ./t.sh -l 10 -f 5 foo.txt bar.txt
parsing line(s) 10 and field(s) 5 of file foo.txt
parsing line(s) 10 and field(s) 5 of file bar.txt
$ ./t.sh -f 5 foo.txt bar.txt
parsing line(s) all and field(s) 5 of file foo.txt
parsing line(s) all and field(s) 5 of file bar.txt
#!/bin/bash
while getopts "l:f:" opts; do
case $opts in
l)
lOn=1
;;
f)
fOn=1
;;
esac
done
if [[ -n $lOn && -n $fOn ]]; then
echo 'both l and f'
elif [[ -n $lOn ]]; then
echo 'just l'
elif [[ -n $fOn ]]; then
echo 'just f'
else
echo 'nothing'
fi
If statements give you more flexibility in situations when you want to check for other variables or do more complex stuff. This wont work in sh unless you change [[ ]] to [ ].
I made a concept script. Please try.
#!/bin/bash
PARSE_SPECIFIC_LINE=0
PARSE_SPECIFIC_FIELD=0
shopt -s extglob
while getopts "l:f:" opts; do
case $opts in
l)
if [[ $OPTARG != +([[:digit:]]) || OPTARG -lt 1 ]]; then
echo "Invalid argument to -l: $OPTARG"
exit 1
fi
PARSE_SPECIFIC_LINE=$OPTARG
;;
f)
if [[ $OPTARG != +([[:digit:]]) || OPTARG -lt 1 ]]; then
echo "Invalid argument to -f: $OPTARG"
exit 1
fi
PARSE_SPECIFIC_FIELD=$OPTARG
;;
esac
done
FILES=("${#:OPTIND}")
function parse_line {
local LINE=$1
if [[ -n $LINE ]]; then
if [[ PARSE_SPECIFIC_FIELD -gt 0 ]]; then
read -ra FIELDS <<< "$LINE"
echo "${FIELDS[PARSE_SPECIFIC_FIELD - 1]}"
else
echo "$LINE"
fi
fi
}
for F in "${FILES[#]}"; do
if [[ -e $F ]]; then
if [[ PARSE_SPECIFIC_LINE -gt 0 ]]; then
parse_line "$(sed -n "${PARSE_SPECIFIC_LINE}{p;q}" "$F")"
else
while read -r LINE; do
parse_line "$LINE"
done < "$F"
fi
else
echo "File does not exist: $F"
fi
done
I ran it with
bash script.sh -f 2 <(for i in {1..20}; do echo "$RANDOM $RANDOM $RANDOM $RANDOM $RANDOM"; done)
And I got
1031
1072
4350
12471
31129
32318
...
Adding -l 5 I got
11604

Resources