Related
I'm trying to make a getopt command such that when I pass the "-ab" parameter to a script,
that script will treat -ab as a single parameter.
#!/bin/sh
args=`getopt "ab":fc:d $*`
set -- $args
for i in $args
do
case "$i" in
-ab) shift;echo "You typed ab $1.";shift;;
-c) shift;echo "You typed a c $1";shift;;
esac
done
However, this does not seem to work. Can anyone offer any assistance?
getopt doesn't support what you are looking for. You can either use single-letter (-a) or long options (--long). Something like -ab is treated the same way as -a b: as option a with argument b. Note that long options are prefixed by two dashes.
i was struggling with this for long - then i got into reading about getopt and getopts
single char options and long options .
I had similar requirement where i needed to have number of multichar input arguments.
so , i came up with this - it worked in my case - hope this helps you
function show_help {
echo "usage: $BASH_SOURCE --input1 <input1> --input2 <input2> --input3 <input3>"
echo " --input1 - is input 1 ."
echo " --input2 - is input 2 ."
echo " --input3 - is input 3 ."
}
# Read command line options
ARGUMENT_LIST=(
"input1"
"input2"
"input3"
)
# read arguments
opts=$(getopt \
--longoptions "$(printf "%s:," "${ARGUMENT_LIST[#]}")" \
--name "$(basename "$0")" \
--options "" \
-- "$#"
)
echo $opts
eval set --$opts
while true; do
case "$1" in
h)
show_help
exit 0
;;
--input1)
shift
empId=$1
;;
--input2)
shift
fromDate=$1
;;
--input3)
shift
toDate=$1
;;
--)
shift
break
;;
esac
shift
done
Note - I have added help function as per my requirement, you can remove it if not needed
That's not the unix way, though some do it e.g. java -cp classpath.
Hack: instead of -ab arg, have -b arg and a dummy option -a.
That way, -ab arg does what you want. (-b arg will too; hopefully that's not a bug, but a shortcut feature...).
The only change is your line:
-ab) shift;echo "You typed ab $1.";shift;;
becomes
-b) shift;echo "You typed ab $1.";shift;;
GNU getopt have --alternative option
-a, --alternative
Allow long options to start with a single '-'.
Example:
#!/usr/bin/env bash
SOPT='a:b'
LOPT='ab:'
OPTS=$(getopt -q -a \
--options ${SOPT} \
--longoptions ${LOPT} \
--name "$(basename "$0")" \
-- "$#"
)
if [[ $? > 0 ]]; then
exit 2
fi
A=
B=false
AB=
eval set -- $OPTS
while [[ $# > 0 ]]; do
case ${1} in
-a) A=$2 && shift ;;
-b) B=true ;;
--ab) AB=$2 && shift ;;
--) ;;
*) ;;
esac
shift
done
printf "Params:\n A=%s\n B=%s\n AB=%s\n" "${A}" "${B}" "${AB}"
$ ./test.sh -a aaa -b -ab=test
Params:
A=aaa
B=true
AB=test
getopt supports long format. You can search SO for such examples.
See here, for example
I am writing a Bash script to parse a CSV file (values separated by ; character) and extract some arguments. Depending on the current argument read from the file a specific string is appended to a variable. However, the case statement always enters the default state *) and I can't figure out why.
I have no problems reading the .csv file. Already did that and displayed the output of the arguments read from the file. Everything works perfectly. The problem is that the case statement is not processed as expected.
So, the problem is not reading the .csv file but the processing of the arguments in the case statement.
This is my code:
while IFS=";" read -r arg
do
case ${arg} in
"valueX")
var+="blabla"
;;
"valueY")
var+="blublu"
;;
*)
echo -e "Argument ${arg} not supported"
exit 1
;;
esac
done < filename
Assuming "valueX" is the current argument read from the file. Somehow the script always outputs:
Argument "valueX" not supported
Apparently, the argument (here: "valueX") read from the file is correct, but the script won't enter the corresponding state. Instead, it always enters the default state, no matter what value ${arg} holds.
[EDIT]
I thought it would be a good idea to ask the question more generally, but it turns out to be confusing. So here is the full bash script and .csv file:
Script:
#!/bin/bash
# style reset
STYLE_RESET='\e[0m'
# Red foreground color
FOREGROUND_RED='\e[31m'
# Green foreground color
FOREGROUND_GREEN='\e[32m'
# Blue foreground color
FOREGROUND_BLUE='\e[34m'
# Red background color
BACKGROUND_RED='\e[41m'
# Green background color
BACKGROUND_GREEN='\e[42m'
# Blue background color
BACKGROUND_BLUE='\e[44m'
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )"
usage()
{
echo "ToDo"
exit 1
}
# --------------------------------------- #
# --- Checking Command Line Arguments --- #
# --------------------------------------- #
# Supported command line arguments:
# -h|--help
# -a|--address IP of SSH server for remote VMAF
# -u|--user User for SSH server login
# -d|--doe DoE worksheet exported as CSV UTF-8 file
PARAMS=""
while (( "$#" )); do
case "$1" in
-h|--help)
usage
shift
;;
-u|--user)
if [ -n "$2" ] && [ ${2:0:1} != "-" ]; then # Check length of argument and first character of argument != '-'
SSH_USER=$2
shift 2
else
echo "Error: Argument for $1 is missing" >&2
exit 1
fi
;;
-a|--address)
if [ -n "$2" ] && [ ${2:0:1} != "-" ]; then # Check length of argument and first character of argument != '-'
SSH_IP=$2
shift 2
else
echo "Error: Argument for $1 is missing" >&2
exit 1
fi
;;
-d|--doe)
if [ -n "$2" ] && [ ${2:0:1} != "-" ]; then # Check length of argument and first character of argument != '-'
DOE_FILE=$2
shift 2
else
echo "Error: Argument for $1 is missing" >&2
exit 1
fi
;;
-*|--*=) # unsupported flags
echo "Error: Unsupported flag $1" >&2
exit 1
;;
*) # DEFAULT
PARAMS="${PARAMS} $1" # preserve positional arguments
shift
;;
esac
done
# set positional arguments in their proper place
eval set -- "${PARAMS}"
# ---------------------- #
# --- Processing DoE --- #
# ---------------------- #
echo -e "${BACKGROUND_BLUE}Processing DoE specified in file ${DOE_FILE}:${STYLE_RESET}"
echo -e "${BACKGROUND_BLUE}Configuring Video source for GStreamer pipeline...${STYLE_RESET}"
GSTPIPE_SRC="gst-launch-1.0 -e "
run=1
while IFS=";" read -r motion bitrate_in bitrate_out twopass iframe quantI quantP quantB mvbuffer cabac vbv
do
echo -e "\n\n${BACKGROUND_BLUE}Setting #${run}:${STYLE_RESET}"
echo -e "${BACKGROUND_BLUE}${motion} ${bitrate_in} ${bitrate_out} ${twopass} ${iframe} ${quantI} ${quantP} ${quantB} ${mvbuffer} ${cabac} ${vbv}${STYLE_RESET}"
echo -e "\n${BACKGROUND_BLUE}Generating GStreamer pipelines...${STYLE_RESET}"
case ${motion} in
"low")
GSTPIPE_SRC+="videotestsrc pattern=colors num-buffers=300 ! " # -> no motion content
case ${bitrate_in} in # -> bitrate of video source (width*height*framerate)
"low") # -> 640x480
GSTPIPE_SRC+="'video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, format=(string)I420' "
width=640
height=480
fps=30
;;
"high") # -> 3840x2160
GSTPIPE_SRC+="'video/x-raw, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, format=(string)I420' "
width=3840
height=2160
fps=30
;;
*)
echo -e "\n\n${BACKGROUND_RED}Input bitrate ${bitrate_in} not supported${STYLE_RESET}"
echo -e "Use low, or high instead"
exit 1
;;
esac
;;
"high")
GSTPIPE_SRC+="filesrc location=${SCRIPT_DIR}/extensive.mp4 " # -> high motion content
case ${bitrate_in} in # -> bitrate of video source (width*height*framerate)
"low") # -> 640x480
GSTPIPE_SRC+="blocksize=460800 ! " # blocksize=width*height*bytesPerPixel (I420->12bit->bytesPerPixel=1.5)
GSTPIPE_SRC+="'video/x-raw, width=(int)640, height=(int)480, framerate=(fraction)30/1, format=(string)I420' "
width=640
height=480
fps=30
;;
"high") # -> 3840x2160
GSTPIPE_SRC+="blocksize=12441600 ! " # blocksize=width*height*bytesPerPixel (I420->12bit->bytesPerPixel=1.5)
GSTPIPE_SRC+="'video/x-raw, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, format=(string)I420' "
width=3840
height=2160
fps=30
;;
*)
echo -e "\n\n${BACKGROUND_RED}Input bitrate ${bitrate_in} not supported${STYLE_RESET}"
echo -e "Use low, or high instead"
exit 1
;;
esac
;;
*)
echo -e "${BACKGROUND_RED}Argument ${motion} for DoE factor 'motion' not supported${STYLE_RESET}"
echo -e "Use low, or high instead"
exit 1
;;
esac
GSTPIPE_ENC=$GSTPIPE_SRC
GSTPIPE_REF=$GSTPIPE_SRC
GSTPIPE_REF+="! y4menc ! filesink location=${SCRIPT_DIR}/reference${fps}fps_run${run}.y4m"
GSTPIPE_ENC+="! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvv4l2h264enc "
GSTPIPE_ENC+="bitrate=${bitrate_out} EnableTwopassCBR=${twopass} "
case ${iframe} in
"low")
GSTPIPE_ENC+="iframeinterval=20 SliceIntraRefreshInterval=10 "
;;
"high")
GSTPIPE_ENC+="iframeinterval=120 SliceIntraRefreshInterval=60 "
;;
*)
echo -e "${BACKGROUND_RED}Argument ${motion} for DoE factor iframe is not supported${STYLE_RESET}"
echo -e "Use low, or high instead"
exit 1
;;
esac
# The range of B frames does not take effect if the number of B frames is 0. (https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/accelerated_gstreamer.html#wwpID0E0YX0HA)
GSTPIPE_ENC+="quant-i-frames=${quantI} quant-p-frames=${quantP} quant-b-frames=${quantB} num-b-frames=1 EnableMVBufferMeta=${mvbuffer} cabac-entropy-coding=${cabac} "
GSTPIPE_ENC+="! nvv4l2decoder ! nvvidconv ! 'video/x-raw' ! y4menc ! filesink location=${SCRIPT_DIR}/distorted${fps}fps_run${run}.y4m"
echo -e "${BACKGROUND_BLUE}Distorted Video:${STYLE_RESET}"
echo -e "${FOREGROUND_BLUE}${GSTPIPE_ENC}${STYLE_RESET}"
echo -e "${BACKGROUND_BLUE}Reference Video:${STYLE_RESET}"
echo -e "${FOREGROUND_BLUE}${GSTPIPE_REF}${STYLE_RESET}"
# --- Launching GStreamer pipelines (surpress detailed output) --- #
echo -e "${BACKGROUND_BLUE}Launching GStreamer pipeline for encoded video (distorted):${STYLE_RESET}"
eval "${GSTPIPE_ENC[#]}" #> /dev/null
echo -e "${BACKGROUND_BLUE}Launching GStreamer pipeline for uncompressed video (reference):${STYLE_RESET}"
eval "${GSTPIPE_REF[#]}" #> /dev/null
# --- Create and Check Remote Directories --- #
echo -e "\n${BACKGROUND_BLUE}Video transfer to remote machine:${STYLE_RESET}"
SSH_DIR_ADD="v4l2h264/motion_${motion}/bitrate_${bitrate_in}/"
SSH_DIR="/home/${SSH_USER}/metrics/${SSH_DIR_ADD}" # Create variable holding path of directory for both:
# 1.) reference.y4m and distorted.y4m videos
# 2.) remote VMAF
ssh ${SSH_USER}#${SSH_IP} "test -d ${SSH_DIR}" < /dev/null # see https://stackoverflow.com/questions/9393038/ssh-breaks-out-of-while-loop-in-bash
if [ $? -ne 0 ]; then # Directory does not exist
echo -e "${BACKGROUND_BLUE}Creating remote directory for run #${run}: ${SSH_DIR}...${STYLE_RESET}"
ssh ${SSH_USER}#${SSH_IP} "mkdir -p ${SSH_DIR}" < /dev/null # see https://stackoverflow.com/questions/9393038/ssh-breaks-out-of-while-loop-in-bash
else # Directory already exists
echo -e "${BACKGROUND_BLUE}Remote directory ${SSH_DIR} already exists${STYLE_RESET}"
fi
# --- Transfer Video Files --- #
echo -e "${BACKGROUND_BLUE}Transfering reference and distorted videos of run #${run}:${STYLE_RESET}"
scp ${SCRIPT_DIR}/distorted${fps}fps_run${run}.y4m ${SSH_USER}#${SSH_IP}:${SSH_DIR} < /dev/null # see https://stackoverflow.com/questions/9393038/ssh-breaks-out-of-while-loop-in-bash
scp ${SCRIPT_DIR}/reference${fps}fps_run${run}.y4m ${SSH_USER}#${SSH_IP}:${SSH_DIR} < /dev/null # see https://stackoverflow.com/questions/9393038/ssh-breaks-out-of-while-loop-in-bash
# --- Run VMAF on Remote Machine --- #
echo -e "\n${BACKGROUND_BLUE}Running VMAF metric for DoE run #${run} on remote machine...${STYLE_RESET}"
ssh ${SSH_USER}#${SSH_IP} "vmaf -r ${SSH_DIR}/reference${fps}fps_run${run}.y4m -d ${SSH_DIR}/distorted${fps}fps_run${run}.y4m -w ${width} -h ${height} -p 420 -b 12 -o ${SSH_DIR}/log${fps}fps_run${run}.xml --threads 8" < /dev/null # see https://stackoverflow.com/questions/9393038/ssh-breaks-out-of-while-loop-in-bash
echo -e "${BACKGROUND_BLUE}VMAF metric for DoE run #${run} finished.${STYLE_RESET}"
# --- Remove Videos on Remote Machine (Laptop) --- #
echo -e "\n${BACKGROUND_BLUE}Removing videos from remote machine${STYLE_RESET}"
ssh ${SSH_USER}#${SSH_IP} "rm ${SSH_DIR}/distorted${fps}fps_run${run}.y4m ${SSH_DIR}/reference${fps}fps_run${run}.y4m" < /dev/null # see https://stackoverflow.com/questions/9393038/ssh-breaks-out-of-while-loop-in-bash
# --- Remove videos on local machine (SPU) --- #
echo -e "\n${BACKGROUND_BLUE}Removing videos from local machine${STYLE_RESET}"
rm distorted${fps}fps_run${run}.y4m reference${fps}fps_run${run}.y4m
((run++))
done < <(cut -d ";" -f5,6,7,8,9,10,11,12,13,14,15,16 ${DOE_FILE} | tail -n +2) # read from the second line of the file (no header) and only read the columns specified with -f
((run--))
exit 0
.csv file:
"StdOrder";"RunOrder";"CenterPt";"Blocks";"motion";"bitrate_in";"bitrate_out";"twopass";"iframe";"quantI";"quantP";"quantB";"mvbuffer";"cabac"
1;1;1;1;"low";"low";200000;"false";"low";0;0;0;"true";"true"
6;2;1;1;"high";"low";80000000;"false";"low";51;0;51;"true";"false"
8;3;1;1;"high";"high";80000000;"false";"high";0;0;0;"false";"true"
2;4;1;1;"high";"low";200000;"false";"high";0;51;51;"false";"false"
3;5;1;1;"low";"high";200000;"false";"high";51;0;51;"false";"false"
7;6;1;1;"low";"high";80000000;"false";"low";0;51;51;"true";"false"
10;7;1;1;"high";"low";200000;"true";"high";51;0;0;"true";"false"
9;8;1;1;"low";"low";200000;"true";"low";51;51;51;"false";"true"
4;9;1;1;"high";"high";200000;"false";"low";51;51;0;"true";"true"
13;10;1;1;"low";"low";80000000;"true";"high";0;0;51;"true";"true"
5;11;1;1;"low";"low";80000000;"false";"high";51;51;0;"false";"true"
12;12;1;1;"high";"high";200000;"true";"low";0;0;51;"false";"true"
15;13;1;1;"low";"high";80000000;"true";"low";51;0;0;"false";"false"
14;14;1;1;"high";"low";80000000;"true";"low";0;51;0;"false";"false"
16;15;1;1;"high";"high";80000000;"true";"high";51;51;51;"true";"true"
11;16;1;1;"low";"high";200000;"true";"high";0;51;0;"true";"false"
Apparently, the argument (here: "valueX") read from the file is correct
Since you consider the quotes being part of the argument to be correct, in order to match them with a case pattern, you have to escape the pattern quotes:
case ${arg} in
\"valueX\")
Interestingly, this appears to be a Bash (documentation) error, as that says only:
Each pattern undergoes tilde expansion, parameter expansion, command substitution, and arithmetic expansion.
It doesn't tell that the pattern undergoes quote removal.
None of your sample data contains valueX, but assuming you were looking for e.g. "high" in the fifth field, you need to ask the shell to split into at least six fields;
while IFS=";" read -r _first _second _third _fourth arg _rest
do
case ${arg} in
'"high"')
var+="blabla"
;;
'"low"')
var+="blublu"
;;
*)
echo -e "Argument ${arg} not supported"
exit 1
;;
esac
done < filename
Probably a better solution here is to use Awk, though.
awk -F ';' '$5 !~ /"(high|low)"/ { print "Argument " $5 " is not supported" }' filename
Without the rest of your script, it's hard to tell in which direction to continue this; very often, if you are using Awk anyway, it makes sense to implement the rest of the logic in Awk, too.
Save the following code as testme.sh.
OPTS=$(getopt -o a:b:c -- "$#")
eval set -- "$OPTS"
while true; do
case "$1" in
-a )
echo "i am in a",$OPTIND
shift 2;;
-b )
echo "i am in b",$OPTIND
shift 2;;
-c )
echo "i am in c",$OPTIND
shift;;
-- )
shift;;
*)
break;;
esac
done
Run it with bash /tmp/testme.sh -a a1 -b b1 -c.
i am in a,1
i am in b,1
i am in c,1
Now wrap all content in testme.sh as a function.
testme(){
OPTS=$(getopt -o a:b:c -- "$#")
eval set -- "$OPTS"
while true; do
case "$1" in
-a )
echo "i am in a",$OPTIND
shift 2;;
-b )
echo "i am in b",$OPTIND
shift 2;;
-c )
echo "i am in c",$OPTIND
shift;;
-- )
shift;;
*)
break;;
esac
done
}
Run it with testme -a a1 -b b1 -c.
i am in a,
i am in b,
i am in c,
There are 2 issues confused me.
1.Why all $OPTINDs value is 1 when to run bash /tmp/testme.sh -a a1 -b b1 -c ?
2.Why no $OPTINDs value at all when to run testme -a a1 -b b1 -c ?
getopt is an external command and runs in a subprocess, so it can't modify the original shell's variables. Because of this, it can't set variables like OPTIND and OPTARG, the way the built-in command getopts does. It simply outputs a modified version of the argument list, which can be assigned to positional parameters with set.
So when you use getopt rather than getopts, $OPTIND is not updated. It's initialized to 1 when a shell script starts. Since your testme.sh script never does anything that updates the variable, you get 1 every time through the loop.
When I try your testme function, I also see 1 every time. If you're not seeing this, you must have reassigned OPTIND before running the function. Try:
OPTIND=123
testme -a a1 -b b1 -c
and you should see
i am in a,123
i am in b,123
i am in c,123
I am creating a BASH script and working with the BASH getopt command to parse command line arguments. Instead of returning the arguments provided on the command line, getopt returns the arguments that were supplied to the getopt command. I am not sure what could be going on, as the code that I have was working perfect, and seemingly out of nowhere, it has stoped working correctly (and no, I haven't updated anything or changed any code or environment settings). I can't use getopts (with the extra 's') because it is, for some unknown reason, not installed on the machine that will be running this script.
Even though the script is supplied with zero command line arguments, the getopt command is for some reason returning all of the arguments that I have supplied, minus the -o flag, instead of the expected -- value indicating the end of the options. The code that I have is as follows:
SHORT_OPTS=":hvso:l:d:n:p:t:"
LONG_OPTS="help,version,submit-job,output:,library:,job-dir:"
LONG_OPTS="${LONG_OPTS},num-nodes:,num-procs:,max-time:"
OPTS=$(getopt -o "${SHORT_OPTS}" -l "${LONG_OPTS}" -a -n "${PROG_NAME}" -- "${#}")
# Check for invalid command line options and arguments
if [[ ${?} -ne ${SUCCESS} ]] ; then
echo -e "${PROG_NAME}: error: Invalid option or argument\n" >&2
usage ; exit ${FAILURE}
else
echo "BEFORE $#"
eval set -- ${OPTS}
echo "AFTER $#"
fi
# Process command line options and their arguments
while true ; do
case "${1}" in
-h | --help)
# Display script usage information and exit
usage ; exit ${SUCCESS} ;;
-v | --version)
# Display script version information and exit
echo "${PROG_NAME} v${PROG_VERSION}" ; exit ${SUCCESS} ;;
-s | --submit-job)
# Enable automatic submission of the Moab job
JOB_AUTO_SUBMIT="${PREF_YES}" ; shift 1 ;;
-o | --output)
# Set the base name for output file names
TARGET="${2}" ; shift 2 ;;
-l | --library)
# Set the library to use for NWChem atomic configurations
NW_LIB="${2}" ; shift 2 ;;
-d | --job-dir)
# Ensure the specified directory for the Moab job exists
if [[ -e "${2}" ]] ; then
JOB_WORK_DIR=$(resolvePath "${2}") ; shift 2
else
echo -e "${PROG_NAME}: error: -d ${2}: No such directory\n"
usage ; exit ${FAILURE}
fi ;;
-n | --num-nodes)
# Ensure the number of compute nodes is greater than zero
if positiveInt "${2}" ; then
JOB_NODES="${2}" ; shift 2
else
echo -n "${PROG_NAME}: error: -n ${1}: Number of "
echo -e "job nodes must be a positive integer\n"
usage ; exit ${FAILURE}
fi ;;
-p | --num-procs)
# Ensure the number of processors per node is greater than zero
if positiveInt "${2}" ; then
JOB_PROCS="${2}" ; shift 2
else
echo -n "${PROG_NAME}: error: -p ${2}: Number of "
echo -e "processors per node must be a positive integer\n"
usage ; exit ${FAILURE}
fi ;;
-t | --max-time)
# Ensure the maximum job runtime is in the correct format
if [[ "${2}" == [0-9][0-9]:[0-9][0-9]:[0-9][0-9] ]] ; then
JOB_MAX_TIME="${2}" ; shift 2
else
echo -n "${PROG_NAME}: error: -t ${2}: Invalid time "
echo -e "format, please use hh:mm:ss format\n"
usage ; exit ${FAILURE}
fi ;;
--)
# No more options to process
shift ; break ;;
esac
done
# Check to see if POTCAR and CONTCAR locations were specified
if [[ ${#} -eq 2 ]] ; then
# Regular expressions for identifying POTCAR and CONTCAR files
PCAR_REGEX="[Pp][Oo][Tt][Cc][Aa][Rr]"
CCAR_REGEX="[Cc][Oo][Nn][Tt][Cc][Aa][Rr]"
# Attempt to identify POTCAR and CONTCAR argument ordering
if [[ ${1} =~ ${PCAR_REGEX} && ${2} =~ ${CCAR_REGEX} ]] ; then
POTCAR="${1}" ; CONTCAR="${2}" ; shift 2
else
POTCAR="${2}" ; CONTCAR="${1}" ; shift 2
fi
# Accept exactly two or zero command line arguments
elif [[ ${#} -ne 0 ]] ; then
echo "${PROG_NAME}: error: ${#}: Invalid argument count, expected [2|0]"
echo "$#"
exit ${FAILURE}
fi
Given this code, and running the application, I get the following output:
BEFORE
AFTER -- :hvso:l:d:n:p:t: -l help,version,submit-job,output:,library:,job-dir:,num-nodes:,num-procs:,max-time: -a -n vasp2nwchem --
vasp2nwchem: error: 7: Invalid argument count, expected [2|0]
:hvso:l:d:n:p:t: -l help,version,submit-job,output:,library:,job-dir:,num-nodes:,num-procs:,max-time: -a -n vasp2nwchem --
So, the code enters the while loop portion of the code, jumps to the last case, and shifts off the first --, leaving me with all of the arguments that I supplied to getopt, minus the -o flag.
Any light that anyone could shed on this conundrum would be immensely appreciated, because it is seriously about to send me over the edge, especially since this code was functional no less than thrity minutes ago, and has now stopped working entirely!!!
I don't see anything wrong. I have GNU getopt installed as /usr/gnu/bin/getopt (and BSD getopt in /usr/bin), so this script (chk.getopt.sh) is almost equivalent to the start of yours, though I do set the PROG_NAME variable. This is more or less the SSCCE (Short, Self-Contained, Complete Example) for your rather substantial script.
#!/bin/bash
PROG_NAME=$(basename $0 .sh)
SHORT_OPTS=":hvso:l:d:n:p:t:"
LONG_OPTS="help,version,submit-job,output:,library:,job-dir:"
LONG_OPTS="${LONG_OPTS},num-nodes:,num-procs:,max-time:"
OPTS=$(/usr/gnu/bin/getopt -o "${SHORT_OPTS}" -l "${LONG_OPTS}" -a -n "${PROG_NAME}" -- "$#")
# Check for invalid command line options and arguments
if [[ ${?} -ne ${SUCCESS} ]] ; then
echo -e "${PROG_NAME}: error: Invalid option or argument\n" >&2
usage ; exit ${FAILURE}
else
echo "BEFORE $#"
eval set -- ${OPTS}
echo "AFTER $#"
fi
When I run it, this is the output:
$ bash -x chk.getopt.sh -ooutput -nnumber -pperhaps -ppotato -- -o oliphaunt
++ basename chk.getopt.sh .sh
+ PROG_NAME=chk.getopt
+ SHORT_OPTS=:hvso:l:d:n:p:t:
+ LONG_OPTS=help,version,submit-job,output:,library:,job-dir:
+ LONG_OPTS=help,version,submit-job,output:,library:,job-dir:,num-nodes:,num-procs:,max-time:
++ /usr/gnu/bin/getopt -o :hvso:l:d:n:p:t: -l help,version,submit-job,output:,library:,job-dir:,num-nodes:,num-procs:,max-time: -a -n chk.getopt -- -ooutput -nnumber -pperhaps -ppotato -- -o oliphaunt
+ OPTS=' -o '\''output'\'' -n '\''number'\'' -p '\''perhaps'\'' -p '\''potato'\'' -- '\''-o'\'' '\''oliphaunt'\'''
+ [[ 0 -ne '' ]]
+ echo 'BEFORE -ooutput' -nnumber -pperhaps -ppotato -- -o oliphaunt
BEFORE -ooutput -nnumber -pperhaps -ppotato -- -o oliphaunt
+ eval set -- -o ''\''output'\''' -n ''\''number'\''' -p ''\''perhaps'\''' -p ''\''potato'\''' -- ''\''-o'\''' ''\''oliphaunt'\'''
++ set -- -o output -n number -p perhaps -p potato -- -o oliphaunt
+ echo 'AFTER -o' output -n number -p perhaps -p potato -- -o oliphaunt
AFTER -o output -n number -p perhaps -p potato -- -o oliphaunt
$
This all looks correct; the options before the double dash have been split from their arguments, and the ones after the double dash are OK.
So, it is not obvious that there's a problem with your code. Even with an empty string as the program name, it worked OK for me.
May be you should show the output of this much of the script on your machine?
I want to design a shell script as a wrapper for a couple of scripts. I would like to specify parameters for myshell.sh using getopts and pass the remaining parameters in the same order to the script specified.
If myshell.sh is executed like:
myshell.sh -h hostname -s test.sh -d waittime param1 param2 param3
myshell.sh param1 param2 -h hostname param3 -d waittime -s test.sh
myshell.sh param1 -h hostname -d waittime -s test.sh param2 param3
All of the above should be able to call as
test.sh param1 param2 param3
Is it possible to utilize the options parameters in the myshell.sh and post remaining parameters to underlying script?
I wanted to do something similar to the OP, and I found the relevant information I required here and here
Essentially if you want to do something like:
script.sh [options] ARG1 ARG2
Then get your options like this:
while getopts "h:u:p:d:" flag; do
case "$flag" in
h) HOSTNAME=$OPTARG;;
u) USERNAME=$OPTARG;;
p) PASSWORD=$OPTARG;;
d) DATABASE=$OPTARG;;
esac
done
And then you can get your positional arguments like this:
ARG1=${#:$OPTIND:1}
ARG2=${#:$OPTIND+1:1}
More information and details are available through the link above.
myshell.sh:
#!/bin/bash
script_args=()
while [ $OPTIND -le "$#" ]
do
if getopts h:d:s: option
then
case $option
in
h) host_name="$OPTARG";;
d) wait_time="$OPTARG";;
s) script="$OPTARG";;
esac
else
script_args+=("${!OPTIND}")
((OPTIND++))
fi
done
"$script" "${script_args[#]}"
test.sh:
#!/bin/bash
echo "$0 $#"
Testing the OP's cases:
$ PATH+=:. # Use the cases as written without prepending ./ to the scripts
$ myshell.sh -h hostname -s test.sh -d waittime param1 param2 param3
./test.sh param1 param2 param3
$ myshell.sh param1 param2 -h hostname param3 -d waittime -s test.sh
./test.sh param1 param2 param3
$ myshell.sh param1 -h hostname -d waittime -s test.sh param2 param3
./test.sh param1 param2 param3
What's going on:
getopts will fail if it encounters a positional parameter. If it's used as a loop condition, the loop would break prematurely whenever positional parameters appear before options, as they do in two of the test cases.
So instead, this loop breaks only once all parameters have been processed. If getopts doesn't recognize something, we just assume it's a positional parameter, and stuff it into an array while manually incrementing getopts's counter.
Possible improvements:
As written, the child script can't accept options (only positional parameters), since getopts in the wrapper script will eat those and print an error message, while treating any argument like a positional parameter:
$ myshell.sh param1 param2 -h hostname -d waittime -s test.sh -a opt1 param3
./myshell.sh: illegal option -- a
./test.sh param1 param2 opt1 param3
If we know the child script can only accept positional parameters, then myshell.sh should probably halt on an unrecognized option. That could be as simple as adding a default last case at the end of the case block:
\?) exit 1;;
$ myshell.sh param1 param2 -h hostname -d waittime -s test.sh -a opt1 param3
./myshell.sh: illegal option -- a
If the child script needs to accept options (as long as they don't collide with the options in myshell.sh), we could switch getopts to silent error reporting by prepending a colon to the option string:
if getopts :h:d:s: option
Then we'd use the default last case to stuff any unrecognized option into script_args:
\?) script_args+=("-$OPTARG");;
$ myshell.sh param1 param2 -h hostname -d waittime -s test.sh -a opt1 param3
./test.sh param1 param2 -a opt1 param3
Mix opts and args :
ARGS=""
echo "options :"
while [ $# -gt 0 ]
do
unset OPTIND
unset OPTARG
while getopts as:c: options
do
case $options in
a) echo "option a no optarg"
;;
s) serveur="$OPTARG"
echo "option s = $serveur"
;;
c) cible="$OPTARG"
echo "option c = $cible"
;;
esac
done
shift $((OPTIND-1))
ARGS="${ARGS} $1 "
shift
done
echo "ARGS : $ARGS"
exit 1
Result:
bash test.sh -a arg1 arg2 -s serveur -c cible arg3
options :
option a no optarg
option s = serveur
option c = cible
ARGS : arg1 arg2 arg3
getopts won't parse the mix of param1 and -n options.
It is much better to put param1-3 into options like others.
Furthermore you can use already existing libraries such as shflags. It is pretty smart and it is easy to use.
And the last way is to write your own function to parse params without getopts, just iterating all params through case construction. It is the hardest way but it is the only way to match your expectations exactly.
I thought up one way that getopts can be extended to truly mix options and positional parameters. The idea is to alternate between calling getopts and assigning any positional parameters found to n1, n2, n3, etc.:
parse_args() {
_parse_args 1 "$#"
}
_parse_args() {
local n="$1"
shift
local options_func="$1"
shift
local OPTIND
"$options_func" "$#"
shift $(( OPTIND - 1 ))
if [ $# -gt 0 ]; then
eval test -n \${n$n+x}
if [ $? -eq 0 ]; then
eval n$n="\$1"
fi
shift
_parse_args $(( n + 1 )) "$options_func" "$#"
fi
}
Then in the OP's case, you could use it like:
main() {
local n1='' n2='' n3=''
local duration hostname script
parse_args parse_main_options "$#"
echo "n1 = $n1"
echo "n2 = $n2"
echo "n3 = $n3"
echo "duration = $duration"
echo "hostname = $hostname"
echo "script = $script"
}
parse_main_options() {
while getopts d:h:s: opt; do
case "$opt" in
d) duration="$OPTARG" ;;
h) hostname="$OPTARG" ;;
s) script="$OPTARG" ;;
esac
done
}
main "$#"
Running it shows the output:
$ myshell.sh param1 param2 -h hostname param3 -d waittime -s test.sh
n1 = param1
n2 = param2
n3 = param3
duration = waittime
hostname = hostname
script = test.sh
Just a proof of concept, but maybe it's useful to someone.
Note: there's a gotcha if one function that uses parse_args calls another function that uses parse_args and the outer function declares e.g. local n4='', but the inner one doesn't and 4 or more positional parameters are passed to the inner function
Just mashed up a quickie, which easily handles a mixture of options and positional-parameters (leaving only positional-params in $#):
#!/bin/bash
while [ ${#} -gt 0 ];do OPTERR=0;OPTIND=1;getopts "p:o:hvu" arg;case "$arg" in
p) echo "Path: [$OPTARG]" ;;
o) echo "Output: [$OPTARG]" ;;
h) echo "Help" ;;
v) echo "Version" ;;
\?) SET+=("$1") ;;
*) echo "Coding error: '-$arg' is not handled by case">&2 ;;
esac;shift;[ "" != "$OPTARG" ] && shift;done
[ ${#SET[#]} -gt 0 ] && set "" "${SET[#]}" && shift
echo -e "=========\nLeftover (positional) parameters (count=$#) are:"
for i in `seq $#`;do echo -e "\t$i> [${!i}]";done
Sample output:
[root#hots:~]$ ./test.sh 'aa bb' -h -v -u -q 'cc dd' -p 'ee ff' 'gg hh' -o ooo
Help
Version
Coding error: '-u' is not handled by case
Path: [ee ff]
Output: [ooo]
=========
Leftover (positional) parameters (count=4) are:
1> [aa bb]
2> [-q]
3> [cc dd]
4> [gg hh]
[root#hots:~]$
Instead of using getopts, you can directly implement your own bash argument parser. Take this as a working example. It can handle simultaneously name and position arguments.
#!/bin/bash
function parse_command_line() {
local named_options;
local parsed_positional_arguments;
yes_to_all_questions="";
parsed_positional_arguments=0;
named_options=(
"-y" "--yes"
"-n" "--no"
"-h" "--help"
"-s" "--skip"
"-v" "--version"
);
function validateduplicateoptions() {
local item;
local variabletoset;
local namedargument;
local argumentvalue;
variabletoset="${1}";
namedargument="${2}";
argumentvalue="${3}";
if [[ -z "${namedargument}" ]]; then
printf "Error: Missing command line option for named argument '%s', got '%s'...\\n" "${variabletoset}" "${argumentvalue}";
exit 1;
fi;
for item in "${named_options[#]}";
do
if [[ "${item}" == "${argumentvalue}" ]]; then
printf "Warning: Named argument '%s' got possible invalid option '%s'...\\n" "${namedargument}" "${argumentvalue}";
exit 1;
fi;
done;
if [[ -n "${!variabletoset}" ]]; then
printf "Warning: Overriding the named argument '%s=%s' with '%s'...\\n" "${namedargument}" "${!variabletoset}" "${argumentvalue}";
else
printf "Setting '%s' named argument '%s=%s'...\\n" "${thing_name}" "${namedargument}" "${argumentvalue}";
fi;
eval "${variabletoset}='${argumentvalue}'";
}
# https://stackoverflow.com/questions/2210349/test-whether-string-is-a-valid-integer
function validateintegeroption() {
local namedargument;
local argumentvalue;
namedargument="${1}";
argumentvalue="${2}";
if [[ -z "${2}" ]];
then
argumentvalue="${1}";
fi;
if [[ -n "$(printf "%s" "${argumentvalue}" | sed s/[0-9]//g)" ]];
then
if [[ -z "${2}" ]];
then
printf "Error: The %s positional argument requires a integer, but it got '%s'...\\n" "${parsed_positional_arguments}" "${argumentvalue}";
else
printf "Error: The named argument '%s' requires a integer, but it got '%s'...\\n" "${namedargument}" "${argumentvalue}";
fi;
exit 1;
fi;
}
function validateposisionaloption() {
local variabletoset;
local argumentvalue;
variabletoset="${1}";
argumentvalue="${2}";
if [[ -n "${!variabletoset}" ]]; then
printf "Warning: Overriding the %s positional argument '%s=%s' with '%s'...\\n" "${parsed_positional_arguments}" "${variabletoset}" "${!variabletoset}" "${argumentvalue}";
else
printf "Setting the %s positional argument '%s=%s'...\\n" "${parsed_positional_arguments}" "${variabletoset}" "${argumentvalue}";
fi;
eval "${variabletoset}='${argumentvalue}'";
}
while [[ "${#}" -gt 0 ]];
do
case ${1} in
-y|--yes)
yes_to_all_questions="${1}";
printf "Named argument '%s' for yes to all questions was triggered.\\n" "${1}";
;;
-n|--no)
yes_to_all_questions="${1}";
printf "Named argument '%s' for no to all questions was triggered.\\n" "${1}";
;;
-h|--help)
printf "Print help here\\n";
exit 0;
;;
-s|--skip)
validateintegeroption "${1}" "${2}";
validateduplicateoptions g_installation_model_skip_commands "${1}" "${2}";
shift;
;;
-v|--version)
validateduplicateoptions branch_or_tag "${1}" "${2}";
shift;
;;
*)
parsed_positional_arguments=$((parsed_positional_arguments+1));
case ${parsed_positional_arguments} in
1)
validateposisionaloption branch_or_tag "${1}";
;;
2)
validateintegeroption "${1}";
validateposisionaloption g_installation_model_skip_commands "${1}";
;;
*)
printf "ERROR: Extra positional command line argument '%s' found.\\n" "${1}";
exit 1;
;;
esac;
;;
esac;
shift;
done;
if [[ -z "${g_installation_model_skip_commands}" ]];
then
g_installation_model_skip_commands="0";
fi;
}
You would call this function as:
#!/bin/bash
source ./function_file.sh;
parse_command_line "${#}";
Usage example:
./test.sh as 22 -s 3
Setting the 1 positional argument 'branch_or_tag=as'...
Setting the 2 positional argument 'skip_commands=22'...
Warning: Overriding the named argument '-s=22' with '3'...
References:
example_installation_model.sh.md
Checking for the correct number of arguments
https://unix.stackexchange.com/questions/129391/passing-named-arguments-to-shell-scripts
An example of how to use getopts in bash
There are some standards for unix option processing, and in shell programming, getopts is the best way of enforcing them. Almost any modern language (perl, python) has a variant on getopts.
This is just a quick example:
command [ options ] [--] [ words ]
Each option must start with a dash, -, and must consist of a single character.
The GNU project introduced Long Options, starting with two dashes --,
followed by a whole word, --long_option. The AST KSH project has a getopts that also supports long options, and long options starting with a single dash, -, as in find(1) .
Options may or may not expect arguments.
Any word not starting with a dash, -, will end option processing.
The string -- must be skipped and will end option processing.
Any remaining arguments are left as positional parameters.
The Open Group has a section on Utility Argument Syntax
Eric Raymond's The Art of Unix Programming has a chapter on traditional unix choices for option letters and their meaning.
You can try this trick: after while loop with optargs, just use this snippet
#shift away all the options so that only positional agruments
#remain in $#
for (( i=0; i<OPTIND-1; i++)); do
shift
done
POSITIONAL="$#"
However, this approach has a bug:
all the options after the first positional argument are ingored by getopts and are considered as positional arguments - event those that are correct (see sample output: -m and -c are among positional arguments)
Maybe it has even more bugs...
Look at the whole example:
while getopts :abc opt; do
case $opt in
a)
echo found: -a
;;
b)
echo found: -b
;;
c)
echo found: -c
;;
\?) echo found bad option: -$OPTARG
;;
esac
done
#OPTIND-1 now points to the first arguments not beginning with -
#shift away all the options so that only positional agruments
#remain in $#
for (( i=0; i<OPTIND-1; i++)); do
shift
done
POSITIONAL="$#"
echo "positional: $POSITIONAL"
Output:
[root#host ~]# ./abc.sh -abc -de -fgh -bca haha blabla -m -c
found: -a
found: -b
found: -c
found bad option: -d
found bad option: -e
found bad option: -f
found bad option: -g
found bad option: -h
found: -b
found: -c
found: -a
positional: haha blabla -m -c