I would like to know which chars are valid shortopts for the GNU getopt, which is implemented in many different languages, like Bash or PHP.
I didn't find an official documentation where it is defined, or I have overread it.
I tested following Bash script:
#!/bin/bash
while getopts ":a%2" opt; do
case $opt in
a)
echo "-a was triggered!" >&2
;;
%)
echo "-% was triggered!" >&2
;;
2)
echo "-2 was triggered!" >&2
;;
\?)
echo "Invalid option: -$OPTARG" >&2
;;
esac
done
The result:
$ ./test -a%2x
-a was triggered!
-% was triggered!
-2 was triggered!
Invalid option: -x
So I can tell that at least -% and -2 are valid short opts, or the getopt implementation of Bash is not correct.
Which characters are excluded/illegal short opts? I know that ':' is excluded, obviously, and that '?' is defined as always illegal, therefore '-?' is mostly used to show an usage information page.
POSIX says:
All option characters allowed by Utility Syntax Guideline 3 are allowed in optstring. The implementation may accept other characters as an extension.
Guideline 3 says:
Each option name should be a single alphanumeric character (the alnum character classification) from the portable character set. The -W (capital-W) option shall be reserved for vendor options.
Multi-digit options should not be allowed.
In turn, the alnum class is defined as these characters:
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
a b c d e f g h i j k l m n o p q r s t u v w x y z
0 1 2 3 4 5 6 7 8 9
The Linux getopt(3) manpage says:
The use of '+' and '-' in optstring is a GNU extension.
So it accepts those as well. If others work for you, I guess that's good, but beware of making use of undocumented behaviour.
Related
In the dist_train.sh from mmdetection3d, what does ${#:3} do at the last line ?
I can't understand its bash grammar.
#!/usr/bin/env bash
CONFIG=$1
GPUS=$2
NNODES=${NNODES:-1}
NODE_RANK=${NODE_RANK:-0}
PORT=${PORT:-29500}
MASTER_ADDR=${MASTER_ADDR:-"127.0.0.1"}
PYTHONPATH="$(dirname $0)/..":$PYTHONPATH \
python -m torch.distributed.launch \
--nnodes=$NNODES \
--node_rank=$NODE_RANK \
--master_addr=$MASTER_ADDR \
--nproc_per_node=$GPUS \
--master_port=$PORT \
$(dirname "$0")/train.py \
$CONFIG \
--seed 0 \
--launcher pytorch ${#:3}
It is standard parameter expansion:
${parameter:offset}
${parameter:offset:length}
This is referred to as Substring Expansion. It expands to up to
length characters of the value of parameter starting at the character
specified by offset. If parameter is #, an indexed array
subscripted by # or *, or an associative array name, the
results differ as described below. If length is omitted, it expands
to the substring of the value of parameter starting at the character
specified by offset and extending to the end of the value. length
and offset are arithmetic expressions (see Shell Arithmetic).
[...]
If parameter is #, the result is length positional parameters
beginning at offset. A negative offset is taken relative to one
greater than the greatest positional parameter, so an offset of -1
evaluates to the last positional parameter. It is an expansion error
if length evaluates to a number less than zero.
The following examples illustrate substring expansion using positional parameters:
$ set -- 1 2 3 4 5 6 7 8 9 0 a b c d e f g h
$ echo ${#:7}
7 8 9 0 a b c d e f g h
$ echo ${#:7:0}
$ echo ${#:7:2}
7 8
$ echo ${#:7:-2}
bash: -2: substring expression < 0
$ echo ${#: -7:2}
b c
$ echo ${#:0}
./bash 1 2 3 4 5 6 7 8 9 0 a b c d e f g h
$ echo ${#:0:2}
./bash 1
$ echo ${#: -7:0}
Per the Bash Hackers wiki on the Positional Parameters syntax, the ${#:3} means any script argument starting at the third argument.
In other words, the ${#:3} syntax means "all arguments EXCEPT the first and second". A similar SO question exists from which you can infer the same conclusion.
A contrived example:
foo() {
echo "${#:3}"
}
foo a b c d e f g h i
# prints c d e f g h i
Great question.
In bash this is one kind of something called variable expansion. In this case the variable is $# representing all the parameters received by the program (or function), as a string.
Using the colon : means that you want to 'expand' $# to a subset of it's original string (ie. a substring).
So in this instance you're saying give me the string representing all the incoming parameters, but start from the 3rd one.
Taking count from file, say if count = 5, I want to print 5 variables. i.e. A B C D E.
If count = 2, Print 2 variables A B, etc.
I have tried using the ASCII values but couldn't go through it.
for i in {1..5}; do
count=5; a=0;
printf "\x$(printf %x '65+$a')";
count=count+1;
done
if count = 5, I want to print 5 variables. i.e. A B C D E. If count = 2, Print 2 variables A B, etc.
Here's a program that matches your style that does what you are looking for:
a=0
for i in {1..5}; do
printf "\x$(printf %x $(( 65 + a )) )";
a=$((a+1));
done
The first thing to note is that in order to do math in bash, you'll need to use the $(( )) operation. Above, you can see I replaced you '65+$a' with $(( 65 + a )) . That's the big news that you need to get math done.
There were a couple of other little issues, but you were stuck on the $(()) stuff so they weren't clear yet. Incidentally, the 'a' variable can be completely removed from the program to just use the 'i' variable like this:
for i in {1..5}; do
printf "\x$(printf %x $(( 64 + i )) )";
done
I had to change the constant to 64 since we are now counting starting at 1.
The {1..5} expression is a good short cut for 1 2 3 4 5, but you won't be able to put a variable into it. So, if you need to add a count variable back in, consider using the seq program instead like this:
count=$1
for i in $(seq 1 $count); do
printf "\x$(printf %x $(( 64 + i )) )";
done
Note that $() is different than the math operator $(()). $() runs a subcommand returning the results.
method 1: simple brace expansion
#!/bin/bash
# generate a lookup table
vars=( - $(echo {A..Z}) )
# use the elements
for i in {1..5}; do
echo ${vars[$i]}
done
{A..Z} generates 26 strings: A, B, ..., Z
which get stored in an array variable by vars=(...)
we prepend a - that we'll ignore
we can then do 1-based indexing into the array
limited to 26 variables (or whatever range we choose)
method 2: multiple brace expansion to generate arbitrary long variables
#!/bin/bash
if [[ ! $1 =~ ^[0-9]+$ ]]; then
echo "Usage: $0 count"
exit
fi
cmd='{A..Z}'
for (( i=$1; i>26; i=i/26 )); do
cmd="${A..Z}$cmd"
done
vars=( $(eval echo $cmd) )
for (( i=0; i<$1; i++ )); do
echo ${vars[$i]}
done
i/26 does integer division (throws away the remainder)
I'm lazy and generate "more than enough" variables rather than attempting to calculate how many is "exactly enough"
{a..b}{a..b}{a..b} becomes aaa aab aba abb baa bab bba bbb
using eval lets us do the brace expansion without knowing in advance how many sets are needed
Sample output:
$ mkvar.sh 10000 |fmt -64 | tail -5
ORY ORZ OSA OSB OSC OSD OSE OSF OSG OSH OSI OSJ OSK OSL OSM
OSN OSO OSP OSQ OSR OSS OST OSU OSV OSW OSX OSY OSZ OTA OTB
OTC OTD OTE OTF OTG OTH OTI OTJ OTK OTL OTM OTN OTO OTP OTQ
OTR OTS OTT OTU OTV OTW OTX OTY OTZ OUA OUB OUC OUD OUE OUF
OUG OUH OUI OUJ OUK OUL OUM OUN OUO OUP
There is a shell script (bash) that check a csv file for lines that don't match a pattern and send a mail with the wrong lines. Thats works fine but while combine the wrong lines linux give a \r as line break, in the E-Mail there is no linebreak. So I try to send \r\n as line break but this has no effect, perl or bash delete this \n newline.
Here is a minimal working script as example:
SUBJECT="Error while parse CSV"
TO="rcpt#domain.tld"
wrongLines=$(perl -ne 'print "Row $.: $_\r\n" if not /^00[1-9]\d{4,}$/' $file)
MESSAGE="Error while parse following Lines, pattern dont match: \r\n $wrongLines"
echo $MESSAGE |od -c
The output of od is:
0000000 E r r o r w h i l e p a r s
0000020 e f o l l o w i n g L i n e
0000040 s , p a t t e r n d o n t
0000060 m a t c h : \ r \ n R o w
0000100 2 : 4 9 2 7 8 3 8 7 4 3 \r R
0000120 o w 3 : 4 8 2 3 2 8 9 7 3 8
0000140 \r \n
0000143
But what is the reason that in the od output the \n between the rows is deleted? I also try \x0D\x0A instead of \r\n but this also don't help. Any suggestions?
Your problem is that you're not using quotes!
Look:
$ a="A multi-line
input
variable"
$ echo $a
A multi-line input variable
$ echo "$a"
A multi-line
input
variable
$
Without quotes, you'll be victim of word splitting and filename expansion (not illustrated in the example above).
Also, adding \r or \n (that is, verbatim backslash followed by r or n) is not going to help at all.
Conclusion: Quote every variable expansion! always! (unless you really mean a glob pattern — in which case you will also add a comment in the code to explain why you purposely didn't quote the expansion).
Side note: don't use upper case variable names!
It is recommended you use lower-case names for your own parameters so as not to confuse them with the all-uppercase variable names used by Bash internal variables and environment variables.
What I'm after is to have the most compact expression that expands the special parameter # with an offset of 2 or else to a default value of foobar if the subscript expands to the empty string or null. I tried the following notations but without luck:
"$#:2:-foobar"
"${#:2:-foobar}"
"${#:2: -foobar}"
Is there such a compact notation? Alternatively what would be a similar solution; ideally without temporary variables?
You may combine the expansion for the second parameter or its default value, followed by the expansion from the next offset.
Assuming your array is the program or function's argument array $# then,
#!/bin/bash
echo A "${#:2}"
echo B "${#:2:}" # your attempt #1
echo C "${#:2-foobar}" # your attempt #2
echo D "${#:2: -foobar}" # your attempt #3
echo E "${2:-foobar}"
echo F "$1" "${2:-foobar}" ${#:3}
G=("$1" "${2:-foobar}" ${#:3})
echo G "${G[#]}"
Will yield the desired result for line F and G (G uses a temp variable though).
Ex:
$ bash expand.sh 1
A
B
C
D
E foobar
F 1 foobar
G 1 foobar
$ bash expand.sh 1 2 3 4
A 2 3 4
B
C 2 3 4
D
E 2
F 1 2 3 4
G 1 2 3 4
If you're trying to do this with a different array than "$#", say H=(1 2 3), providing defaults to index expansions ("${H[2]:-foobar}") doesn't seem to work. Your best bet in this case, assuming you don't want to introduce temporary variables is to use a function or eval. But at that point you might be better off just adding a conditional e.g.,
# assuming that H wasn't sparse. redefine H based on its values
H=(
"${H[0]}"
$([[ -n "${H[1]}" ]] && echo "${H[1]}" || echo "foobar")
${H[#]:2}
)
But, readability will suffer.
I'm having issues declaring all printable characters in an array in a bash script. I’d like to display all printable characters through a loop 4 times.
Example
array=( a b c d … z A B C … Z 1 2 3 … 0 ! # # $ % ^ & * ( ) _ +)
For chr1 in ${array[#]}
Do
For chr2 in ${array[#]}
Do
Echo $chr1$chr2
Done
Done
I've been able to get the space character to print with using ${array[value of space]} but I still haven't been able to get the * character to print. It tends to print a list of files for some reason.
Any idea how I can get this to work?
Quotes! More quotes!
array=( a b c d z A B C Z 1 2 3 0 '!' '#' '#' '$' '%' '^' '&' '*' '(' ')' '_' '+')
for chr1 in "${array[#]}"
do
for chr2 in "${array[#]}"
do
echo "$chr1$chr2"
done
done
Slap quotes around the special characters in your array declaration, and slap double quotes around the variable accesses in the loops.
In shell scripts, quoting is your friend. Always.
array=(a b c d e f g h i j k l m z A B C Z 1 2 3 0 \! \# \# \$ \% \^ \& \* \( \) _ +)
for chr1 in "${array[#]}"; do
for chr2 in "${array[#]}"; do
echo "$chr1$chr2"
done
done
Works fine here.
chr () { printf "\\$(($1/64*100+$1%64/8*10+$1%8))"; }
ord () { printf '%s' "$(( ( 256 + $(printf '%d' "'$1"))%256 ))"; }
for i in {32..126}
do
for j in {32..126}
do
chr $i; chr $j
echo
done
done
the * is a special character to bash.
wild card [asterisk]. The * character serves as a "wild card" for filename expansion in globbing. By itself, it matches every filename in a given directory.
From the absolute bash scripting guide[1], you'll want to escape it like the first answer does.
[1] http://tldp.org/LDP/abs/html/special-chars.html