Fixing POSIX sh warning in a small Bash program - bash

I wrote the following code in Bash:
#!/bin/sh
host=$1
regex="^(((git|ssh|http(s)?)|(git#[\w\.]+))(:(\/\/)?)([A-Za-z0-9.#:_/-]+)\.com)(.*)"
if [[ "$host" =~ $regex ]]; then
d=${BASH_REMATCH[1]}
if [[ "$d" = *github* ]]; then
return
fi
fi
die "Current repository is not stored in Github."
I want to learn how to write a better Bash code so I use the shellcheck.net.
Line 5:
if [[ "$host" =~ $regex ]]; then
^-- SC2039: In POSIX sh, [[ ]] is undefined.
Line 6:
d=${BASH_REMATCH[1]}
^-- SC2039: In POSIX sh, array references are undefined.
Line 7:
if [[ "$d" = *github* ]]; then
^-- SC2039: In POSIX sh, [[ ]] is undefined.
I'm trying to understand how to fix those warnings. I understand that in order to fix [[ ]] I need it to switch to [ ] but then I get an error due globs. Also how should I replace the =~ operator?

When you write #!/bin/sh then you shouldn't use bash-specific features like [[. But you don't need to change [[ to [ or anything like that; just change the shebang line to #!/bin/bash. Then you can use all the bash features you like.

Use grep and sed in posix.
# use grep -q to match with regex
if printf "%s\n" "$host" | grep -q '\(git\|ssh\|http\(s\)\)etc. etc. etc.'; then
# use sed to extract part of the string matching regex
d=$(printf "%s\n" "$host" | sed 's/\(g\|ssh\|http\(s\)\)etc. etc. etc./\2/')
if printf "%s\n" "$d" | grep -q github; then
return
fi
fi
Finding out proper regexes is left to others.

You could try to parse out the different parts with parameter expansions though it's going to get a bit tedious. (The link is to the Bash manual; only a few of the expansions supported by Bash are POSIX.)
Assuming the input is a valid, well-formed URL (which may or may not be warranted) maybe try
host=$1
tail=${1#*://*/}
case $tail in "$host") tail=${host#*/};; esac
case ${host%/$tail} in
*github.com) return ;;
esac
die "Current repository is not stored in Github."
(where of course we assume that this is in a context where return makes sense, and where die is defined separately, like we have to assume in the original code).
This is quite a lot simpler than the regex you presented, and definitely does not cover all the strings that the regex would be able to handle; but perhaps it doesn't have to be all that complex if we can assume that the URL has gone through some sort of validation (i.e. if it's the output from git remote it's pretty safe to assume that the user has verified it by other means already).

Related

how to match a specific file extension in shellscript

I looked some other posts and learnt to match file extension in the following way but why my code is not working? Thanks.
1 #!/bin/sh
2
3 for i in `ls`
4 do
5 if [[ "$i" == *.txt ]]
6 then
7 echo "$i is .txt file"
8 else
9 echo "$i is NOT .txt file"
10 fi
11 done
eidt:
I realized #!/bin/sh and #!/bin/bash are different, if you are looking at this post later, remember to check which one you are using.
The [[ ]] expression is only available in some shells, like bash and zsh. Some more basic shells, like dash, do no support it. I'm guessing you're running this on a recent version of Ubuntu or Debian, where /bin/sh is actually dash, and hence doesn't recognize [[. And actually, you shouldn't use [[ ]] with a #!/bin/sh shebang anyway, since it's unsafe to depend on a feature that the shebang doesn't request.
So, what to do about it? You'll have the [ ] type of test expression available, but it doesn't do pattern matching (like *.txt). There are a number of alternate ways to do it:
The case statement is available in even basic shells, and has the same pattern matching capability as [[ = ]]. This is the most common way to do this type of thing, especially when you have a list of different patterns to check against.
More indirectly, you can use ${var%pattern} to try remove .txt from the end of the end of the value (see "Remove Smallest Suffix Pattern" here), and then check to see if that changed the value:
if [ "$i" != "${i%.txt}" ]
More explanation: suppose $i is "file.txt"; then this expands to [ "file.txt" != "file" ], so they're not equal, and the test (for !=) succeeds. On the other hand, if $i is "file.pdf", then it expands to [ "file.pdf" != "file.pdf" ], which fails because the strings are the same.
Other notes: when using [ ], use a single equal sign for string comparison, and be sure to properly double-quote all variable references to avoid confusion. Also, if you use anything that has special meaning to the shell (like < or >), you need to quote or escape them.
You could use the expr command's : operator to do regular expression matching. (Regular expressions are a different type of pattern from the basic wildcard or "glob" expression.) You could do this, but don't.
#!/bin/sh
for i in `ls`
do
if [[ "$i" = *".txt" ]] ; then
echo "$i is .txt file"
else
echo "$i is NOT .txt file"
fi
done
You don't have to loop in ls output, and sh implementation might vary among OS distributions.
Consider:
#! /bin/sh
for i in *
do
if [[ "$i" == *.txt ]]
then
echo "$i is txt file"
else
echo "$i is NOT txt file"
fi
done

shell script how to compare file name with expected filename but different extention in single line

I have doubt in shell script
I will describe the scenario, $file is containing the file name of my interest,
consider $file can contain foo.1, foo.2, foo.3 here foo will be constant,
but .1,.2,.3 will change, i want to test this in single line in if statement something like
if [ $file = "foo.[1-9]" ]; then
echo "File name is $file"
fi'
i know above script doesn't work :) can anyone suggest what should i refer for this ?
Trim any extension, then see if it's "foo"?
base=${file%.[1-9]}
if [ "$base" = "foo" ]; then
echo Smashing success
fi
Equivalently, I always like to recommend case because it's portable and versatile.
case $file in
foo.[1-9] ) echo Smashing success ;;
esac
The syntax may seem weird at first but it's well worth knowing.
Both of these techniques should be portable to any Bourne-compatible shell, including Dash and POSIX sh.
Use [[ instead for regex matching.
if [[ $file =~ ^foo\.[1-9]$ ]] ; ...

Script to extract a part of string

ns/APAC_BankTransfers_Publish/CMB/services/svcPublishBankTransfers/flow.xml
In the above line and similar lines like this I want to extract whatever is present in between services and flow.xml and save it to a variable DIST.
The output should be svcPublishBankTransfers.
Using parameter expansion mechanisms available in POSIX sh:
s=ns/APAC_BankTransfers_Publish/CMB/services/svcPublishBankTransfers/flow.xml
s=${s%/flow.xml} # remove "/flow.xml"
s=${s##*/services/} # remove everything before "services"
echo "$s"
This has the advantages of being purely in-process (so faster than approaches that require piping through an external tool), and compatible with all POSIX shells (ash, dash, ksh, etc).
References:
BashFAQ #100 ("How do I do string manipulation in bash?")
BashFAQ #73 ("How can I use parameter expansion? How can I get substrings?")
Using BASH regex:
s='ns/APAC_BankTransfers_Publish/CMB/services/svcPublishBankTransfers/flow.xml'
[[ "$s" =~ /([^/]+)/[^/]*$ ]] && echo "${BASH_REMATCH[1]}"
svcPublishBankTransfers
OR else:
[[ "$s" =~ /services/([^/]+)/flow\.xml ]] && echo "${BASH_REMATCH[1]}"
svcPublishBankTransfers
bash$ echo "ns/APAC_BankTransfers_Publish/CMB/services/svcPublishBankTransfers/flow.xml" | cut -d '/' -f 5
svcPublishBankTransfers
bash$

Check execute command after cheking file type

I am working on a bash script which execute a command depending on the file type. I want to use the the "file" option and not the file extension to determine the type, but I am bloody new to this scripting stuff, so if someone can help me I would be very thankful! - Thanks!
Here the script I want to include the function:
#!/bin/bash
export PrintQueue="/root/xxx";
IFS=$'\n'
for PrintFile in $(/bin/ls -1 ${PrintQueue}) do
lpr -r ${PrintQueue}/${PrintFile};
done
The point is, all files which are PDFs should be printed with the lpr command, all others with ooffice -p
You are going through a lot of extra work. Here's the idiomatic code, I'll let the man page provide the explanation of the pieces:
#!/bin/sh
for path in /root/xxx/* ; do
case `file --brief $path` in
PDF*) cmd="lpr -r" ;;
*) cmd="ooffice -p" ;;
esac
eval $cmd \"$path\"
done
Some notable points:
using sh instead of bash increases portability and narrows the choices of how to do things
don't use ls when a glob pattern will do the same job with less hassle
the case statement has surprising power
First, two general shell programming issues:
Do not parse the output of ls. It's unreliable and completely useless. Use wildcards, they're easy and robust.
Always put double quotes around variable substitutions, e.g. "$PrintQueue/$PrintFile", not $PrintQueue/$PrintFile. If you leave the double quotes out, the shell performs wildcard expansion and word splitting on the value of the variable. Unless you know that's what you want, use double quotes. The same goes for command substitutions $(command).
Historically, implementations of file have had different output formats, intended for humans rather than parsing. Most modern implementations have an option to output a MIME type, which is easily parseable.
#!/bin/bash
print_queue="/root/xxx"
for file_to_print in "$print_queue"/*; do
case "$(file -i "$file_to_print")" in
application/pdf\;*|application/postscript\;*)
lpr -r "$file_to_print";;
application/vnd.oasis.opendocument.*)
ooffice -p "$file_to_print" &&
rm "$file_to_print";;
# and so on
*) echo 1>&2 "Warning: $file_to_print has an unrecognized format and was not printed";;
esac
done
#!/bin/bash
PRINTQ="/root/docs"
OLDIFS=$IFS
IFS=$(echo -en "\n\b")
for file in $(ls -1 $PRINTQ)
do
type=$(file --brief $file | awk '{print $1}')
if [ $type == "PDF" ]
then
echo "[*] printing $file with LPR"
lpr "$file"
else
echo "[*] printing $file with OPEN-OFFICE"
ooffice -p "$file"
fi
done
IFS=$OLDIFS

Checking if a file is readable and exists in one if condition: if [[ -r -f "/file.png" ]]

I was writing an if statement which checked if a file is readable and exists by doing the following:
if [[ -r "$upFN" && -f "$upFN" ]]; then
....
fi
Then I thought, surly you can make this smaller, something maybe like this:
if [[ -r -f "$upFN" ]]; then
....
fi
But this doesn't work, it returns errors:
./ftp.sh: line 72: syntax error in conditional expression
./ftp.sh: line 72: syntax error near `"$upFN"'
./ftp.sh: line 72: `if [[ -r -f "$upFN" ]]; then'
AFAICT, there is no way to combine them further. As a portability note, [[ expr ]] is less portable than [ expr ] or test expr. The C-style && and || are only included in bash so you might want to consider using the POSIX syntax of -a for and and -o for or. Personally, I prefer using test expr since it is very explicit. Many shells (bash included) include a builtin for it so you do not have to worry about process creation overhead.
In any case, I would rewrite your test as:
if test -r "$upFN" -a -f "$upFN"
then
...
fi
That syntax will work in traditional Bourne shell, Korn shell, and Bash. You can use the [ syntax portably just as well.
Is there ever a case where a file would be readable but it doesn't exist? Don't bother checking for existence when readability will tell you all you need.

Resources