why the blackslash is not url encoded in this shell script? - shell

Question

I am trying to the url encode a string based on shell scripting.
I have downloaded a script from internet.
it is:
#!/bin/sh
url_encoder()
{
echo -n "$1" | awk -v ORS="" '{ gsub(/./,"&\n") ; print }' | while read l;
do
case "$l" in
[-_.~/a-zA-Z0-9] ) echo -n ${l} ;;
"" ) echo -n %20 ;;
* ) printf '%%%02X' "'$l"
esac
done
}
echo ""
}
The basic idea of the above codes is to
(1) convert a input string into the rows, each row has one character
(2) for each row, url encode the character
So If I run
$url_encoder "abc:"
the output would be "abc%3A", which is correct
But if I run
$url_encoder "\\" # I want to encode the backslash, so I use 2 "\" here
there is no output at all.
Do you know the reason why?

Answer 1

no need to use read which is slow, variable expansion can do a substring, no need to handle the space character specially, it can be handled as the default
url_encoder() {
local i str=$1 c
for ((i=0;i<${#str};i+=1)); do
c=${str:i:1}
case "$c" in
[-_.~/a-zA-Z0-9] ) echo -n "${c}" ;;
* ) printf '%%%02X' "'$c" ;;
esac
done
}
l='\'
printf '%%%02X' "'$l"
The reason why the backslash disapears is because it has a special meaning for read, -r option should be used to avoid.
https://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html#index-read
Note ~ should also be encoded http://www.rfc-editor.org/rfc/rfc1738.txt
printf argument starting with a quote (single or double), handles only ascii character "'$c" (<128).
url_encoder() { (
LC_ALL=C
str=$1
for ((i=0;i<${#str};i+=1)); do
c=${str:i:1}
if [[ $c = [-_./a-zA-Z0-9] ]]; then
echo -n "${c}"
elif [[ $c = [$'\1'-$'\x7f'] ]]; then
printf '%%%02X' "'$c"
else
printf '%%%s' $(echo -n "$c" | od -An -tx1)
fi
done
)}

Answer 2

Nahuel Fouilleul's helpful answer explains the problem with your approach (-r is missing from your read command, resulting in unwanted interpretation of \ chars.) and offers a more efficient bash solution.
Here's a more efficient, POSIX-compliant solution (sh-compatible) that performs the encoding with a single awk command, assuming that the input string is composed only of characters in the ASCII/Unicode code-point range between 32 and 127, inclusively:
#!/bin/sh
url_encoder()
{
awk -v url="$1" -v ORS= 'BEGIN {
# Create lookup table that maps characters to their code points.
for(n=32;n<=127;n++) ord[sprintf("%c",n)]=n
# Process characters one by one, either passing them through, if they
# need no encoding, or converting them to their %-prefixed hex equivalent.
for(i=1;i<=length(url);++i) {
char = substr(url, i, 1)
if (char !~ "[-_.~/a-zA-Z0-9]") char = sprintf("%%%x", ord[char])
print char
}
printf "\n"
}'
}