bash shell text manipulation: I can extract a domain from a URL, how would I extend this to also exclude ".com" or ".co.uk" etc - bash

Question

"get a domain from a url" is quite a common question here on this site and the answer I have used for a long time is from this question:
How to extract domain name from url?
The most popular answer has a comment from user "sakumatto" which also handles sub-domains too, it is this:
echo http://www.test.example.com:3030/index.php | sed -e "s/[^/]*\/\/\([^#]*#\)\?\([^:/]*\).*/\2/" | awk -F. '{print $(NF-1) "." $NF}'
How would I further extend this command to exclude ".com" or ".co.uk" etc???
Insight:
I am writing a bash script for an amazing feature that Termux (Terminal emulator for Android) has, "termux-url-opener" that allows one to write a script that is launched when you use the native Android "share" feature, lets say i'm in the browser, github wants me to login, I press "share", then select "Termux", Termux opens and runs the script, echos the password to my clipboard and closes, now im automatically back in the browser with my password ready to paste!
Its very simple and uses pass (password-store) with pass-clip extension, gnupg and pinentry here is what I have so far which works fine, but currently its dumb (it would need me to continue writing if/elif statements for every password I have in pass) so I would like to automate things, all I need is to cut ".com" or ".co.uk" etc.
Here is my script so far:
#!/data/data/com.termux/files/usr/bin/bash
URL="$1"
WEBSITE=$(echo "$URL" | sed -e "s/[^/]*\/\/\([^#]*#\)\?\([^:/]*\).*/\2/" | awk -F. '{print $(NF-1) "." $NF}')
if [[ $WEBSITE =~ "github" ]]; then
# website contains "github"
pass -c github
elif [[ $WEBSITE =~ "codeberg" ]]; then
# website contains "codeberg"
pass -c codeberg
else
# is another app or website, so list all passwords entries.
pass clip --fzf
fi
As my pass password entries are just website names e.g "github" or "codeberg" if I could cut the ".com" or ".co.uk" from the end then I could add something like:
PASSWORDS=$(pass ls)
Now I can check if "$1" (my shared URL) is a listed within pass ls and this stops having to write:
elif [[ $WEBSITE =~ "codeberg" ]]; then
For every single entry in pass.
Thank you! its really appreciated!

Answer 1

i might be missing something, but why don't you just strip the offending TLDs from the hostname?
as in:
sed \
-e "s|[^/]*//\([^#]*#\)\?\([^:/]*\).*|\2|" \
-e 's|\.$||' \
-e 's|\.com$||' \
-e 's|\.co\.[a-zA-Z]*$||' \
-e 's|.*\.\([^.]*\.[^.]*\)|\1|'
"s|[^/]*//\([^#]*#\)\?\([^:/]*\).*|\2|" - this is your original regex, but using | as delimiter rather than / (gives you less quoting)
's|\.$||' - drop any accidently trailing dot (example.com. is a valid hostname!)
's|\.com$||' - remove trailing .com
's|\.co\.[a-zA-Z]*$||' - remove trailing .co.uk, .co.nl,...
's|.*\.\([^.]*\.[^.]*\)|\1|' - remove all components from the hostname except for the last two (this is basically your awk-script)

Answer 2

How about doing it entirely within bash:
if [[ $WEBSITE =~ ^(.*)([.]co)[.][a-z]+$ || $WEBSITE =~ ^(.*)[.][a-z]+$ ]]
then
pass=${BASH_REMATCH[1]}
else
echo WARNING: Unexpected value for WEBSITE: $WEBSITE
pass=$WEBSITE # Fallback
fi
I used two clauses (for the .co case and for the other cases), because bash a regexp does not understand non-greedy matching (i.e. .*?).

Answer 3

I propose you to work around a very simple modification like this grep command add:
WEBSITE=$(echo $1 | grep -vE ".com|.uk" | sed -e "s/[^/]*\/\/\([^#]*#\)\?\([^:/]*\).*/\2/" | awk -F. '{print $(NF-1) "." $NF}')
test -z $WEBSITE && exit 1 # if empty (.com or .uk generates an empty variable)
$ cat > toto
WEBSITE=$(echo $1 | grep -vE ".com|.uk" | sed -e "s/[^/]*\/\/\([^#]*#\)\?\([^:/]*\).*/\2/" | awk -F. '{print $(NF-1) "." $NF}')
test -z $WEBSITE && exit 1
echo $WEBSITE
With an example:
$ bash toto http://www.google.fr
google.fr
$ bash toto http://www.google.com
$ bash toto http://www.google.uk
$ bash toto http://www.google.gertrude
google.gertrude
$ rm toto
$
I used .uk in my example so do not just copy/paste the line.