bash and readline: tab completion in a user input loop? - bash

Question

I'm making a bash script which presents a command line to the user.
The cli code is as this:
#!/bin/bash
cmd1() {
echo $FUNCNAME: "$#"
}
cmd2() {
echo $FUNCNAME: "$#"
}
cmdN() {
echo $FUNCNAME: "$#"
}
__complete() {
echo $allowed_commands
}
shopt -qs extglob
fn_hide_prefix='__'
allowed_commands="$(declare -f | sed -ne '/^'$fn_hide_prefix'.* ()/!s/ ().*//p' | tr '\n' ' ')"
complete -D -W "this should output these words when you hit TAB"
echo "waiting for commands"
while read -ep"-> "; do
history -s $REPLY
case "$REPLY" in
#(${allowed_commands// /|})?(+([[:space:]])*)) $REPLY ;;
\?) __complete ;;
*) echo "invalid command: $REPLY" ;;
esac
done
Clarification: made and tested in Bash 4
So, "read -e" gives readline capabilities, i can recall commands, edit the input line, etc. What i cannot do in any way is to have readline's tab completion to work!!
I tried two things:
How it should be supposedly done: using the bash builtins "complete" and "compgen", which is reported to work here Update: it's not reported to work in scripts.
This ugly workaround
Why doesn't readline behave correctly when using "complete" inside the script? it works when i try it from bash in interactive mode...

Answer 1

After trying a custom completion script that I know works (I use it every day) and running into the same issue (when rigging it up similar to yours), I decided to snoop through the bash 4.1 source, and found this interesting block in bash-4.1/builtins/read.def:edit_line():
old_attempted_completion_function = rl_attempted_completion_function;
rl_attempted_completion_function = (rl_completion_func_t *)NULL;
if (itext)
{
old_startup_hook = rl_startup_hook;
rl_startup_hook = set_itext;
deftext = itext;
}
ret = readline (p);
rl_attempted_completion_function = old_attempted_completion_function;
old_attempted_completion_function = (rl_completion_func_t *)NULL;
It appears that before readline() is called, it resets the completion function to null for some reason that only a bash-hacking long beard might know. Thus, doing this with the read builtin may simply be hard-coded to be disabled.
EDIT: Some more on this: The wrapping code to stop completion in the read builtin occurred between bash-2.05a and bash-2.05b. I found this note in that version's bash-2.05b/CWRU/changelog file:
edit_line (called by read -e) now just does readline's filename completion by setting rl_attempted_completion_function to NULL, since e.g., doing command completion for the first word on the line wasn't really useful
I think it's a legacy oversight, and since programmable completion has come a long way, what you're doing is useful. Maybe you can ask them to add it back in, or just patch it yourself, if that'd be feasible for what you're doing.
Afraid I don't have a different solution aside from what you've come up with so far, but at least we know why it doesn't work with read.
EDIT2: Right, here's a patch I just tested that seems to "work". Passes all unit and reg tests, and shows this output from your script when run using the patched bash, as you expected:
$ ./tabcompl.sh
waiting for commands
-> **<TAB>**
TAB hit output should these this when words you
->
As you'll see, I just commented out those 4 lines and some timer code to reset the rl_attempted_completion_function when read -t is specified and a timeout occurs, which is no longer necessary. If you're going to send Chet something, you may wish to excise the entirety of the rl_attempted_completion_function junk first, but this will at least allow your script to behave properly.
Patch:
--- bash-4.1/builtins/read.def 2009-10-09 00:35:46.000000000 +0900
+++ bash-4.1-patched/builtins/read.def 2011-01-20 07:14:43.000000000 +0900
## -394,10 +394,12 ##
}
old_alrm = set_signal_handler (SIGALRM, sigalrm);
add_unwind_protect (reset_alarm, (char *)NULL);
+/*
#if defined (READLINE)
if (edit)
add_unwind_protect (reset_attempted_completion_function, (char *)NULL);
#endif
+*/
falarm (tmsec, tmusec);
}
## -914,8 +916,10 ##
if (bash_readline_initialized == 0)
initialize_readline ();
+/*
old_attempted_completion_function = rl_attempted_completion_function;
rl_attempted_completion_function = (rl_completion_func_t *)NULL;
+*/
if (itext)
{
old_startup_hook = rl_startup_hook;
## -923,8 +927,10 ##
deftext = itext;
}
ret = readline (p);
+/*
rl_attempted_completion_function = old_attempted_completion_function;
old_attempted_completion_function = (rl_completion_func_t *)NULL;
+*/
if (ret == 0)
return ret;
Keep in mind the patched bash would have to be distributed or made available somehow wherever people would be using your script...

Answer 2

I've been struggling with same issue for some time now and I think I have a solution that works, in my real world case I'm using compgen to generate possible completions. But here is an example that illustrates the core logic:
#!/bin/bash
set -o emacs;
tab() {
READLINE_LINE="foobar"
READLINE_POINT="${#READLINE_LINE}"
}
bind -x '"\t":"tab"';
read -ep "$ ";
Set the emacs option to enable key binding, bind the tab key to a function, change READLINE_LINE to update the line after the prompt, and set READLINE_POINT to reflect the line's new longer length.
In my use case I actually mimic the COMP_WORDS, COMP_CWORD and COMPREPLY variables but this should be sufficient to understand how to go about adding custom tab completion when using read -ep.
You must update READLINE_LINE to change the prompt line (completion single match), printing to stdin prints before the prompt as readline has put the terminal in raw mode and is capturing input.

Answer 3

Well, it seems i finally stumped on the answer, and it sadly is: actually there isn't full support for readline when interfacing it via "read -e".
The answer is given by the BASH maintainer, Chet Ramey. In this thread the exact same issue is addressed:
I'm writing a script with a command line interpreter and I can most things
working (eg. history etc.) except for one thing. The filename completion
works well for some of the commands, but I'd like to use other completion
options for others. Works well from the "real" command line, but I can't
get it to work properly in my "read -e, eval" loop..
You won't be able to do it. `read -e' uses only the readline default
completions.
Chet
So, unless i'm missing something //rant// while bash hands to the programmer the "read -e" mechanism as the mean for full, proper CLI user interfacing, the functionality is crippled, even though the underlying mechanism (readline) works and integrates with the rest of bash flawlessly //end rant//
I have exposed the question to the kind folks at #bash in freenode and been suggested to try with a Readline wrapper like rlfe or rlwrap.
Finally, i contacted Chet himself by mail yesterday, and he confirmed that this is by design, and that he doesn't feel like changing it as the only use case for programmable completion into "read", i.e. presenting a list of commands to the script user, doesn't look like a compelling reason to spend time working on this. Nevertheless he expressed that in case someone actually does the work he would certainly look at the result.
IMHO, not considering worth of the effort the ability to bring up a full CLI with just 5 lines of code, something one wish were possible in a lot of languages, is a mistake.
In this context, i think Simon's answer is brilliant and right in place. I'll try to follow your steps and perhaps with some luck i'll get more info. Been a while since i don't hack in C however, and i assume the amount of code i'll have to grasp to implement will not be trivial. But anyway i'll try.

Answer 4

I'm not sure if this exactly answers the OP question - but I was searching for which command could one use, to obtain the default bash tab completion of known executable commands (as per $PATH), as shown when pressing TAB. Since I was first led to this question (which I think is related), I thought I'd post a note here.
For instance, on my system, typing lua and then TAB gives:
$ lua<TAB>
lua lua5.1 luac luac5.1 lualatex luatex luatools
It turns out, there is a bash built-in (see #949006 Linux command to list all available commands and aliases), called compgen - and I can feed it with the same string lua as in the interactive case, and obtain the same results as if I pressed TAB:
$ compgen -c lua
luac
lua5.1
lua
luac5.1
luatex
lualatex
luatools
... and that is exactly what I was looking for :)
Hope this helps someone,
Cheers!

Answer 5

If you're going to that much effort, why not just add the cost of a fork or two and use something that is more than capable of providing everything you want. https://github.com/hanslub42/rlwrap
#!/bin/bash
which yum && yum install rlwrap
which zypper && zypper install rlwrap
which port && port install rlwrap
which apt-get && apt-get install rlwrap
REPLY=$( rlwrap -o cat )
Or as the man page puts it:
In a shell script, use rlwrap in ’one−shot’ mode as a replacement for read
order=$(rlwrap -p Yellow -S 'Your pizza? ' -H past_orders -P Margherita -o cat)