In a shell, I run following commands without problem,
ls -al
!ls
the second invocation to ls also list files with -al flag. However, when I put the above script to a bash script, complaints are thrown,
!ls, command not found.
how to realise the same effects in script?
You would need to turn on both command history and !-style history expansion in your script (both are off by default in non-interactive shells):
set -o history
set -o histexpand
The expanded command is also echoed to standard error, just like in an interactive shell. You can prevent that by turning on the histverify shell option (shopt -s histverify), but in a non-interactive shell, that seems to make the history expansion a null-op.
Well, I wanted to have this working as well, and I have to tell everybody that the set -o history ; set -o histexpand method will not work in bash 4.x. It's not meant to be used there, anyway, since there are better ways to accomplish this.
First of all, a rather trivial example, just wanting to execute history in a script:
(bash 4.x or higher ONLY)
#!/bin/bash -i
history
Short answer: it works!!
The spanking new -i option stands for interactive, and history will work. But for what purpose?
Quoting Michael H.'s comment from the OP:
"Although you can enable this, this is bad programming practice. It will make your scripts (...) hard to understand. There is a reason it is disabled by default. Why do you want to do this?"
Yes, why? What is the deeper sense of this?
Well, THERE IS, which I'm going to demonstrate in the follow-up section.
My history buffer has grown HUGE, while some of those lines are script one-liners, which I really would not want to retype every time. But sometimes, I also want to alter these lines a little, because I probably want to give a third parameter, whereas I had only needed two in total before.
So here's an ideal way of using the bash 4.0+ feature to invoke history:
$ history
(...)
<lots of lines>
(...)
1234 while IFS='whatever' read [[ $whatever -lt max ]]; do ... ; done < <(workfile.fil)
<25 more lines>
So 1234 from history is exactly the line we want. Surely, we could take the mouse and move there, chucking the whole line in the primary buffer? But we're on *NIX, so why can't we make our life a bit easier?
This is why I wrote the little script below. Again, this is for bash 4.0+ ONLY (but might be adapted for bash 3.x and older with the aforementioned set -o ... stuff...)
#!/bin/bash -i
[[ $1 == "" ]] || history | grep "^\s*$1" |
awk '{for (i=2; i<=NF; i++) printf $i" "}' | tr '\n' '\0'
If you save this as xselauto.sh for example, you may invoke
$ ./xselauto.sh 1234
and the contents of history line #1234 will be in your primary buffer, ready for re-use!
Now if anyone still says "this has no purpose AFAICS" or "who'd ever be needing this feature?" - OK, I won't care. But I would no longer want to live without this feature, as I'm just too lazy to retype complex lines every time. And I wouldn't want to touch the mouse for each marked line from history either, TBH. This is what xsel was written for.
BTW, the tr part of the pipe is a dirty hack which will prevent the command from being executed. For "dangerous" commands, it is extremely important to always leave the user a way to look before he/she hits the Enter key to execute it. You may omit it, but ... you have been warned.
P.S. This scriptlet is in fact a workaround, simulating !1234 typed on a bash shell. As I could never make the ! work directly in a script (echo would never let me reveal the contents of history line 1234), I worked around the problem by simply greping for the line I wanted to copy.
History expansion is part of the interactive command-line editing features of a shell, not part of the scripting language. It's not generally available in the context of a script, only when interacting with a (pseudo-)human operator. (pseudo meaning that it can be made to work with things like expect or other keystroke repeating automation tools that generally try to play act a human, not implying that any particular operator might be sub-human or anything).
Related
There seems to be quite a lot of information on how to edit and execute a command using your editor using "edit-and-execute-command (C-x C-e)", but what I would like to achieve is take the current shell command, apply certain filtering (using a script) and then return it to prompt for further approval/manual changes before execution. Is this possible with bash?
Latest update based on my experience
The part 0"+y$dd in the following mapping is really something that you should carefully think about and tailor it to your taste/workflow/experience.
For instance, very frequently I've found myself ending up with multiple lines in the buffer, where I only want to execute the one the cursor is on; in this case I can use 0"+y$dd:%d<CR> instead of 0"+y$dd.
And this is just one of the possible scenarios.
Final answer for those who like vim
Set vim as your EDITOR/VISUAL, so that when editing a command line, you will use vim to edit it.
Put au BufEnter /tmp/bash-fc.* nn <Leader>d 0"+y$dd:wq<CR> in your ~/.vimrc file to map Leaderd (which you will rarely use when editing a command) to the action "delete the current line into the + register without the trailing EOL".
you can use either the + or the * register in the mapping above; the ways to paste into the terminal will likely differ; you need the +clipboard option for these registers to be available.
When finished editing a command in the vim editor, hit EscapeLeaderd.
Paste the clipboard into the terminal (this is terminal-dependent).
Original answer
I often need to do the same, and I do it as follows. (I normally use the set -o vi in bash, so points 1 and 2 in the following are different if you use set -o emacs, the default; based on your question it looks like points 1 and 2 are unified in Ctrl+x followed by Ctrl+e, which is harder to type, imho.)
hit Escape to be in normal mode,
hit v to enter the editor to edit the command,
edit the command as I like,
(This is where you ask the question.)
hit Escape0"+y$dd:wq,
Note: 0"+y$, not simply "+yy, as the latter would copy the newline too, and this would result in executing the command upon pasting it in the command line,
paste the clipboard on the command line
how to do this depends on the terminal you are using, I guess; I hit Ctrl+Alt+v in URxvt.
proceed to approval/manual edit.
Clearly this is just a workaround, consisting in copying the edited command into the clipboard before deleting the whole command, so that nothing gets executed upon exiting the editor; however it's the best I can get for myself.
Update
As my EDITOR (and VISUAL) is equal to vim, when I edit the command, I edit it in vim.
In this respect, I have noticed that the buffer is named /tmp/bash-fc.random, where random is a 6-characters alphanumeric random string.
This gives space to a lot of possiblities, if you use vim as your editor, as you can define some mapping in your .vimrc to execute the whole sequence Escape0"+y$dd:wq. For instance, one command that you'd rarely use when editing a command line is Leaderd; therefore you can put the following mapping in your .vimrc file
au BufEnter /tmp/bash-fc.* nn <Leader>d 0"+y$dd:wq<CR>
so that step 4 in the above recipe becomes
hit EscapeLeaderd
It's not possible to do that in Bash/readline but it's possible in zsh
using edit-command-line command:
darkstar% autoload edit-command-line; zle -N edit-command-line
darkstar% bindkey "^X^E" edit-command-line
Now press Control-x Control-e to open your editor, edit line, leave the editor - you will see the updated command line but it will not be executed automatically.
Now that I think about it, maybe a variation of what #kenorb suggested in a comment is the best workaround (as it seems no solution exists), if we want to stick to bash.
What you can do is prepend a # (the comment character in bash) to the command, rather than echo. Then when you exit the editor, the command will be ineffective, and you will only have to press arrow up (or k, if you use set -o vi), remove the # and confirming.
Note that this strategy adds just a few keystrokes, so it can be fairly efficient, depending on your typing level.
These pieces might get you closer:
a) replace the the normal binding for newline newline (ctrl-M)
bind -x '"\C-M":date"
b) grab the current line from the history using !#
replace date with whatever script you want.
c) edit manually
d) if necessary, somehow tie in !!:p which prints the new command to the command line but does not execute it, thus letting you manually edit it.
e) using ctrl-J submit edited command rather than a newline
or they might not ....
There is an option in bash to modify command from history without executing it. I'm not sure it it's possible to use script for this, doesn't seem to be likely. Although, you can make modifications using history modifiers.
Enable option histverify to prevent execution of modified command
Use chain of modifiers to change last command
Use "!!" to put your result to command line for final edit
Here is how it looks:
$ shopt -s histverify
$ ls *.sh
script1.sh script2.sh script3.sh script-return.sh
$ !!:s/*/script1/:p
ls script1.sh
$ !!:s/1/2/:p
ls script2.sh
$ !!
$ ls script2.sh
script2.sh
I'd like to point you to the Composure framework for Bash (I'm not affiliated with it): https://github.com/erichs/composure
It provides draft and revise functions that sound like they could help with what you're trying to do. Here's a (long) quote from the project's readme file:
Composure helps by letting you quickly draft simple shell functions,
breaking down your long pipe filters and complex commands into
readable and reusable chunks.
Draft first, ask questions later
Once you've crafted your gem of a command, don't throw it away! Use
draft () and give it a good name. This stores your last command as a
function you can reuse later. Think of it like a rough draft.
$ cat servers.txt
bashful: up
doc: down
up-arrow
$ cat servers.txt | grep down
doc: down
$ draft finddown
$ finddown | mail -s "down server(s)" admin#here.com
Revise, revise, revise!
Now that you've got a minimal shell function, you may want to make it
better through refactoring and revision. Use the revise () command
to revise your shell function in your favorite editor.
generalize functions with input parameters
add or remove functionality
add supporting metadata for documentation
$ revise finddown
finddown ()
{
about finds servers marked 'down' in text file
group admin
cat $1 | grep down
}
$ finddown servers.txt
doc: down
It does not seem possible with a keyboard shortcut, at least:
$ bind -P | grep -e command -e edit
complete-command can be found on "\e!".
edit-and-execute-command can be found on "\C-x\C-e".
emacs-editing-mode is not bound to any keys
possible-command-completions can be found on "\C-x!".
vi-editing-mode is not bound to any keys
This can be done in native bash using readline specifically READLINE_LINE and READLINE_POINT variables. I use this functionality all the time though not through vim, you would need to get the value of $selected from your vim command and if not empty it takes your original line + your input and replaces your original line with the combination without executing. output as a variable
_main() {
selected="$(__coms_select__ "$#")"
origonal_text=$READLINE_LINE READLINE_LINE="${READLINE_LINE:0:$READLINE_POINT}$selected${READLINE_LINE:$READLINE_POINT}"
READLINE_POINT=$(( READLINE_POINT + ${#selected} ))
}
bind -m emacs-standard -x '"\C-e": _main '
bind -m vi-command -x '"\C-e": _main '
bind -m vi-insert -x '"\C-e": _main '
Edit
Just remembered these two utilities that will let you do this as well.
Vipe allows you to run your editor in the middle of a unix pipeline and edit the data that is being piped between programs.
vp, up, vipe, Neomux (upgrade of nvim terminal) you can do some pretty neat throwing buffers between the terminal and split window.
and Athame (full vim on the command line)
https://github.com/ardagnir/athame
careful with that one though plugins work on the cli and it can get funky if you got tons of plugins
This is sort of one of those things that I figured a lot of people would use a lot, but I can't seem to find any people who have written about this sort of thing.
I find that a lot of times I do a lot of iteration on a command-line one-liner and when I end up using it a lot, or anticipate wanting to use it in the future, or when it becomes cumbersome to work with in one line, it generally is a good idea to turn the one-liner into a shell script and stick it somewhere reasonable and easily accessible like ~/bin.
It's obviously too cumbersome to use any sort of roundabout method involving a text editor to get this done, and it's possible to simply do it on the shell, for instance in zsh typing
echo "#!/usr/bin/env sh" > ~/bin/command_from_history_number_523.sh && echo !523 >> ~/bin/command_from_history_number_523.sh
followed by pressing Tab to inject the !523rd command and somehow shoehorning it into an acceptable string to be saved.
This is particularly cumbersome and has at minimum three problems:
Does not work in bash as it does not complete the !523
Requires some manual inspection and string escapement
Requires too much typing such as the script name must be entered twice
So it looks like I need to do some meta shell scripting here.
I think a good solution would function under both bash and zsh, and it should probably work by taking two arguments, an integer for the history command number and a name for the shell script to poop out in a hardcoded directory which contains that one command. Furthermore, under bash, it appears that multi-line commands are treated as separate commands, but I'm willing to assume that we only care about one-liners here and I only use zsh anyway at this point.
The stumbling block here is that i think I'll still be running shell scripts through bash even when using zsh, so it won't likely then be able to parse zsh's history files. I may need to make this into two separate programs then.
Update: I agree with #Floris 's comment that direct use of the commands like !! would be helpful though I am not sure how to make this work. Suppose I have the usage be
mkscript command_number_24 !24
this is inadequate because mkscript will be receiving the expanded out contents of the 24th command. if the 24th command contains any file globs or somesuch they will have been expanded already. This is bad, and I basically want the contents of the history file, i.e. the raw command string. I guess this can be worked around by manually implementing those shortcuts in here. Or just screw it and just take an integer argument.
function mkscript() {
echo '#!/bin/bash' > ~/bin/$2
history -p '!'$1 >> ~/bin/$2
}
Only tested in Bash.
Update from OP: In zsh I can accomplish this with fc -l $2 $2
I'm making a bash script which presents a command line to the user.
The cli code is as this:
#!/bin/bash
cmd1() {
echo $FUNCNAME: "$#"
}
cmd2() {
echo $FUNCNAME: "$#"
}
cmdN() {
echo $FUNCNAME: "$#"
}
__complete() {
echo $allowed_commands
}
shopt -qs extglob
fn_hide_prefix='__'
allowed_commands="$(declare -f | sed -ne '/^'$fn_hide_prefix'.* ()/!s/ ().*//p' | tr '\n' ' ')"
complete -D -W "this should output these words when you hit TAB"
echo "waiting for commands"
while read -ep"-> "; do
history -s $REPLY
case "$REPLY" in
#(${allowed_commands// /|})?(+([[:space:]])*)) $REPLY ;;
\?) __complete ;;
*) echo "invalid command: $REPLY" ;;
esac
done
Clarification: made and tested in Bash 4
So, "read -e" gives readline capabilities, i can recall commands, edit the input line, etc. What i cannot do in any way is to have readline's tab completion to work!!
I tried two things:
How it should be supposedly done: using the bash builtins "complete" and "compgen", which is reported to work here Update: it's not reported to work in scripts.
This ugly workaround
Why doesn't readline behave correctly when using "complete" inside the script? it works when i try it from bash in interactive mode...
After trying a custom completion script that I know works (I use it every day) and running into the same issue (when rigging it up similar to yours), I decided to snoop through the bash 4.1 source, and found this interesting block in bash-4.1/builtins/read.def:edit_line():
old_attempted_completion_function = rl_attempted_completion_function;
rl_attempted_completion_function = (rl_completion_func_t *)NULL;
if (itext)
{
old_startup_hook = rl_startup_hook;
rl_startup_hook = set_itext;
deftext = itext;
}
ret = readline (p);
rl_attempted_completion_function = old_attempted_completion_function;
old_attempted_completion_function = (rl_completion_func_t *)NULL;
It appears that before readline() is called, it resets the completion function to null for some reason that only a bash-hacking long beard might know. Thus, doing this with the read builtin may simply be hard-coded to be disabled.
EDIT: Some more on this: The wrapping code to stop completion in the read builtin occurred between bash-2.05a and bash-2.05b. I found this note in that version's bash-2.05b/CWRU/changelog file:
edit_line (called by read -e) now just does readline's filename completion by setting rl_attempted_completion_function to NULL, since e.g., doing command completion for the first word on the line wasn't really useful
I think it's a legacy oversight, and since programmable completion has come a long way, what you're doing is useful. Maybe you can ask them to add it back in, or just patch it yourself, if that'd be feasible for what you're doing.
Afraid I don't have a different solution aside from what you've come up with so far, but at least we know why it doesn't work with read.
EDIT2: Right, here's a patch I just tested that seems to "work". Passes all unit and reg tests, and shows this output from your script when run using the patched bash, as you expected:
$ ./tabcompl.sh
waiting for commands
-> **<TAB>**
TAB hit output should these this when words you
->
As you'll see, I just commented out those 4 lines and some timer code to reset the rl_attempted_completion_function when read -t is specified and a timeout occurs, which is no longer necessary. If you're going to send Chet something, you may wish to excise the entirety of the rl_attempted_completion_function junk first, but this will at least allow your script to behave properly.
Patch:
--- bash-4.1/builtins/read.def 2009-10-09 00:35:46.000000000 +0900
+++ bash-4.1-patched/builtins/read.def 2011-01-20 07:14:43.000000000 +0900
## -394,10 +394,12 ##
}
old_alrm = set_signal_handler (SIGALRM, sigalrm);
add_unwind_protect (reset_alarm, (char *)NULL);
+/*
#if defined (READLINE)
if (edit)
add_unwind_protect (reset_attempted_completion_function, (char *)NULL);
#endif
+*/
falarm (tmsec, tmusec);
}
## -914,8 +916,10 ##
if (bash_readline_initialized == 0)
initialize_readline ();
+/*
old_attempted_completion_function = rl_attempted_completion_function;
rl_attempted_completion_function = (rl_completion_func_t *)NULL;
+*/
if (itext)
{
old_startup_hook = rl_startup_hook;
## -923,8 +927,10 ##
deftext = itext;
}
ret = readline (p);
+/*
rl_attempted_completion_function = old_attempted_completion_function;
old_attempted_completion_function = (rl_completion_func_t *)NULL;
+*/
if (ret == 0)
return ret;
Keep in mind the patched bash would have to be distributed or made available somehow wherever people would be using your script...
I've been struggling with same issue for some time now and I think I have a solution that works, in my real world case I'm using compgen to generate possible completions. But here is an example that illustrates the core logic:
#!/bin/bash
set -o emacs;
tab() {
READLINE_LINE="foobar"
READLINE_POINT="${#READLINE_LINE}"
}
bind -x '"\t":"tab"';
read -ep "$ ";
Set the emacs option to enable key binding, bind the tab key to a function, change READLINE_LINE to update the line after the prompt, and set READLINE_POINT to reflect the line's new longer length.
In my use case I actually mimic the COMP_WORDS, COMP_CWORD and COMPREPLY variables but this should be sufficient to understand how to go about adding custom tab completion when using read -ep.
You must update READLINE_LINE to change the prompt line (completion single match), printing to stdin prints before the prompt as readline has put the terminal in raw mode and is capturing input.
Well, it seems i finally stumped on the answer, and it sadly is: actually there isn't full support for readline when interfacing it via "read -e".
The answer is given by the BASH maintainer, Chet Ramey. In this thread the exact same issue is addressed:
I'm writing a script with a command line interpreter and I can most things
working (eg. history etc.) except for one thing. The filename completion
works well for some of the commands, but I'd like to use other completion
options for others. Works well from the "real" command line, but I can't
get it to work properly in my "read -e, eval" loop..
You won't be able to do it. `read -e' uses only the readline default
completions.
Chet
So, unless i'm missing something //rant// while bash hands to the programmer the "read -e" mechanism as the mean for full, proper CLI user interfacing, the functionality is crippled, even though the underlying mechanism (readline) works and integrates with the rest of bash flawlessly //end rant//
I have exposed the question to the kind folks at #bash in freenode and been suggested to try with a Readline wrapper like rlfe or rlwrap.
Finally, i contacted Chet himself by mail yesterday, and he confirmed that this is by design, and that he doesn't feel like changing it as the only use case for programmable completion into "read", i.e. presenting a list of commands to the script user, doesn't look like a compelling reason to spend time working on this. Nevertheless he expressed that in case someone actually does the work he would certainly look at the result.
IMHO, not considering worth of the effort the ability to bring up a full CLI with just 5 lines of code, something one wish were possible in a lot of languages, is a mistake.
In this context, i think Simon's answer is brilliant and right in place. I'll try to follow your steps and perhaps with some luck i'll get more info. Been a while since i don't hack in C however, and i assume the amount of code i'll have to grasp to implement will not be trivial. But anyway i'll try.
I'm not sure if this exactly answers the OP question - but I was searching for which command could one use, to obtain the default bash tab completion of known executable commands (as per $PATH), as shown when pressing TAB. Since I was first led to this question (which I think is related), I thought I'd post a note here.
For instance, on my system, typing lua and then TAB gives:
$ lua<TAB>
lua lua5.1 luac luac5.1 lualatex luatex luatools
It turns out, there is a bash built-in (see #949006 Linux command to list all available commands and aliases), called compgen - and I can feed it with the same string lua as in the interactive case, and obtain the same results as if I pressed TAB:
$ compgen -c lua
luac
lua5.1
lua
luac5.1
luatex
lualatex
luatools
... and that is exactly what I was looking for :)
Hope this helps someone,
Cheers!
If you're going to that much effort, why not just add the cost of a fork or two and use something that is more than capable of providing everything you want. https://github.com/hanslub42/rlwrap
#!/bin/bash
which yum && yum install rlwrap
which zypper && zypper install rlwrap
which port && port install rlwrap
which apt-get && apt-get install rlwrap
REPLY=$( rlwrap -o cat )
Or as the man page puts it:
In a shell script, use rlwrap in ’one−shot’ mode as a replacement for read
order=$(rlwrap -p Yellow -S 'Your pizza? ' -H past_orders -P Margherita -o cat)
I'm implementing an interactive bash script similar to the MySQL client, /usr/bin/mysql. In this script I need to issue various types of 'commands'. I also need to provide a history mechanism whereby the user can use the up/down arrow keys to scroll through the commands entered so far.
The snippet listed here (Example 15-6, Detecting the arrow keys) does not exactly do what I want it to. I really want the following:
The up/down arrow keys should operate in silent mode. Meaning, they should not echo their character codes on the terminal.
The other keys however (which will be used to read the command names and their arguments) must not operate in silent mode.
The problem with read -s -n3 is that it does not satisfy my simultaneously conflicting requirements of silent mode and echo mode, based solely on the character code. Also, the value -n3 will work for arrow keys but, for other/regular keys, it won't 'return control' to the calling program until 3 characters have been consumed.
Now, I could try -n1 and manually assemble the input, one character at a time (yuck!). But the character-code based silent-/echo-mode switching problem would still persist!
Has anyone attempted this thing in bash? (Note: I cannot use C, nor other scripting languages like Perl, Python, etc.)
EDIT
Continuing with Dennis' answer... You will also need to manually add your desired entries to your history via history -s, like so...
while read -e x; do
history -s "$x"
# ...
done
You can use read -e to have read use readline. It will process your cursor keys and maintain the history for you. You will also need to manually add your desired entries to your history via history -s, like so:
while read -e x; do
history -s "$x"
# ...
done
MySQL and Bash use the Readline library to implement this. Maybe you could use something like rlwrap or rlfe?
rlwrap has a special "one-shot" mode to act as a replacement for the 'read' shell command. If you wish, every occurrence of this command in your script can be given its own history and completion word list.
Use it like this:
REPLY=$(rlwrap -o cat)
or, specifying a history file and a completion wordlist:
REPLY=$(rlwrap -H my_history -f my_completions -o cat)
I work in a place that has gazillions of tools which require tons of options, so I rely on my shell's history significantly. I even back it up every now and then just to make sure I don't lose useful, lengthy commands.
I just typed one of these commands and I want to make sure it's flushed to the history file, but I have a long-running job in the background and I can't type exec zsh. Is there something else I can do in this situation?
(Sure, I could copy and paste it into a file, but it would be more logical for there to exist a flush-history command.)
To write the shell history to the history file, do
fc -W
fc has some useful flags, see them all in man zshbuiltins.
You can also fully automate reading and writing the history file after each command (thus sharing your history file automatically with each running zsh) by saying setopt -o sharehistory. Read more history-related options in man zshoptions.
I also just found:
setopt INC_APPEND_HISTORY
From man zshoptions:
INC_APPEND_HISTORY
This options works like APPEND_HISTORY except that new history
lines are added to the $HISTFILE incrementally (as soon as they
are entered), rather than waiting until the shell exits. The
file will still be periodically re-written to trim it when the
number of lines grows 20% beyond the value specified by $SAVE-
HIST (see also the HIST_SAVE_BY_COPY option).
And use
fc -R
to read in the history (after writing it) in an existing zsh shell.
appen below line to ~/.zshrc, it will save 1000 entry we increase by changing value of HISTSIZE and SAVEHIST
HISTSIZE=1000
if (( ! EUID )); then
HISTFILE=~/.zsh_history_root
else
HISTFILE=~/.zsh_history
fi
SAVEHIST=1000