I was just trying to figure out EOF and here documents. I read something online that seemed like a cool idea I could expand upon. It is supposed to create a text file, and add text to it, all using a here document. The syntax is as follows:
#!/bin/sh
filename=test.txt
vim $filename <<EndOfCommands
i
This file was created automatically from
a shell script.
^[
ZZ
EndOfCommands
Now unfortunately I get an error from this:
./EOF.sh
Vim: Warning: Input is not from a terminal
Vim: Error reading input, exiting...
Vim: Finished.
Can anyone help fix this error? Also im assuming ^[ stands for the escape button being hit?
vi and vim are meant to be used interactively in a terminal, not scripted.
Using ed instead, which can happily accept input from standard input sources of all types:
#!/bin/sh
filename=test.txt
ed -s "$filename" <<EndOfCommands
i
This file was created automatically from
a shell script.
.
w
EndOfCommands
(In insert mode, a line with a single period indicates end-of-input and goes back to command mode, kind of like escape in vi(m).)
Using ex is another option; on systems with vim installed it's often a non-visual version of it.
With vim:
filename="test.txt"
vim -c ":wq! $filename" - << EOF
This file was created automatically from
a shell script.
EOF
See: man vim
Related
There seems to be quite a lot of information on how to edit and execute a command using your editor using "edit-and-execute-command (C-x C-e)", but what I would like to achieve is take the current shell command, apply certain filtering (using a script) and then return it to prompt for further approval/manual changes before execution. Is this possible with bash?
Latest update based on my experience
The part 0"+y$dd in the following mapping is really something that you should carefully think about and tailor it to your taste/workflow/experience.
For instance, very frequently I've found myself ending up with multiple lines in the buffer, where I only want to execute the one the cursor is on; in this case I can use 0"+y$dd:%d<CR> instead of 0"+y$dd.
And this is just one of the possible scenarios.
Final answer for those who like vim
Set vim as your EDITOR/VISUAL, so that when editing a command line, you will use vim to edit it.
Put au BufEnter /tmp/bash-fc.* nn <Leader>d 0"+y$dd:wq<CR> in your ~/.vimrc file to map Leaderd (which you will rarely use when editing a command) to the action "delete the current line into the + register without the trailing EOL".
you can use either the + or the * register in the mapping above; the ways to paste into the terminal will likely differ; you need the +clipboard option for these registers to be available.
When finished editing a command in the vim editor, hit EscapeLeaderd.
Paste the clipboard into the terminal (this is terminal-dependent).
Original answer
I often need to do the same, and I do it as follows. (I normally use the set -o vi in bash, so points 1 and 2 in the following are different if you use set -o emacs, the default; based on your question it looks like points 1 and 2 are unified in Ctrl+x followed by Ctrl+e, which is harder to type, imho.)
hit Escape to be in normal mode,
hit v to enter the editor to edit the command,
edit the command as I like,
(This is where you ask the question.)
hit Escape0"+y$dd:wq,
Note: 0"+y$, not simply "+yy, as the latter would copy the newline too, and this would result in executing the command upon pasting it in the command line,
paste the clipboard on the command line
how to do this depends on the terminal you are using, I guess; I hit Ctrl+Alt+v in URxvt.
proceed to approval/manual edit.
Clearly this is just a workaround, consisting in copying the edited command into the clipboard before deleting the whole command, so that nothing gets executed upon exiting the editor; however it's the best I can get for myself.
Update
As my EDITOR (and VISUAL) is equal to vim, when I edit the command, I edit it in vim.
In this respect, I have noticed that the buffer is named /tmp/bash-fc.random, where random is a 6-characters alphanumeric random string.
This gives space to a lot of possiblities, if you use vim as your editor, as you can define some mapping in your .vimrc to execute the whole sequence Escape0"+y$dd:wq. For instance, one command that you'd rarely use when editing a command line is Leaderd; therefore you can put the following mapping in your .vimrc file
au BufEnter /tmp/bash-fc.* nn <Leader>d 0"+y$dd:wq<CR>
so that step 4 in the above recipe becomes
hit EscapeLeaderd
It's not possible to do that in Bash/readline but it's possible in zsh
using edit-command-line command:
darkstar% autoload edit-command-line; zle -N edit-command-line
darkstar% bindkey "^X^E" edit-command-line
Now press Control-x Control-e to open your editor, edit line, leave the editor - you will see the updated command line but it will not be executed automatically.
Now that I think about it, maybe a variation of what #kenorb suggested in a comment is the best workaround (as it seems no solution exists), if we want to stick to bash.
What you can do is prepend a # (the comment character in bash) to the command, rather than echo. Then when you exit the editor, the command will be ineffective, and you will only have to press arrow up (or k, if you use set -o vi), remove the # and confirming.
Note that this strategy adds just a few keystrokes, so it can be fairly efficient, depending on your typing level.
These pieces might get you closer:
a) replace the the normal binding for newline newline (ctrl-M)
bind -x '"\C-M":date"
b) grab the current line from the history using !#
replace date with whatever script you want.
c) edit manually
d) if necessary, somehow tie in !!:p which prints the new command to the command line but does not execute it, thus letting you manually edit it.
e) using ctrl-J submit edited command rather than a newline
or they might not ....
There is an option in bash to modify command from history without executing it. I'm not sure it it's possible to use script for this, doesn't seem to be likely. Although, you can make modifications using history modifiers.
Enable option histverify to prevent execution of modified command
Use chain of modifiers to change last command
Use "!!" to put your result to command line for final edit
Here is how it looks:
$ shopt -s histverify
$ ls *.sh
script1.sh script2.sh script3.sh script-return.sh
$ !!:s/*/script1/:p
ls script1.sh
$ !!:s/1/2/:p
ls script2.sh
$ !!
$ ls script2.sh
script2.sh
I'd like to point you to the Composure framework for Bash (I'm not affiliated with it): https://github.com/erichs/composure
It provides draft and revise functions that sound like they could help with what you're trying to do. Here's a (long) quote from the project's readme file:
Composure helps by letting you quickly draft simple shell functions,
breaking down your long pipe filters and complex commands into
readable and reusable chunks.
Draft first, ask questions later
Once you've crafted your gem of a command, don't throw it away! Use
draft () and give it a good name. This stores your last command as a
function you can reuse later. Think of it like a rough draft.
$ cat servers.txt
bashful: up
doc: down
up-arrow
$ cat servers.txt | grep down
doc: down
$ draft finddown
$ finddown | mail -s "down server(s)" admin#here.com
Revise, revise, revise!
Now that you've got a minimal shell function, you may want to make it
better through refactoring and revision. Use the revise () command
to revise your shell function in your favorite editor.
generalize functions with input parameters
add or remove functionality
add supporting metadata for documentation
$ revise finddown
finddown ()
{
about finds servers marked 'down' in text file
group admin
cat $1 | grep down
}
$ finddown servers.txt
doc: down
It does not seem possible with a keyboard shortcut, at least:
$ bind -P | grep -e command -e edit
complete-command can be found on "\e!".
edit-and-execute-command can be found on "\C-x\C-e".
emacs-editing-mode is not bound to any keys
possible-command-completions can be found on "\C-x!".
vi-editing-mode is not bound to any keys
This can be done in native bash using readline specifically READLINE_LINE and READLINE_POINT variables. I use this functionality all the time though not through vim, you would need to get the value of $selected from your vim command and if not empty it takes your original line + your input and replaces your original line with the combination without executing. output as a variable
_main() {
selected="$(__coms_select__ "$#")"
origonal_text=$READLINE_LINE READLINE_LINE="${READLINE_LINE:0:$READLINE_POINT}$selected${READLINE_LINE:$READLINE_POINT}"
READLINE_POINT=$(( READLINE_POINT + ${#selected} ))
}
bind -m emacs-standard -x '"\C-e": _main '
bind -m vi-command -x '"\C-e": _main '
bind -m vi-insert -x '"\C-e": _main '
Edit
Just remembered these two utilities that will let you do this as well.
Vipe allows you to run your editor in the middle of a unix pipeline and edit the data that is being piped between programs.
vp, up, vipe, Neomux (upgrade of nvim terminal) you can do some pretty neat throwing buffers between the terminal and split window.
and Athame (full vim on the command line)
https://github.com/ardagnir/athame
careful with that one though plugins work on the cli and it can get funky if you got tons of plugins
Running MacOS High Sierra 10.13.5
Good day! Today I attempted to write a script in my .bash_profile to allow me to call Sublime Text with a single command, sublime.
Here is the script I produced with my limited knowledge of bash as well as some research:
sublime_open() # Function which opens the first argument as a text file in Sublime.
{
open -a /Applications/Sublime\ Text.app < $1.txt;
}
export -f sublime_open;
sublime_test() # Function which will open first argument as a text file in Sublime,
{ # creating one if it does not exist in the current directory.
if [ -e "$1" ];
then sublime_open "$1"
else
touch "$1".txt
sublime_open("$1")
fi
}
export -f sublime_test
alias sublime="sublime_test"
export sublime
I separated the function into two pieces for the sake of readability.
Expected Behavior:
sublime foo opens an instance of Sublime to the file foo.txt, creating that file if it does not already exist.
Observed Behavior:
Bash returns:
line 29: syntax error: unexpected end of file
Note that line 29 is the line after the file's final line, which does not exist.
Attempts at solution:
So far I have searched for solutions here and here.
I have tried:
Copying over to a new bash profile and deleting the old one to remove "ghost" characters
Running dos2unix on my .bash_profile
Parsing the script myself looking for individual errors, the problem seems to occur in the sublime_open() portion.
Sorry if my script is elementary or uninformed, this was my first attempt at writing a script that behaves this way. I welcome help or tips outside of the scope of my issue.
Thanks in advance!
Edit: Fixed misused heredoc as per #jwodder's solution as well as improper function calls as per #codeforester's solution.
Edit 2: If you are looking for a more elegant solution to this same problem, have a look at this very helpful gist.
The problem is this line:
open -a /Applications/Sublime\ Text.app << $1.txt;
I assume that what you mean for this to do is to feed the contents of $1.txt to Sublime Text on standard input. However, that would be done with one <, not two. With two <, you get a heredoc, which means that bash will look for the next line containing just $1.txt and will use all lines in between as the input to Sublime Text. As the file ends before bash finds a line with just $1.txt, you get an "unexpected end of file" error.
Keep in mind that on macOS, an *.app is just a folder. Generally speaking, you need to specify the location to a binary (or script). In the case of Sublime Text you have to use the Sublime Text CLI tool:
/Applications/Sublime Text.app/Contents/SharedSupport/bin/subl
See the OS X Command Line documentation for details.
I'm trying to use vim with -s option to run a script that replaces some lines in a file like this (text.txt):
test1
ab
ac
ae
test2
sd
Script file is like this (script):
:silent %s/test1\zs\_.\+\zetest2/\=substitute(submatch(0), '\n\(\w\)', '\n#\1', 'g')/g
:wq
It comments out lines between test1 and test2. Which is what I want. What I don't want though is output before and after prompt. I run it and get:
user#hostname: ~/vimtest$ vim -s script text.txt
^[[?1;2cuser#hostname: ~/vimtest$ 1;2c
So this ^[[?1;2c is bad news already but 1;2c is in the input as if I already typed it. If I hit enter it gives me a bash error. So I have to remove these symbols each time the script is used. Any ideas?
It seems like vim (or some vim startup script) is trying to figure out what type of terminal you are using. The ^[[?1;2c, with the last few characters left in the input buffer, is almost certainly part of your terminal emulator's response to a DA (Device Attributes) query. You can see this yourself by typing in bash:
printf '\033[c'
or, to see the complete return, pause a bit:
printf '\033[c'; sleep 0.1; echo
The response \033[?1;2c means "I'm a VT100 with Advanced Video Option.", which is what xterm and many other console programs respond. (The Linux console itself responds \033[?6c, which means "I'm a VT102.")
The reason that only 1;2c is left in the console input buffer, by the way, is that the initial escape code \033[? was ignored when it was read. The readline library will ignore it without echoing it, whereas normal console input will echo it and then ignore it; that's why the two shell commands above differ.
I can't reproduce this problem with my vim installation, so I don't really even know where to start looking. But you might try to see if disabling all startup files helps:
vim -u NONE -s script text.txt
If that helps, start disabling installed extensions one by one until you find the one which is causing the problem.
:%s/test1\zs\_.\+\ze\ntest2/\=substitute(submatch(0), '\n', '\n#', 'g')/g
:wq
this is tested here, it changed the input file in required way.
Some changes done based on your command:
add \n after \ze
in substitute() function we can just handle the \n, we don't need to capture the word after the \n
I noticed that you tagged the question with bash, so I thought a shell-solution should be accepted too.
awk '/test1/{p=1;print;next}/test2/{p=0;print;next}{$0=(p?"#":"")$0}7' file
this awk oneliner should do that for you. vim is very powerful editor, I love vim. But if you want to do some automatic transformation, I prefer a script or a proper text processing tool. On a linux box you can always find one. It is easier to test and debug.
Test with your input:
kent$ cat f
test1
ab
ac
ae
test2
sd
kent$ awk '/test1/{p=1;print;next}/test2/{p=0;print;next}{$0=(p?"#":"")$0}7' f
test1
#ab
#ac
#ae
test2
sd
If you want to save the text back to your file, you can :
awk '...' file > tmp.file && mv tmp.file file
I am writing a Bash script that runs a command-line program (Gromacs), saves the results, modifies the input files, and then loops through the process again.
I am trying to use Vim to modify the input text files, but I have not been able to find a way to execute internal Vim commands like :1234, w, x, dd, etc. from the .sh file after opening my input files in Vim ("vim conf.gro").
Is there a practical way to execute Vim commands from the shell script?
I think vim -w/W and vim -s is what you are looking for.
The "Vim operations/key sequence" you could also record with vim -w test.keys input.file. You could write the test.keys too. For example, save this in the file:
ggwxjddZZ
This will do:
Move to the first line,
move to the next word,
delete one character,
move to the next line,
delete the line, and
save and quit.
With this test.keys file, you could do:
vim -s test.keys myInput.file
Your "myInput.file" would be processed by the above operations, and saved. You could have that line in your shell script.
VimGolf is using the same way to save the user's solution.
You can script Vim via the -c flag.
vim -c "set ff=dos" -c wq mine.mak
However that only gets you so far.
From the commands you gave it looks like you are trying to run some normal commands. You will need to use :normal. e.g. :norm dd
Writing all this on the command line is asking for trouble. I suggest you make a Vim file (e.g. commands.vim) and then :source via -S.
You probably want to get good and conformable Vim's ex commands. Take a look at :h ex-cmd-index
So you will end up with something like this. With all your Vim commands inside of commands.vim.
vim -S commands.vim mine.mak
You may also want to look into using sed and/or awk for text processing.
Alternatives
Unless you really need special Vim capabilities, you're probably better off using non-interactive tools like sed, awk, or Perl / Python / Ruby / your favorite scripting language here.
That said, you can use Vim non-interactively:
Silent Batch Mode
For very simple text processing (i.e. using Vim like an enhanced 'sed' or 'awk', maybe just benefitting from the enhanced regular expressions in a :substitute command), use Ex-mode.
REM Windows
call vim -N -u NONE -n -es -S "commands.ex" "filespec"
Note: silent batch mode (:help -s-ex) messes up the Windows console, so you may have to do a cls to clean up after the Vim run.
# Unix
vim -T dumb --noplugin -n -es -S "commands.ex" "filespec"
Attention: Vim will hang waiting for input if the "commands.ex" file doesn't exist; better check beforehand for its existence! Alternatively, Vim can read the commands from stdin. You can also fill a new buffer with text read from stdin, and read commands from stderr if you use the - argument.
Full Automation
For more advanced processing involving multiple windows, and real automation of Vim (where you might interact with the user or leave Vim running to let the user take over), use:
vim -N -u NONE -n -c "set nomore" -S "commands.vim" "filespec"
Here's a summary of the used arguments:
-T dumb Avoids errors in case the terminal detection goes wrong.
-N -u NONE Do not load vimrc and plugins, alternatively:
--noplugin Do not load plugins.
-n No swapfile.
-es Ex mode + silent batch mode -s-ex
Attention: Must be given in that order!
-S ... Source script.
-c 'set nomore' Suppress the more-prompt when the screen is filled
with messages or output to avoid blocking.
I have a bash script and I want to use ncftp to do something. if I have this:
#!/bin/sh
HOST='my_IP_FTP_HOST'
USER='username'
PASSWD='password'
ncftp -u $USER -p $PASSWD $HOST <<END_SCRIPT
pwd
quit
END_SCRIPT
I get this error:
Syntax error in parameters or arguments
I don't understand why.
If I give it only the value and not the variables it works...
if I launch
$ sh -x script.sh
I get:
+ HOST=$'xxx.x.xx.xx\r'
+ USER=$'username\r'
+ PASSWD=$'password\r'
+ ncftp -u $'username\r' -p $'password\r' $'xxx.x.xx.xx\r'
NcFTP 3.2.1 (Jul 29, 2007) by Mike Gleason (http://www.NcFTP.com/contact/).
Welcome to FTP server
Syntax error in parameters or arguments
hmmm.... \r creates problems sure.
Are you editing this file on Microsoft Windows? It doesn't works because the original file uses Windows carriage returns, which are passed to variables and then to ncftp, which then halts.
Fix it by using another text editor, or by using dos2unix. Ideally you'd first use dos2unix once and then change the text editor.
Alternatively (but really, really not recommended) you could just parse the variables and remove the extra \r, like below, especially when the problem arises at some input reading, which doesn't seems to be the case here.
HOST=${HOST%\\r}
USER=${USER%\\r}
PASSWD=${PASSWD%\\r}
Take care when copying text from Windows, or when editing without appropriate software.