in my script i run vim like this:
$EDITOR hello.c
then i execute it:
./script.sh hello.c <stdin >>stdout 2>>stderr; echo $? >>errcode
Does anybody know why the output is like:
[?1049h[?1h=[1;24r[?12;25h[?12l[?25h[27m[m[H[2J[?25l[24;1H"/home/sandbox/ios-15-1/desc1_file/xvysta02/hello.c" 6L, 77C[1;1H[35m#include [m[31m<stdio.h>[m
[32mint[m main()
{
printf([31m"Hello world[m[35m\n[m[31m"[m);
[33mreturn[m [31m0[m;
}
[1m[34m~ [8;1H~ [9;1H~ [10;1H~ [11;1H~ [12;1H~ [13;1H~ [14;1H~ [15;1H~ [16;1H~ [17;1H~ [18;1H~ [19;1H~ [20;1H~ [21;1H~ [22;1H~ [23;1H~ [m[24;63H1,1[11CAll[1;1H[?12l[?25h[24;1H[?1l>[?1049lVim: Error reading input, exiting...
Vim: Finished.
[24;1H
Instead of this:
running editor with: /.sanitized./hello.c (hash da427fd)
seems like it printout content of file hello.c
It's my school project so the output is from testing script.
You cannot launch Vim interactively inside a script with its output redirected. Vim prints out various ANSI Escape sequences to control the terminal, and that's what you're seeing. You didn't mention what you're using Vim for, so here's a general introduction to automating text manipulation in a script:
Alternatives
Unless you really need special Vim capabilities, you're probably better off using non-interactive tools like sed, awk, or Perl / Python / Ruby / your favorite scripting language here.
That said, you can use Vim non-interactively:
Silent Batch Mode
For very simple text processing (i.e. using Vim like an enhanced 'sed' or 'awk', maybe just benefitting from the enhanced regular expressions in a :substitute command), use Ex-mode.
REM Windows
call vim -N -u NONE -n -i NONE -es -S "commands.ex" "filespec"
Note: silent batch mode (:help -s-ex) messes up the Windows console, so you may have to do a cls to clean up after the Vim run.
# Unix
vim -T dumb --noplugin -n -i NONE -es -S "commands.ex" "filespec"
Attention: Vim will hang waiting for input if the "commands.ex" file doesn't exist; better check beforehand for its existence! Alternatively, Vim can read the commands from stdin. You can also fill a new buffer with text read from stdin, and read commands from stderr if you use the - argument.
Full Automation
For more advanced processing involving multiple windows, and real automation of Vim (where you might interact with the user or leave Vim running to let the user take over), use:
vim -N -u NONE -n -c "set nomore" -S "commands.vim" "filespec"
Here's a summary of the used arguments:
-T dumb Avoids errors in case the terminal detection goes wrong.
-N -u NONE Do not load vimrc and plugins, alternatively:
--noplugin Do not load plugins.
-n No swapfile.
-i NONE Ignore the |viminfo| file (to avoid disturbing the
user's settings).
-es Ex mode + silent batch mode -s-ex
Attention: Must be given in that order!
-S ... Source script.
-c 'set nomore' Suppress the more-prompt when the screen is filled
with messages or output to avoid blocking.
vim is using control characters to make an interactive screen.
When you want to use vim with redirecting, try something like
vi hello.c >/dev/null 2>#1 <<end
:1,$ s/world/StackOverflow/
:wq
end
The line with end must start at the first column and without spaces after it.
When you want to play more with this construction, you can go into insert mode,
type something and "hit the escape-key". Put an escaped escape-key using CTRL-V ESC.
Related
There seems to be quite a lot of information on how to edit and execute a command using your editor using "edit-and-execute-command (C-x C-e)", but what I would like to achieve is take the current shell command, apply certain filtering (using a script) and then return it to prompt for further approval/manual changes before execution. Is this possible with bash?
Latest update based on my experience
The part 0"+y$dd in the following mapping is really something that you should carefully think about and tailor it to your taste/workflow/experience.
For instance, very frequently I've found myself ending up with multiple lines in the buffer, where I only want to execute the one the cursor is on; in this case I can use 0"+y$dd:%d<CR> instead of 0"+y$dd.
And this is just one of the possible scenarios.
Final answer for those who like vim
Set vim as your EDITOR/VISUAL, so that when editing a command line, you will use vim to edit it.
Put au BufEnter /tmp/bash-fc.* nn <Leader>d 0"+y$dd:wq<CR> in your ~/.vimrc file to map Leaderd (which you will rarely use when editing a command) to the action "delete the current line into the + register without the trailing EOL".
you can use either the + or the * register in the mapping above; the ways to paste into the terminal will likely differ; you need the +clipboard option for these registers to be available.
When finished editing a command in the vim editor, hit EscapeLeaderd.
Paste the clipboard into the terminal (this is terminal-dependent).
Original answer
I often need to do the same, and I do it as follows. (I normally use the set -o vi in bash, so points 1 and 2 in the following are different if you use set -o emacs, the default; based on your question it looks like points 1 and 2 are unified in Ctrl+x followed by Ctrl+e, which is harder to type, imho.)
hit Escape to be in normal mode,
hit v to enter the editor to edit the command,
edit the command as I like,
(This is where you ask the question.)
hit Escape0"+y$dd:wq,
Note: 0"+y$, not simply "+yy, as the latter would copy the newline too, and this would result in executing the command upon pasting it in the command line,
paste the clipboard on the command line
how to do this depends on the terminal you are using, I guess; I hit Ctrl+Alt+v in URxvt.
proceed to approval/manual edit.
Clearly this is just a workaround, consisting in copying the edited command into the clipboard before deleting the whole command, so that nothing gets executed upon exiting the editor; however it's the best I can get for myself.
Update
As my EDITOR (and VISUAL) is equal to vim, when I edit the command, I edit it in vim.
In this respect, I have noticed that the buffer is named /tmp/bash-fc.random, where random is a 6-characters alphanumeric random string.
This gives space to a lot of possiblities, if you use vim as your editor, as you can define some mapping in your .vimrc to execute the whole sequence Escape0"+y$dd:wq. For instance, one command that you'd rarely use when editing a command line is Leaderd; therefore you can put the following mapping in your .vimrc file
au BufEnter /tmp/bash-fc.* nn <Leader>d 0"+y$dd:wq<CR>
so that step 4 in the above recipe becomes
hit EscapeLeaderd
It's not possible to do that in Bash/readline but it's possible in zsh
using edit-command-line command:
darkstar% autoload edit-command-line; zle -N edit-command-line
darkstar% bindkey "^X^E" edit-command-line
Now press Control-x Control-e to open your editor, edit line, leave the editor - you will see the updated command line but it will not be executed automatically.
Now that I think about it, maybe a variation of what #kenorb suggested in a comment is the best workaround (as it seems no solution exists), if we want to stick to bash.
What you can do is prepend a # (the comment character in bash) to the command, rather than echo. Then when you exit the editor, the command will be ineffective, and you will only have to press arrow up (or k, if you use set -o vi), remove the # and confirming.
Note that this strategy adds just a few keystrokes, so it can be fairly efficient, depending on your typing level.
These pieces might get you closer:
a) replace the the normal binding for newline newline (ctrl-M)
bind -x '"\C-M":date"
b) grab the current line from the history using !#
replace date with whatever script you want.
c) edit manually
d) if necessary, somehow tie in !!:p which prints the new command to the command line but does not execute it, thus letting you manually edit it.
e) using ctrl-J submit edited command rather than a newline
or they might not ....
There is an option in bash to modify command from history without executing it. I'm not sure it it's possible to use script for this, doesn't seem to be likely. Although, you can make modifications using history modifiers.
Enable option histverify to prevent execution of modified command
Use chain of modifiers to change last command
Use "!!" to put your result to command line for final edit
Here is how it looks:
$ shopt -s histverify
$ ls *.sh
script1.sh script2.sh script3.sh script-return.sh
$ !!:s/*/script1/:p
ls script1.sh
$ !!:s/1/2/:p
ls script2.sh
$ !!
$ ls script2.sh
script2.sh
I'd like to point you to the Composure framework for Bash (I'm not affiliated with it): https://github.com/erichs/composure
It provides draft and revise functions that sound like they could help with what you're trying to do. Here's a (long) quote from the project's readme file:
Composure helps by letting you quickly draft simple shell functions,
breaking down your long pipe filters and complex commands into
readable and reusable chunks.
Draft first, ask questions later
Once you've crafted your gem of a command, don't throw it away! Use
draft () and give it a good name. This stores your last command as a
function you can reuse later. Think of it like a rough draft.
$ cat servers.txt
bashful: up
doc: down
up-arrow
$ cat servers.txt | grep down
doc: down
$ draft finddown
$ finddown | mail -s "down server(s)" admin#here.com
Revise, revise, revise!
Now that you've got a minimal shell function, you may want to make it
better through refactoring and revision. Use the revise () command
to revise your shell function in your favorite editor.
generalize functions with input parameters
add or remove functionality
add supporting metadata for documentation
$ revise finddown
finddown ()
{
about finds servers marked 'down' in text file
group admin
cat $1 | grep down
}
$ finddown servers.txt
doc: down
It does not seem possible with a keyboard shortcut, at least:
$ bind -P | grep -e command -e edit
complete-command can be found on "\e!".
edit-and-execute-command can be found on "\C-x\C-e".
emacs-editing-mode is not bound to any keys
possible-command-completions can be found on "\C-x!".
vi-editing-mode is not bound to any keys
This can be done in native bash using readline specifically READLINE_LINE and READLINE_POINT variables. I use this functionality all the time though not through vim, you would need to get the value of $selected from your vim command and if not empty it takes your original line + your input and replaces your original line with the combination without executing. output as a variable
_main() {
selected="$(__coms_select__ "$#")"
origonal_text=$READLINE_LINE READLINE_LINE="${READLINE_LINE:0:$READLINE_POINT}$selected${READLINE_LINE:$READLINE_POINT}"
READLINE_POINT=$(( READLINE_POINT + ${#selected} ))
}
bind -m emacs-standard -x '"\C-e": _main '
bind -m vi-command -x '"\C-e": _main '
bind -m vi-insert -x '"\C-e": _main '
Edit
Just remembered these two utilities that will let you do this as well.
Vipe allows you to run your editor in the middle of a unix pipeline and edit the data that is being piped between programs.
vp, up, vipe, Neomux (upgrade of nvim terminal) you can do some pretty neat throwing buffers between the terminal and split window.
and Athame (full vim on the command line)
https://github.com/ardagnir/athame
careful with that one though plugins work on the cli and it can get funky if you got tons of plugins
To do repetitive edits, like replacing a string with another in many files, I've been using running vim from windows batch files (help on stack overflow to do that was very useful!). My follow up questions are how to handle it if:
a) the vim command contains a quotation mark as well as spaces
b) how do I "escape" upon inserting in vim, i.e. what is the batch file
equivalent of escape in vim
For instance in regard to (a) I'd like to do:
call vim fn.txt +"1,$s/$/\"/" +wq
To add a quotation mark at the end of every line in file fn.txt, but of course this will not work because windows does not know that the 2nd quotation mark int the line above is quoted.
As an example of (b), how do I insert a line containing the text "inserted line" after line number 7 in file fn.txt
call vim fn.txt +:7 +oinserted line????
where the question marks above indicate where I don't know how to continue.
Alternatives
Unless you really need special Vim capabilities, you're probably better off using non-interactive tools like sed, awk, or Perl / Python / Ruby / your favorite scripting language here.
That said, you can use Vim non-interactively. However, as you've found out, the escaping on the Windows command-line is tricky. The escape character should be ^ (so " becomes ^"), but I also couldn't get it to work in some few quick tests. Just read the documentation for the /S switch on cmd.exe /? and try to understand it :-( Because of that, it's better to put the Vim commands into a separate script, as pass only the script's filespec to Vim. The following details all use that trick.
Silent Batch Mode
For very simple text processing (i.e. using Vim like an enhanced 'sed' or 'awk', maybe just benefitting from the enhanced regular expressions in a :substitute command), use Ex-mode.
REM Windows
call vim -N -u NONE -n -i NONE -es -S "commands.ex" "filespec"
Note: silent batch mode (:help -s-ex) messes up the Windows console, so you may have to do a cls to clean up after the Vim run.
# Unix
vim -T dumb --noplugin -n -i NONE -es -S "commands.ex" "filespec"
Attention: Vim will hang waiting for input if the "commands.ex" file doesn't exist; better check beforehand for its existence! Alternatively, Vim can read the commands from stdin. You can also fill a new buffer with text read from stdin, and read commands from stderr if you use the - argument.
Full Automation
For more advanced processing involving multiple windows, and real automation of Vim (where you might interact with the user or leave Vim running to let the user take over), use:
vim -N -u NONE -n -c "set nomore" -S "commands.vim" "filespec"
Here's a summary of the used arguments:
-T dumb Avoids errors in case the terminal detection goes wrong.
-N -u NONE Do not load vimrc and plugins, alternatively:
--noplugin Do not load plugins.
-n No swapfile.
-i NONE Ignore the |viminfo| file (to avoid disturbing the
user's settings).
-es Ex mode + silent batch mode -s-ex
Attention: Must be given in that order!
-S ... Source script.
-c 'set nomore' Suppress the more-prompt when the screen is filled
with messages or output to avoid blocking.
I have a set of multiple files requiring the same set of edits, I am trying to create a bash script for editing them in vi, however I don't know how to use vi within the scripts to make the edits? Any suggestions would be helpful.
I'd highly recommend using sed or awk, both programs use regular expressions for selecting and processing text.
But here's how you can do it using vim too:
Vim has an ex mode (aka commandline version) which solves this purpose and is much easier to use in scripts. Taking the solution from this answer:
You could simply include the following in your bashscript:
ex $yourfile <<EOEX
:%s/$string_to_replace/$string_to_replace_it_with/g
:x
EOEX
For example:
ex file.txt << EOEX
:%s/hello/world/g
:x
EOEX
Or you can use the -c option to pass ex commands to vim.
For example:
vim file.txt -c ':%s/hello/world/g' -c 'wq'
If you want to do a single :s command, then you may we well use sed, or you can use vim -c as #fnatic_shank suggests. For more complex scripts, you can use -S script.vim instead of several -c arguments. I like to use
$ vim -e -s -N -V0vim.log -S script.vim infile.txt
See the help for -N, -V, -S, and especially
:help -s-ex
which includes the warning
If Vim appears to be stuck try typing "qa!". You don't
get a prompt thus you can't see Vim is waiting for you to type
something.
(I ran into this while testing my answer. I kept infile.txt open in vim, so it asked what to do about the existing swap file when I tried to start a new vim as above.) If you want to send the file to stdout, then you might have these two lines at the end of script.vim:
g/^
silent q!
I am writing a Bash script that runs a command-line program (Gromacs), saves the results, modifies the input files, and then loops through the process again.
I am trying to use Vim to modify the input text files, but I have not been able to find a way to execute internal Vim commands like :1234, w, x, dd, etc. from the .sh file after opening my input files in Vim ("vim conf.gro").
Is there a practical way to execute Vim commands from the shell script?
I think vim -w/W and vim -s is what you are looking for.
The "Vim operations/key sequence" you could also record with vim -w test.keys input.file. You could write the test.keys too. For example, save this in the file:
ggwxjddZZ
This will do:
Move to the first line,
move to the next word,
delete one character,
move to the next line,
delete the line, and
save and quit.
With this test.keys file, you could do:
vim -s test.keys myInput.file
Your "myInput.file" would be processed by the above operations, and saved. You could have that line in your shell script.
VimGolf is using the same way to save the user's solution.
You can script Vim via the -c flag.
vim -c "set ff=dos" -c wq mine.mak
However that only gets you so far.
From the commands you gave it looks like you are trying to run some normal commands. You will need to use :normal. e.g. :norm dd
Writing all this on the command line is asking for trouble. I suggest you make a Vim file (e.g. commands.vim) and then :source via -S.
You probably want to get good and conformable Vim's ex commands. Take a look at :h ex-cmd-index
So you will end up with something like this. With all your Vim commands inside of commands.vim.
vim -S commands.vim mine.mak
You may also want to look into using sed and/or awk for text processing.
Alternatives
Unless you really need special Vim capabilities, you're probably better off using non-interactive tools like sed, awk, or Perl / Python / Ruby / your favorite scripting language here.
That said, you can use Vim non-interactively:
Silent Batch Mode
For very simple text processing (i.e. using Vim like an enhanced 'sed' or 'awk', maybe just benefitting from the enhanced regular expressions in a :substitute command), use Ex-mode.
REM Windows
call vim -N -u NONE -n -es -S "commands.ex" "filespec"
Note: silent batch mode (:help -s-ex) messes up the Windows console, so you may have to do a cls to clean up after the Vim run.
# Unix
vim -T dumb --noplugin -n -es -S "commands.ex" "filespec"
Attention: Vim will hang waiting for input if the "commands.ex" file doesn't exist; better check beforehand for its existence! Alternatively, Vim can read the commands from stdin. You can also fill a new buffer with text read from stdin, and read commands from stderr if you use the - argument.
Full Automation
For more advanced processing involving multiple windows, and real automation of Vim (where you might interact with the user or leave Vim running to let the user take over), use:
vim -N -u NONE -n -c "set nomore" -S "commands.vim" "filespec"
Here's a summary of the used arguments:
-T dumb Avoids errors in case the terminal detection goes wrong.
-N -u NONE Do not load vimrc and plugins, alternatively:
--noplugin Do not load plugins.
-n No swapfile.
-es Ex mode + silent batch mode -s-ex
Attention: Must be given in that order!
-S ... Source script.
-c 'set nomore' Suppress the more-prompt when the screen is filled
with messages or output to avoid blocking.
This question already has answers here:
How to trick an application into thinking its stdout is a terminal, not a pipe
(9 answers)
Closed 5 years ago.
Various bash commands I use -- fancy diffs, build scripts, etc, produce lots of color output.
When I redirect this output to a file, and then cat or less the file later, the colorization is gone -- presumably b/c the act of redirecting the output stripped out the color codes that tell the terminal to change colors.
Is there a way to capture colorized output, including the colorization?
One way to capture colorized output is with the script command. Running script will start a bash session where all of the raw output is captured to a file (named typescript by default).
Redirecting doesn't strip colors, but many commands will detect when they are sending output to a terminal, and will not produce colors by default if not. For example, on Linux ls --color=auto (which is aliased to plain ls in a lot of places) will not produce color codes if outputting to a pipe or file, but ls --color will. Many other tools have similar override flags to get them to save colorized output to a file, but it's all specific to the individual tool.
Even once you have the color codes in a file, to see them you need to use a tool that leaves them intact. less has a -r flag to show file data in "raw" mode; this displays color codes. edit: Slightly newer versions also have a -R flag which is specifically aware of color codes and displays them properly, with better support for things like line wrapping/trimming than raw mode because less can tell which things are control codes and which are actually characters going to the screen.
Inspired by the other answers, I started using script. I had to use -c to get it working though. All other answers, including tee, different script examples did not work for me.
Context:
Ubuntu 16.04
running behavior tests with behave and starting shell command during the test with python's subprocess.check_call()
Solution:
script --flush --quiet --return /tmp/ansible-output.txt --command "my-ansible-command"
Explanation for the switches:
--flush was needed, because otherwise the output is not well live-observable, coming in big chunks
--quiet supresses the own output of the script tool
-c, --command directly provides the command to execute, piping from my command to script did not work for me (no colors)
--return to make script propagate the exit code of my command so I know if my command has failed
I found that using script to preserve colors when piping to less doesn't really work (less is all messed up and on exit, bash is all messed up) because less is interactive. script seems to really mess up input coming from stdin even after exiting.
So instead of running:
script -q /dev/null cargo build | less -R
I redirect /dev/null to it before piping to less:
script -q /dev/null cargo build < /dev/null | less -R
So now script doesn't mess with stdin and gets me exactly what I want. It's the equivalent of command | less but it preserves colors while also continuing to read new content appended to the file (other methods I tried wouldn't do that).
some programs remove colorization when they realize the output is not a TTY (i.e. when you redirect them into another program). You can tell some of those to use color forcefully, and tell the pager to turn on colorization, for example use less -R
This question over on superuser helped me when my other answer (involving tee) didn't work. It involves using unbuffer to make the command think it's running from a shell.
I installed it using sudo apt install expect tcl rather than sudo apt-get install expect-dev.
I needed to use this method when redirecting the output of apt, ironically.
I use tee: pipe the command's output to teefilename and it'll keep the colour. And if you don't want to see the output on the screen (which is what tee is for: showing and redirecting output at the same time) then just send the output of tee to /dev/null:
command| teefilename> /dev/null