Today I was just experimenting with Linux shell. Here is what I did.
I wrote the output of clear to a file :
clear > clear.txt
Now I have some contents on the shell, (non-empty). Then I try to cat the contents of clear.txt
cat clear.txt
For my surprise, the entire screen got cleared. Can somone explain me why? if this is true, why cant we do the same for all the commands?
clear works very simply by transmitting a sequence of control codes which are interpreted by your terminal. The magic of actually clearing the screen is handled by your terminal (or terminal emulator, or console interface, or whatever you happen to be using) in response to receiving these control codes.
See also e.g. https://en.wikipedia.org/wiki/ANSI_escape_code
Well it's not a real explanation but it's how it is supposed to work:
http://man7.org/linux/man-pages/man1/clear.1.html
#CLEAR# writes to the standard output. You can redirect the standard
output to a file (which prevents #CLEAR# from actually clearing the
screen), and later cat the file to the screen, clearing it at that
point.
It is a visual representation of the clear command. Editing the text file yields:
^[[3J^[[H^[[2J
source: https://unix.stackexchange.com/questions/400142/terminal-h2j-caret-square-bracket-h-caret-square-bracket-2-j
Related
How can I capture the input/ output from a script in realtime (such as with tee), but line-by-line instead of character-by-character? My goal is to capture the input typed into the interactive prompts of a script only after backspaces and auto-completion have finished processing (after the RETURN key is hit).
Specifically, I am trying to create a wrapper script for ssh that creates a timestamped log of commands used on remote servers. The script, which uses tee to redirect the output for filtering, works well, but the redirected output gets jumbled with unsubmitted characters whenever I use the backspace key or the up/down keys to scroll through my remote history. For example: service test stopexitservice test stopart or cd ..logs[1Pls -al.
Perhaps there is a way to capture the terminal's scrollback and redirect that like with tee?
Update: I have found a character-based cleanup solution that does what I want most of the time. However, I am still hoping for an answer to this question (which may well be msw's answer that it is very difficult to do).
In the Unix world there are two primary modes of handling keyboard input. These are known as 'raw' in which characters are passed from the terminal to the reading program one at a time. This is the mode that editors (and such) will use because the editor needs to respond immediately when you press a key.
The other terminal discipline is called 'cooked' which is the line by line behavior that you think of as the bash line by line input where you get to backspace and the command is not executed until you press return. Ssh has to take your input in raw, character-by-character mode because it has no idea what is running on the other side. For example, if you are running an editor on the far side, it can't wait for a return before sending the key-press. So, as some have suggested, grabbing shell history on the far side is the only reasonable way to get a command-by-command record of the bash commands you typed.
I oversimplified for clarity; actually most installations of bash take input in raw mode because they allow editor like command modification. For example, Ctrl-P scrolls up the command history or Ctrl-A goes to the beginning of the line. And bash needs to be able to get those keys the moment they are typed not waiting for a return.
This is another reason that capturing on the local side is obnoxiously difficult: if you capture on the local side, the stream will be filled with Backspaces and all of bash's editing commands. To get a true transcript of what the remote shell actually executed you have to parse the character stream as if you were the remote shell. There also a problem if you run something like
vi /some_file/which_is_on_the_remote/machine
the input stream to the local ssh will be filled with movement commands snippets of text including backspaces and so on and it would be bloody difficult to figure out what is part of a bash command and what is you talking to the editor.
Few things involving computers are impossible; getting clean input from the local side of an ssh invocation is really, really hard.
I question the actual utility of recording the commands that you execute on a local or remote machine. The reason is that there is so much state which is not visible from a command log. As a simple example here's a log of two commands:
17:00$ cp important_file important_file.bak
17:15$ rm important_file
and two days later you are trying to figure out whether important_file.bak should have the contents you intended or not. Given that log you can't answer that simple question. Even if you had the sequence
16:58$ cat important_file
17:00$ cp important_file important_file.bak
17:15$ rm important_file
If you aren't capturing the output, the cat in the log will not tell you anything. Give me almost any command sequence and I can envision a scenario in which it will not give you the information you need to make sense of what was done.
For a very similar purpose I use GNU screen which offer the option to record everything you do in a shell session (INPUT/OUTPUT). The log it creates also comes with undesirable characters but I clean them with perl:
perl -ne 's/\x1b[[()=][;?0-9]*[0-9A-Za-z]?//g;s/\r//g;s/\007//g;print' < screenlog.0
I hope this helps.
Some features of screen:
http://speaking-my-language.blogspot.com/2010/09/top-5-underused-gnu-screen-features.html
Site I found the perl-oneliner:
https://superuser.com/questions/99128/removing-the-escape-characters-from-gnu-screens-screenlog-n
When I SSH into a particular server and launch screen, it ignores my CTRL+a key combo. Instead of CTRL+a c creating a new screen window, it instead acts as if I had just typed c. Other key combos fail in a similar way.
I've tried launching screen using screen -e ^jj to bind to j instead of a, but I still get the same result as above.
I tried adding a .screenrc file to my homedir that I know works on other machines, but it has no impact.
I also tried launch a zsh shell instead of bash.
Any ideas where to start to try and fix this? This basically renders screen unusable.
Thanks
screen reads commands from several configuration files during startup, as described in the FILES section of the man page or the Customizing Screen section of the User's Manual (also available, if it's installed, by typing info screen).
The files read are:
The file named by the $SYSSCREENRC environment variable (this may or may not be enabled)
/etc/screenrc
The file named by the $SCREENRC environment variable`
$HOME/.screenrc
(These interact and are searched in various ways that I don't entirely understand.)
In your particular case, based on the comments, the system on which you're running screen happens to have a /etc/screenrc file that contains an escape command that overrides the default.
A digression: in my own $HOME/.screenrc I have:
escape ^#^#
This sets the escape character to the null character, which can be entered by typing Ctrl-space. I find it easier to type, and less likely to conflict with other uses, than the default Ctrl-A. The only conflict I run into is the Emac set-mark-command function, and for that it's easy enough to type it twice.
Relative Emacs newbie here, just trying to adapt my programming workflow to fit with emacs. So far I've discovered shell-pop and I'm quite enjoying on-demand terminals that pop up when needed for banging out the odd commands.
What I understand so far about Emacs is that shell is a "dumb" terminal that doesn't support any ansi control codes, and that makes it incompatible with things like ncurses that attempt to draw complex UI's on a terminal emulator. This is why you can't use less or top or similar in shell-mode.
However, I seem to be having trouble with ansi-term, it's not the be-all, end-all that it's cracked up to be. Sure, it has no problems running less or git log or even nano, but there are a few things that can't quite seem to display properly when they're running in an ansi-term, such as apt-get and nosetests. I'm not sure quite what the name is for it, but apt-get's output is characterised by live-updating what is displayed on the very last line, and then having unchanging lines of text scroll out above that line. It seems to be halfway between something like less and something dumber, like cat. Somehow ansi-term doesn't like this at all, and I get very garbled output, where it seems to output everything on one line only or just generally lose it's place and output things all over, randomly. In the case of nosetests, it starts off ok, but if any libraries spew out any STDERR, the output all goes to hell in a similar way.
With some fiddling it seems possible to fix this by mashing C-l and RET, but it's not always reliable.
Does anybody know what's going on here? Is there some way to fix ansi-term so that it can display everything properly? Or is there perhaps some other mode that I don't know about that is way better? Ideally I'd like something that "just works" as effortlessly as, eg, Gnome Terminal, which can run all of the above mentioned programs without a single hiccup.
Thanks!
I resolved this issue by commenting out my entire .emacs.el and then uncommenting and restarting emacs for every single line in the file. I discovered that the following line alone was responsible for the issue:
'(fringe-mode 0 nil (fringe))
(this line disables the fringes from inside custom-set-variables).
I guess this is a bug in Emacs, that disabling the fringe causes term-mode to garble it's output really badly whenever any output line exceeds $COLUMN columns.
Anyway, I don't really like the fringes much at all, and it seems I was able to at least disable the left fringe without triggering this issue:
(set-fringe-mode (cons 0 8))
Maybe apt-get does different things based on the $TERM environment variable. What happens if you set TERM=dumb? If that makes things work, then you can experiment with different values until you find one that supports enough features but still works.
Note that git 2.0.1 (June 25th, 2014) now better detects dumb terminal when displaying verbose messages.
That might help Emacs better display some of the messages received from git, but the fringe-mode bug reported above is certainly the main cause.
See commit 38de156 by Michael Naumov (mnaoumov)
sideband.c: do not use ANSI control sequence on non-terminal
Diagnostic messages received on the sideband #2 from the server side are sent to the standard error with ANSI terminal control sequence "\033[K" that erases to the end of line appended at the end of each line.
However, some programs (e.g. GitExtensions for Windows) read and interpret and/or show the message without understanding the terminal control sequences, resulting them to be shown to their end users.
To help these programs, squelch the control sequence when the standard error stream is not being sent to a tty.
I'm writing a command interpreter like BASH, and a \ followed by a newline implies a continuation of the input stream; how can I implement that in Win32?
If I use the console mode with ENABLE_LINE_INPUT, then the user can't press backspace in order to go back to the previous line; Windows prevents him from doing so. But if I don't set ENABLE_LINE_INPUT, then I have to manually reposition the cursor, which is rather tedious given that (1) the user might have redirected the input stream, and that (2) it might be prone to race conditions, and I'd rather have Windows do it if I can.
Any way to have my newline and eat it too?
Edit:
If this would require undocumented CSRSS port requests, then I'm still interested!
Assuming you want this to run in a window, like command prompt does by default, rather than full screen, you could create a GUI application with a large textbox. Users would type into the textbox, and you could parse whatever was entered, and output to the same box (effectively emulated the Win32 Console).
This way whatever rules you want to set for how the console behaves is completely up to you.
I might be mistaken is saying this, but I believe that the Win32 Console from XP onward works exactly like this, and it just listens for output on stdout; there shouldn't be any reason you can't do the same.
Hope this was helpful.
I have a Win32 application written in C that can have its console output via printf() redirected to a log file.
It would be nice if I could have my app. detect if it had been started with or without a redirect '>'.
Any ideas?
Tom, thanks for the input.
I did some experiments and found this works for me..
fpost_t pos ;
fgetpos (stdout, & pos) ;
When an application's output is being redirected to a file, fgetpos() sets 'pos' to zero. It makes sense since its freshly opened stderr for you. EDIT: Actually, the value returned may be a positive integer if text has already been redirected to the log/file. So in your code you'd have something like "if (pos >= 0) bfRedirected = TRUE ;"
When an application's output is not being redirected - it's going to the console device - not a file, so fgetpos() will set 'pos' to -1.
I think that pipes are blind by nature, so you don't have a built-in way to tell whether you're being redirected. Moreover, trying to find out what's happening behind an abstraction layer is a bad habit.
If you really need to control your output, add the log file name as a command line parameter!
That being said, you can make some smart guesswork to find out:
A program can query the shell command history to find out the most recent commands executed.
If you know the path to the logfiles, you can scan that directory and see if a file has been created or changed its size.
Benchmark writing speed when redirected and not redirected. This would work only if your system is ultra-stable, and environment condition won't change.
There may be a way to do this - a quick google yielded this hit that might give you the hint in the right direction.
Method from Microsoft: WriteConsole fails if it is used with a standard handle that is redirected to a file. If an application processes multilingual output that can be redirected, determine whether the output handle is a console handle (one method is to call the GetConsoleMode function and check whether it succeeds).
AFAIK the answer to this is that you can't. Output redirection works by simply reading the stream of output from a given program and redirecting it to another pipe / stream. The design of file / streams is such that the writer is ignorant of the reader to the point that you shouldn't know you are being read.
Even detecting that there was a reader would be of no use because there is one in the normal case. The console is reading the output of your program and displaying it to the screen already.