I'm fairly new at programming, but I've wondered how shell text editors such as vim, emacs, nano, etc are able to control the command-line window. I'm primarily a Windows programmer, so maybe it's different on *nix. As far as I know, it's only possible to print text to a console, and ask for input. How do text editors create a navigable, editable window in a command line environment?
By using libraries such as the following which, in turn, use escape character sequences
NAME
ncurses - CRT screen handling and optimization package
SYNOPSIS
#include
DESCRIPTION
The ncurses library routines give the user a terminal-independent
method of updating character screens with reasonable optimization. This
implementation is ‘‘new curses’’ (ncurses) and is the approved replacement
for 4.4BSD classic curses, which has been discontinued.
[...snip....]
The ncurses package supports: overall screen, window and pad
manipulation; output to windows and pads; reading terminal input; control
over terminal and curses input and output options; environment query
routines; color manipulation; use of soft label keys; terminfo capabilities;
and access to low-level terminal-manipulation routines.
Short answer: there are libraries for it (like curses, slang).
Longer answer: doing things like jumping around with the cursor or changing colors are done by printing special character sequences (called escape-secquences, because they start with the ESC character).
Learning about ncurses might be a good starting point.
There is an old protocol called vt100 based on a "VT100" terminal. It used codes starting with escape to control cursor position, color, clearing the screen, etc.
It's also how you get colored prompts.
Google VT100 or "terminal escape codes"
edit: I Googled it for you: http://www.termsys.demon.co.uk/vtansi.htm
You will also notice this if you type "edit" in a Windows command line console. This "feature" is not unique to unix-like systems, though the concepts for manipulating the windows console in that way are quite different to in unix.
On Unix systems, a console window emulates an ancient serial terminal (usually a VT100). You can print special control characters and escape sequences to move the cursor around, change colors, and do other special effects. There are libraries to help handle the details; ncurses is the most popular.
On Windows, the [Win32 Console API](http://msdn.microsoft.com/en-us/library/ms682073(VS.85%29.aspx) provides similar functionality, but in a rather different manner.
Type "c:\winnt\system32\edit" or "c:\windows\system32\edit" at the command line, and you'll be shown a command line text editor.
People are mostly right about the ESC character being used to control the command screen, but some older programs also write characters directly to the memory space used by the Windows Command Line screen.
In order to control the command line window, you used to have to write your own windowing forms, entry box, menus, etc. You'd also have to wrap all that up in a big loop for handling events.
More Windows command line specific, the app typically calls DOS or BIOS functions that do the same. Sometimes ANSI command code support is available, sometimes it isn't (depending on exact MS OS version and whether or not it's configured to load it).
Related
Is it possible to change the behavior of ⌥⌫ (option+delete) to include certain characters, like a period, as a word boundary?
For example, in some.variable, I would like ⌥⌫ to delete up to the period but it currently deletes the entire string. The screencast below shows the behavior using ⌥⇧←.
Screencast GIF
I've looked high and low for a solution to this issue since the Cocoa-era, as far as I can tell there isn't a way to customize the "Option-Delete" word-delimiter (word-boundary) list. In addition to my own experiments, I'm basing this conclusion largely on the enormous number of MacOS keybinding modifications that can be found at the following 3 URLs (from oldest to most recently updated):
Customizing the Cocoa Text System (circa 2006)
DefaultKeybinding.dict (circa 2016, some info previously at the URL "lri.me")
Keybindings -- BrettTerpstra.com (last updated 2018)
AFAIK, word-boundaries on MacOS default to whitespace and can't be changed. [This conclusion is invalid if there have been recent radical changes to MacOS text-input system].
It's no secret that many of the MacOS keyboard shortcuts are based upon eMacs-style 'keyboard-chording', as opposed to Vim-style 'modal-editing'. Indeed much of the Keybinding work cited above deals with remapping these chords. Vim of course is well-known for having a customizable delimiter list, it's the basis for differential word-movements between (for example) w, W and b and B (see vimhelp.org).
There have been a few extensions that have tried to bring 'Vim-style' editing to the MacOS, one of which is Karabiner. Karabiner (now re-written/updated to Karabiner-Elements) used to have a mode called "Ubiquitous_Vim". However without an OS-level 'hook' personal tests indicate that (for example) moving backwards via b and B works identically (i.e. there's no difference between a Vim word and a Vim WORD, see this StackOverflow post).
While I'm unable to find a way to modify the MacOS default 'word-boundary' list, if Vim-style editing does appeal to you, then you might try installing a project such as Vim-Anywhere, which enables you to 1. use a MacOS keyboard shortcut to open a Vim window whenever you need to enter text, and 2. will automatically copy your text to the MacOS System Clipboard when you close the Vim window. See this OSTechNix article: "How To Use Vim Editor To Input Text Anywhere" for more information.
Several *NIX commands, such as screen, man, vim and others, create a temporary canvas/screen/overlay in a shell environment. When such programs execute, they cover or hide whatever content was displayed in the terminal before — almost like a "full screen" mode, within the terminal window. When they terminate, however, they reveal or restore whatever had been on the terminal before.
In the example below, I create some filler text on the screen, then invoke man bash. The man page opens up and covers all other characters on the terminal display. When I close the man page, the characters that had been covered up are again shown.
Before
While an example full-screen program is running
After
I would expect that programs writing to stdout/stderr could accomplish the first step (replacing the content of the terminal with program-specific content), but then it would produce a ton of text that I could scroll through, and therefore couldn't do the second step: restoring the contents of the terminal. That means that somehow either the program memorizes the previous contents of the screen and re-outputs them (I doubt it?), or it creates some sort of sub-window within a terminal and something else keeps track of the previous contents of the terminal.
My Question
How can I accomplish that behavior in my own program and/or script?
Perhaps I should use curses/ncurses, tput, termcap/terminfo, or ANSI escape sequences?
Update:
This revised question is essentially the same as https://unix.stackexchange.com/questions/27941/show-output-on-another-screen-and-return-to-normal-when-done. (I hadn't found it when I had written this question despite lots of searching.) The difference is that my question is more general (any language) whereas that question is specific to Bash. The answers to both questions are essentially the same. If it's too similar to a question on another site, feel free to close it here for that reason.
How do these programs accomplish that behavior?
ANSI escape sequences. Try running this script:
#/bin/bash -
tput smcup
echo 'Hello world!'
sleep 3
tput rmcup
Using infocmp, you can see underlying sequences that create this overlaying effect, e.g:
$ infocmp -1 | grep 'rmcup\|smcup'
rmcup=\E[?1049l\E[23;0;0t,
smcup=\E[?1049h\E[22;0;0t,
is this behavior shell-dependent or system-dependent?
None, it depends on whether the terminal emulator supports save/restore operations.
I am trying to create a program which will have live updates from some data source. And I also want to wait for user input just like a normal terminal. Right now, whenever there is update, I will print the content and print the prompt message for input again which create something like this:
Enter command >
This is a live update message
Enter command >
This is a multi-line li......
......ve update message
Enter command > quit
Bye bye!
The problem is that for every live message I received, I will print it and the "Enter command >" will be displayed again again and again, which is not desired. I want the live update to be update on the main part of the terminal, while the "Enter command >" always stay at the bottom
The closest package I can found on Github is https://github.com/gizak/termui but most of the examples inside is trying to display text, gauge and graphs. So I am not quite sure how to get started.
Is there any package or example of the termui package to achieve this? Thank you.
With github.com/gizak/termui you're heading in the correct direction.
To understand why you can't get that
I want the live update to be update on the main part of the terminal, while the "Enter command >" always stay at the bottom
part sorted out, a little excursion to the history of computing is due. ;-)
The thing is, the mode your teminal emulator¹ works by default originated
in how computers would communicate to the operator in the era which predated
alphanumeric displays — they would print their responses using a line printer. Now think of it: a line printer works like this: it prints whatever is sent to it on a roll of paper. What was output, was output.
The new output always appears physically below the older.
When alphanumeric displays (screens) came into existence they
naturally continued to support this mode:
the line text to be output was rendered at the bottom of the screen
with the text above it scrolled upwards.
That's what you see in your typical terminal emulator all the time when you're working in the command line of a shell (such as bash) running by the emulator window.
This, default, work mode of a terminal is called "canonical" or "cooked".
Then came more advanced displays, for which it was possible to change
individual positions on the screen — identified by their column and
row numbers.
This changed the paradigm of how the information was output: the concept
of a so-called "full-screen application" was born.
Typical examples of them are text editors such as Vim and Emacs.
To support full-screen text output, terminals (and terminal emulators)
were adapted by implementing certain extensions to their protocols.
A full-screen application first requests the terminal to switch into another
mode called "raw", in which the terminal sends most of what is input by the
user directly to the program running on the terminal.
The program handles this input and orders the terminal where and what
to draw.
You can read this good summary
of the distinction between the both modes.
As you are supposedly suspecting by now, to be able to keep some block
of information at a certain fixed place of the terminal's text screen,
you want your program to be a full-screen program and use the terminal's
raw mode and its special commands allowing you to directly modify
text at certain character cells.
Now the problem is that different terminals (and terminal emulators)
have different commands to do that, so there exist libraries to isolate
the programs from these gory details. They rely on the special "terminal
information databases" to figure out what capabilities a terminal has
and how to make it do what the program asks.
See man terminfo for more background.
The most widely known such library (written in C) is called ncurses,
and there exist native solutions for Go with supposedly the most visible
one being github.com/nsf/termbox-go.
The github.com/gizak/termui makes use of termbox-go but for you it might
suffice to use the latter directly.
¹ Chances are very high you're not sitting at
a real hardware terminal
connected to a UNIX® machine but are rather working in a GUI application
such as GNOME Terminal or xterm or Termial.app etc.
These are not "terminals" per se but are rather
terminal emulators —
that is, pieces of software emulating a hardware terminal.
I am using macOS Sierra(unfortunately), and for example when I typed man curl to see what the -LSso flag does, it opens the man page in a editor which resembles vim slightly. I know its not Vim since it doesn't have :syntax on and you can't use :wq and so on.
The heading of the page says man(less).
I can't find what editor this might be and I'm learning vim and emacs configurations. Or maybe it's just a man page and no editor is used?
Less is a program similar to more (1), but which allows backward movement in the file as well as forward movement. Also, less does not have to read the entire input file before starting, so with large
input files it starts up faster than text editors like vi (1). Less uses termcap (or terminfo on some systems), so it can run on a variety of terminals. There is even limited support for hardcopy
terminals. (On a hardcopy terminal, lines which should be printed at the top of the screen are prefixed with a caret.)
Commands are based on both more and vi. Commands may be preceded by a decimal number, called N in the descriptions below. The number is used by some commands, as indicated.
-- Copied from less man page
There are several libraries like ncurses that assist in making command-line GUIs.
Simply put, how do they work?
My first thought was that ncurses intercepts all keyboard input, and draws each "frame" by outputting it line-by-line normally. Closer inspection, however, reveals that each new frame overwrites the previous one. How does it modify lines that have already been outputted? Furthermore, how does it handle color?
EDIT: The same question applies to anything with a "fancy" interface, like vim and emacs.
Text terminals have command sequences that do things like move the cursor to a particular position on the screen, insert characters, delete lines etc.
Each terminal type is different and has its own set of command sequences. ncurses has a databse (see terminfo for details)
Internally ncurses maintains 2 views of the screen: the current contents and what the screen should look like after the current pending changes are applied. Once the program requests a screen redraw, ncurses calculates an efficient way to update the screen to look like the desired view. The exact characters/command sequences output depend on what terminal type is in use.
curses (and ncurses, too, I think) works by moving the cursor around on the screen. There are control sequences to do such things. Take a look at the code again and you'll see them. These sequences are not ASCII control characters, they are strings starting with (umm...) ESC, maybe. Have a look here for a higher-level explanation.