In a C# console application for Windows, I'm am using the Windows Console API WriteConsoleOutput (via PInvoke) to write an entire buffer in a single operation to prevent flickering. This works fine.
Microsoft recommends using virtual terminal sequences to interact with the console. These sequences are great, as they offer much better output, such as colors, etc.
But, as I understand it, WriteConsoleOutput cannot be used with escape sequences (see CHAR_INFO).
My question is,
How can I use virtual terminal sequences to write to the console flicker-free?
I'd like to update different parts of the screen with different characters and colors. Doing this by chaining a lot of Console.Write() and Console.SetCursorPosition will cause a lot of flickering and reduce framerate.
What is the virtual terminal equivalent of writing an entire buffer?
I hate to answer my own question, but I have found a solution after a couple of days' experimenting.
The answer to this question:
What is the virtual terminal equivalent of writing an entire buffer?
There is none.
Not exactly.
I haven't found anything similar to WriteConsoleOutput, which renders a pre-populated buffer of an arbitrary size.
However, virtual terminals have the concept of alternate screen buffers, which can be created by CreateConsoleScreenBuffer and switched with SetConsoleActiveScreenBuffer.
Using that, I came up with a solution for my first question:
How can I use virtual terminal sequences to write to the console flicker-free?
Basically, what I do is I create a new buffer and then use WriteConsole to write VT escape sequences to the back buffer before I switch the buffers.
The thing I don't like about this solution is the call to WriteConsole. There will be a lot of them (writing characters/sequences one by one), or there will be few of them when writing long pre-escaped strings.
In order to test flickering, I created a single string with 120 x 30 characters, with each character given a 256-color. This produced a string of over 41'000 characters, which was used as input to WriteConsole.
This actually seems to work pretty well!
This is the best solution I have found so far. If you find a better one, please write your own answer here!
Related
I am writing a small shell in Rust on Linux as an exercise and I wanted to implement command history as well as cursor moving (i.e. moving back the cursor to edit a typo in the command).
I did not find a way in the Rust standard library to handle arrow key events, but I found the Termion crate which handles key events.
However, handling key events with Termion means entering "raw mode" for stdout, which overrides "legacy" functionalities, as described in this article about the crate:
Without raw mode, you cannot write a proper interactive TTY application. Raw mode gives you complete control over the TTY:
It disables the line buffering: As you might notice, your command-line application tends to behave like the command-line. The programs will first get the input when the user types \n. Raw mode makes the program get the input after every key stroke.
It disables displaying the input: Without raw mode, the things you type appear on the screen, making it insufficient for most interactive TTY applications, where keys can represent controls and not textual input.
It disables canonicalization of the output: For example, \n represents “go one cell down” not “break the line”, for line breaks \n\r is needed.
It disables scrolling.
I find this solution a bit overkill, as I want to retain most of the "legacy" I/O functionalities. Is there another way to handle arrow key input, or will I need to use the raw mode?
There are several crates that provide line editing features for interactive programs. Here are a few that I found by searching crates.io for "readline" (the name of a C library):
rustyline seems to be the most popular on crates.io.
liner
linefeed
linenoise-rust is a set of Rust bindings to the linenoise library written in C.
I haven't used any of them, so this list is not a recommendation. Take a look at a few of them and choose one that suits your needs.
It is my understanding that by default, Character is Latin_1, Wide_Character is UCS-2, and Wide_Wide_Character is UCS-4, but that GNAT can have specified pragma Wide_Character_Encoding(UTF8); or -gnatW8 and that those characters and their strings will be UTF-8 encoded instead.
At least on Linux and FreeBSD, the results fit with my expectations. But on Windows the results are odd.
For either Wide or Wide_Wide variants, once a character moves beyond the ASCII set, I get a garbled mess. I beleive this is called emojibake by some. So I figured it was a codepage issue. After all, the default codepage in Windows, and therefore what the Console Host would load with, is 437 which isn't the UTF-8 codepage. chcp 65001 and now instead of the mess of extra characters, there's an immediate exception raised ADA.IO_EXCEPTIONS.DEVICE_ERROR : a-ztexio.adb:1295. Looking at where the exception occurred, it seems to be in the putc binding of fputc(). But this is Standard_Output, shouldn't an EOF never happen?
Is there some kind of special consideration Windows needs? How can I get UTF-8 output?
edit:
I tried piping the output into a text file. The supposed UTF-8 encoded program still generates emojibake in the file. Not sure why this would immediately throw an exception in the console though.
So then I tried directly opening and writing to a file instead of the console/pipe. Oddly this works exactly as it should. The text is completely correct.
I've never seen this kind of behavior with any other language, so it should still be possible to get proper UTF-8 at the console, right?
The deficiency so many others, not just here, describe in the Windows Console Host has either been fixed or never existed in the first place. Based on this document, I feel it was probably always very misunderstood. Windows doesn't treat the console like files, and it's easy to fall into that trap.
Using this very straight forward code, along with what Windows needs and expects behind the scenes...
It correctly produces the following, as long as either pragma Wide_Character_Encoding(UTF8); or -gnatW8 is used.
Piping the output of this test program into a file works as it should. Similarly, piping the output of this test program into another program works as it should. And also similarly, taking the file from piped output, and piping it into another program works as it should.
Full UTF-8 behavior as one would expect under Linux, on Windows.
What needs to be done is twofold. In the package initializer, the Console Host needs to be told what it's working with, which can be done like this.
Character output is then done through fputwc. According to MS Docs fputc should never be used for UNICODE on Windows, which is part of the problem GNAT has. String output and character/string input is all similar.
Based on others comments and some further research to confirm, I'm pretty sure this is a deficiency of the Windows Console Host.
edit: don't listen to this
My program constantly outputs text to the terminal. I need the user input line to always stay below this output; if the user types something, it shouldn't jump over the output.
Is there any Ruby library that can provide this functionality?
There could not be any ruby library providing this functionality since none of the known terminals support splitting the areas inside themselves.
This is the terminal issue: they are not capable of dividing their internal window into two parts.
What you can do, though, it to use two different shells for input and for output with IO#popen.
I've read most of the manual and am slowly getting my head around the things I need to make major-modes, etc. I've not ran into anything that explains the loop/cycle that Emacs goes through to apply the major mode (or minor-mode even).
For example: I type if while in go-mode and suddenly if is syntax-highlight. I know that just typing common letters amounts to self-insert-command. So how does emacs then react to the change in the buffer unless either self-insert-command fires and event or just changing the buffer fires and event?
W.r.t syntax highlighting, this is triggered by any change to the buffer, no matter which command is used. To do this, the package taking care of keeping the highlighting up-to-date (typically jit-lock on behalf of font-lock) uses after-change-functions. See C-hv after-change-functions RET and also check the corresponding documentation in the Emacs Lisp reference manual (reachable from the "Help" menu).
Relative Emacs newbie here, just trying to adapt my programming workflow to fit with emacs. So far I've discovered shell-pop and I'm quite enjoying on-demand terminals that pop up when needed for banging out the odd commands.
What I understand so far about Emacs is that shell is a "dumb" terminal that doesn't support any ansi control codes, and that makes it incompatible with things like ncurses that attempt to draw complex UI's on a terminal emulator. This is why you can't use less or top or similar in shell-mode.
However, I seem to be having trouble with ansi-term, it's not the be-all, end-all that it's cracked up to be. Sure, it has no problems running less or git log or even nano, but there are a few things that can't quite seem to display properly when they're running in an ansi-term, such as apt-get and nosetests. I'm not sure quite what the name is for it, but apt-get's output is characterised by live-updating what is displayed on the very last line, and then having unchanging lines of text scroll out above that line. It seems to be halfway between something like less and something dumber, like cat. Somehow ansi-term doesn't like this at all, and I get very garbled output, where it seems to output everything on one line only or just generally lose it's place and output things all over, randomly. In the case of nosetests, it starts off ok, but if any libraries spew out any STDERR, the output all goes to hell in a similar way.
With some fiddling it seems possible to fix this by mashing C-l and RET, but it's not always reliable.
Does anybody know what's going on here? Is there some way to fix ansi-term so that it can display everything properly? Or is there perhaps some other mode that I don't know about that is way better? Ideally I'd like something that "just works" as effortlessly as, eg, Gnome Terminal, which can run all of the above mentioned programs without a single hiccup.
Thanks!
I resolved this issue by commenting out my entire .emacs.el and then uncommenting and restarting emacs for every single line in the file. I discovered that the following line alone was responsible for the issue:
'(fringe-mode 0 nil (fringe))
(this line disables the fringes from inside custom-set-variables).
I guess this is a bug in Emacs, that disabling the fringe causes term-mode to garble it's output really badly whenever any output line exceeds $COLUMN columns.
Anyway, I don't really like the fringes much at all, and it seems I was able to at least disable the left fringe without triggering this issue:
(set-fringe-mode (cons 0 8))
Maybe apt-get does different things based on the $TERM environment variable. What happens if you set TERM=dumb? If that makes things work, then you can experiment with different values until you find one that supports enough features but still works.
Note that git 2.0.1 (June 25th, 2014) now better detects dumb terminal when displaying verbose messages.
That might help Emacs better display some of the messages received from git, but the fringe-mode bug reported above is certainly the main cause.
See commit 38de156 by Michael Naumov (mnaoumov)
sideband.c: do not use ANSI control sequence on non-terminal
Diagnostic messages received on the sideband #2 from the server side are sent to the standard error with ANSI terminal control sequence "\033[K" that erases to the end of line appended at the end of each line.
However, some programs (e.g. GitExtensions for Windows) read and interpret and/or show the message without understanding the terminal control sequences, resulting them to be shown to their end users.
To help these programs, squelch the control sequence when the standard error stream is not being sent to a tty.