extended ascii chars and ansi in the mac osx terminal app - macos

i wanted to create some shellscripts that display pretty ansi colored graphics for osx but unfortunately i find just very little information about that topic.
osx seems to use monaco 10 as its default console font. is there some way to find out all displayable characters for this font?
osx terminal runs in UTF-8 per default as far as i found out (can somebody confirm this?)
is there a way to show or enter the extended ascii characters on osx (how it was done on windows/dos with alt gr+entering the digits)
thanks!

In Bash, you should be able to do:
for i in {32..255}; do printf "$i "\\$(($i/64*100+$i%64/8*10+$i%8))"\n"; done | column
to get a table of ASCII characters.

Extended chars ar usually hused thorugh option+character key and shift+option+character key. I beleive terminal is utf-8 by default. Monaco should support the compelete character map. If you open up FontBook you can gain access to the char map there.
Additionally you can customize a lot of this type of thing from the preferences pane for terminal.

Related

Im using vim to check man page, and there is some weird character,how to make it normal?

I'm trying using
man ascii | vi -
to check ascii manual with vim,
then there are some weird character like '^H'
weird character in vim
how to make it normal?
EDIT:
i think this is due to some wrong config of my vim, or terminal,
because i tried some other machine,and it works fine,any body help?
What you're seeing here is the man page convention for typesetting bold and underlined characters.
The ^H character is the "backspace" or BS character, which you can find in ascii(7) as ASCII 8.
Displaying as ^H is a convention to indicate "Control-H", the caret being a symbol for the "Control" key, since that's a key combination that generate this control character (note that "H" is the eighth letter, also it's on the same row as BS in the ASCII chart at the man page.)
Also note that ^H here is a single character and not the separate ^ and H characters. If you move along that line with l motion repeatedly, you'll see Vim skips the Hs. Vim might also display them in a different color to make it clear they're special characters.
The effect you're seeing here is called overstruck characters and it comes from the days when computers would be connected to line printers rather than screens. Man pages were created early in the Unix days and are typeset with a program named nroff (which has roots that predate even Unix) that uses these sequences to encode its text decorations, even to this day.
So one way to make text bold (or, print it stronger) was to print a letter, back out one space, then print it again. In your example, N^HNA^HAM^HME^HE is being used to print the word NAME in bold.
Typically, you'll use a man pager program that knows how to handle those sequences. For instance, less has specific support for overstruck text and is able to recognize those sequences using backspace (or carriage return, also possible to use it for that) and convert into bold or underline text using terminal escape sequences which actually achieve those effects in a terminal.
If you're using a different pager, often you'll use col -b to strip your text from those sequences. (See col(1) for details.) Note that col -b doesn't produce terminal escape sequences around overstruck text, it simply removes all overstriking, leaves the text plain, so any text reader can display it.
(In that sense, man ascii | col -b | vi - is one possible way around your issue, but not the best one.)
Vim doesn't support overstruck text or terminal escape sequences in text it receives as input. So the approach used to use Vim as a man pager is to strip all overstriking (same as col -b does) and then use Vim's syntax highlighting feature to mark man page sections etc. (So instead of seeing the original markings typeset into the man pages, those are all being stripped and you're actually seeing Vim syntax configuration for man pages.)
This is accomplished by the manpager.vim script, which is shipped by default with Vim and which registers a :MANPAGER command that you can access as vim +MANPAGER -.
You'll see that part of what that does is to remove overstriking, same as col -b does. It also sets the file type correctly and enables syntax highlighting, setting up Vim as a suitable viewer for man pages.
I think this is due to some wrong config of my vim, or terminal, because I tried it in some other machine and it works fine there.
I'd say this is most likely due to the man pages being typeset differently between the two systems.
Man pages themselves and the pipelines used to generate them from the source code are not very uniform and you'll find differences across operating systems or even Linux distributions.
It's possible that on your other system all man pages get stripped using col -b at creation time, or that man will strip them when piping them to something that's not what's set in $MANPAGER, or even that they don't have anything marked as bold or underline in their sources.
Hard to tell exactly which (your reference to working fine on another machine lacks details to tell), but this would be my bet.
In short: To use Vim as a pager for man pages, instead of piping them to Vim, set it up so that man will run vim as a pager itself.
In Vim, see :help manpager.vim which has instructions on how to set this up.
In short, all you need is to add this line to your ~/.bash_profile:
export MANPAGER="vim -M +MANPAGER -"
After setting that up and logging in again, all you need is:
$ man ascii
And man will launch vim in the right mode for you.

iTerm2: delete line?

I'm trying to map ⌘+Delete (backspace) to delete to the beginning of the line (like it works in browsers and text editors) in iTerm2 and I'm unable to find a working escape code for it. I tried 1K (^[1K) based on what I read in Wikipedia. It just prints a "K".
Edit: I found Ctrl+U. Now to find out how to map it. Maybe Hex code 21 (U being 21st letter), so 0x15?
I got it. I have no idea why Hex Code mappings in iTerm2 produce the associated Ctrl+key mappings, but they do. No idea what 0x00 means, either, as it's not assigned to A as might be expected. (though I do believe Unix has its own conventions relating to treating null bytes -- we have e.g. xargs accepting a null byte delimiting format from find for example -- It would be neat if we can bind this to a hotkey with iTerm2)
I was able to find that Ctrl+U does nearly the exact task I want (it deletes the entire line rather than deleting only what is before cursor, but whatever... Ctrl+Y as a bonus can bring it all back). Then I curiously saw that I had hex codes 0x1 and 0x5 mapped to ^A and ^E respectively, for my Cmd+Left and Cmd+Right... so 0x15 is for ^U!
For Mac OS, most editor share the common shortcut ⌘ + Delete: delete to start of the line, in iTerm2 we can switch to this key configuration
Mapping hex code 0x15 to ⌘ + ←Delete in most shells deletes the entire line (content to the left and right of the cursor). While sometimes not as compatible, I find that mapping:
⌘+←Delete to Send Hex Codes:
0x18 0x7f
performs the desired functionality. If you're running ZSH, you'll likely also need to add this to your .zshrc file:
$ echo 'bindkey "^X\\x7f" backward-kill-line' >> ~/.zshrc
as by default ZSH doesn't map backward-kill-line to anything.
Furthermore, you can also delete everything to the right of your cursor by mapping:
⌘+fn+←Delete or ⌘+Delete→ to Send Hex Codes:
0x0b
I wrote a comprehensive guide to adding most of OSX's standard keybinding to your terminal here
In OSX, ⌥+⌫ and ⌘+⌫ are the shortcuts for deleting a word and deleting a line respectively. ⌘+ ← and ⌘+ → are for going to the beginning and end of lines. By default, iTerm2 isn't configured this way, and there are a lot of misleading guides online. The following is what I've found to work on my machine.
Open the preferences (⌘+,) and go to the Keys tab.
Add a global shortcut key, and just type in your shortcut
In the Action dropdown, select Send Hex Code
The hex codes for...
Deleting a word: 0x17.
Deleting a line: 0x15.
Moving to the beginning of the line: 0x01.
Moving to the end of the line: 0x05.
Just open a new tab, and it should work!
Here is a screenshot, for clarity:
I hope this may help you
map ⌥ <- Delete to Send Hex Codes: 0x1B 0x08
I had test for it, and it is correct.
18.09.2013 update
this delete one word, not a line.
As pointed ^U deletes the line. You can easily remap the command by using Better Touch Tool.
It also has cool features for automation, mouse, pad and keyboard mapping. Also includes a window feature for smart borders.
On iTerm2 you can set the exact same shortcuts of your OS. In this case, as the default shortcut to delete a line on Mac OS is ⌘+Delete (backspace), you can do so.
To set the default keybindings of iTerm you have to:
Go to Preferences (or ⌘+,)
Profiles
Keys
Key Mappings
Presets...
Select the "Natural text editing" option
The location has changed. In order to enable natural editing go to:
Preferences -> Profile -> Keys -> Load Presets... -> and select Natural Text Editing
For more info and tweeks go https://blog.arturofm.com/install-iterm-terminal-emulator-on-macos/

vim powerline in windows 7

I am trying to install VIM Powerline on window, I found a comment which supposed to fix how it looks in Windows, but encoding of the comment is broken by itself...
The link to the blog post.
I had the same problem on windows. As an alternative to editing “vim-powerline-develop\autoload\Powerline\Functions.vim”, you can also add this to your vimrc:
let g:Powerline_mode_V=”V·LINE
let g:Powerline_mode_cv=”V·BLOCK”
let g:Powerline_mode_S=”S·LINE”
let g:Powerline_mode_cs=”S·BLOCK”
I don't know how to type the cdot operator in my keyboard (sorry for the name, I guess I am too much into Latex ...).
So, what am I supposed to do to make VIM behave under windows?
To get the middle dot, the easiest and pragmatic way would be just pasting it from the clipboard.
For direct text entry, you can enter by the character encoding's numerical value: <C-v>183, see :help i_CTRL-V_digit. On Windows, you often need to use <C-q> instead of <C-v>.
Vim also has a feature called digraphs for quick entry of special characters. The middle dot would be <C-k>.M.
For any of these to work, your text encoding must support the non-ASCII character. Best use a value of
:set encoding=utf-8
and ensure that your ~/.vimrc is UTF-8 encoded.

Control Characters and How OS/TextEditors interprets them?

I was going thru some content about control characters especially newline character(will focus on this).After going thru
http://en.wikipedia.org/wiki/Control_characters, got to know that \n is the line character in unix
while it is \r\n in windows. Now i got the question how OS comes into picture when iterpreting
ASCII Codes becoz i was under impression when we type any given character on keyboard, any OS send the same
bits and editor interprets that bit and display the corresponding character. Looks like this understanding is
wrong, Because different bit is sent in case of unix(\n) and windows(\r\n) when we press ENTER(new line terminator).As per
new understanding if we press ENTER on diff OS(say unix and windows),different bits are sent to editor and its
responsibilty of text editor to show the typed stuff in new line keeping the underlying OS in picture.Please let me
know if my understanding is correct as this will help me to understand other basics also?
Next question is if above is correct, what can be the reason different OS treat some control characters differently
when they treat all other characters equally? Is it becoz specific bits are already reserved in specific OS?
How an application treats keyboard input varies a bit, actually. When you press return the application is under no obligation to actually generate LF or CR+LF anywhere. E.g. it might decide to just end the current paragraph object and start a new one (e.g. in a word processor). If it's a Windows text editor then it will probably just write CR+LF into the file, while on Unix it just writes an LF.
They keyboard itself is very, very far removed from things you see on the screen or even on the disk. This goes through scan codes, keyboard layouts and other transformations before it ends up as text or markup somewhere.

How do shell text editors work?

I'm fairly new at programming, but I've wondered how shell text editors such as vim, emacs, nano, etc are able to control the command-line window. I'm primarily a Windows programmer, so maybe it's different on *nix. As far as I know, it's only possible to print text to a console, and ask for input. How do text editors create a navigable, editable window in a command line environment?
By using libraries such as the following which, in turn, use escape character sequences
NAME
ncurses - CRT screen handling and optimization package
SYNOPSIS
#include
DESCRIPTION
The ncurses library routines give the user a terminal-independent
method of updating character screens with reasonable optimization. This
implementation is ‘‘new curses’’ (ncurses) and is the approved replacement
for 4.4BSD classic curses, which has been discontinued.
[...snip....]
The ncurses package supports: overall screen, window and pad
manipulation; output to windows and pads; reading terminal input; control
over terminal and curses input and output options; environment query
routines; color manipulation; use of soft label keys; terminfo capabilities;
and access to low-level terminal-manipulation routines.
Short answer: there are libraries for it (like curses, slang).
Longer answer: doing things like jumping around with the cursor or changing colors are done by printing special character sequences (called escape-secquences, because they start with the ESC character).
Learning about ncurses might be a good starting point.
There is an old protocol called vt100 based on a "VT100" terminal. It used codes starting with escape to control cursor position, color, clearing the screen, etc.
It's also how you get colored prompts.
Google VT100 or "terminal escape codes"
edit: I Googled it for you: http://www.termsys.demon.co.uk/vtansi.htm
You will also notice this if you type "edit" in a Windows command line console. This "feature" is not unique to unix-like systems, though the concepts for manipulating the windows console in that way are quite different to in unix.
On Unix systems, a console window emulates an ancient serial terminal (usually a VT100). You can print special control characters and escape sequences to move the cursor around, change colors, and do other special effects. There are libraries to help handle the details; ncurses is the most popular.
On Windows, the [Win32 Console API](http://msdn.microsoft.com/en-us/library/ms682073(VS.85%29.aspx) provides similar functionality, but in a rather different manner.
Type "c:\winnt\system32\edit" or "c:\windows\system32\edit" at the command line, and you'll be shown a command line text editor.
People are mostly right about the ESC character being used to control the command screen, but some older programs also write characters directly to the memory space used by the Windows Command Line screen.
In order to control the command line window, you used to have to write your own windowing forms, entry box, menus, etc. You'd also have to wrap all that up in a big loop for handling events.
More Windows command line specific, the app typically calls DOS or BIOS functions that do the same. Sometimes ANSI command code support is available, sometimes it isn't (depending on exact MS OS version and whether or not it's configured to load it).

Resources