Garbage/control characters observed in output from Paramiko when get_pty=True for Windows2019/2022 (CygwinOpenssh) [duplicate] - windows

I am using Python's Paramiko library to SSH a remote machine and fetch some output from command-line. I see a lot of junk printing along with the actual output. How to get rid of this?
chan1.send("ls\n")
output = chan1.recv(1024).decode("utf-8")
print(output)
[u'Last login: Wed Oct 21 18:08:53 2015 from 172.16.200.77\r', u'\x1b[2J\x1b[1;1H[local]cli#BENU>enable', u'[local]cli#BENU#Configure',
I want to eliminate, [2J\x1b[1;1H and u from the output. They are junk.

It's not a junk. These are ANSI escape codes that are normally interpreted by a terminal client to pretty print the output.
If the server is correctly configured, you get these only, when you use an interactive terminal, in other words, if you requested a pseudo terminal for the session (what you should not, if you are automating the session).
The Paramiko automatically requests the pseudo terminal, if you used the SSHClient.invoke_shell, as that is supposed to be used for implementing an interactive terminal. See also How do I start a shell without terminal emulation in Python Paramiko?
If you automate an execution of remote commands, you better use the SSHClient.exec_command, which does not allocate the pseudo terminal by default (unless you override by the get_pty=True argument).
stdin, stdout, stderr = client.exec_command('ls')
See also What is the difference between exec_command and send with invoke_shell() on Paramiko?
Or as a workaround, see How can I remove the ANSI escape sequences from a string in python.
Though that's rather a hack and might not be sufficient. You might have other problems with the interactive terminal, not only the escape sequences.
You particularly are probably not interested in the "Last login" message and command-prompt (cli#BENU>) either. You do not get these with the exec_command.
If you need to use the "shell" channel due to some specific requirements or limitations of the server, note that it is technically possible to use the "shell" channel without the pseudo terminal. But Paramiko SSHClient.invoke_shell does not allow that. Instead, you can create the "shell" channel manually. See Can I call Channel.invoke_shell() without calling Channel.get_pty() beforehand, when NOT using Channel.exec_command().
And finally the u is not a part of the actual string value (note that it's outside the quotes). It's an indication that the string value is in the Unicode encoding. You want that!

This is actually not junk. The u before the string indicates that this is a unicode string. The \x1b[2J\x1b[1;1H is an escape sequence. I don't know exactly what it is supposed to do, but it appears to clear the screen when I print it out.
To see what I mean, try this code:
for string in output:
print string

Related

Configuration verification

ssh to an IP supplied via user input
Inject show running-config command
Search the output of the command and look for specific parameters like ports, QoS, VTY lines, SMTP settings, IP helpers etc.
Output only if the predefined parameters are not in place
If I store the output of the ssh run in a variable, is there a method I can use for parsing through it and just go with endless if, elsif, and else statements? How would that look?
It can be in Python bash or Perl, doesn't matter to me really but I don't know how to structure the thing and then fill in the gaps and expand the script.
Here's where I imagine starting from
use Net::SSH2
use File::Slurp;
print "\nEnter command list filename: ";
my $commandlist = <STDIN>;
chomp($commandlist);
my #commandfile = read_file($commandlist, chomp => 1);
my $commandsize = #commandfile;
How would I store the output of the commands in a variable or a temporary file and parse through it with conditions?
This is a very broad question, and it is unclear exactly what is puzzling you
The documentation for
Net::SSH2
and
Net::SSH2::Channel
describes clearly how to open a channel, send commands to it, and receive the response
You ask how you could store the results of the command in a variable, but it would be very awkward to do anything else. Again, the documentation describes this clearly
I suggest that you try writing some working code and experiment with it. It will be much easier to help you when you have a specific question

How curses differs Enter from Ctrl+Enter and other key strokes with overlapping escape sequences?

As far as I can see, in Konsole both CTRL+ENTER and ENTER look like byte 13 in the stdin of the running app. But when I run mc which obviously uses ncurses lib, pressing CTRL+ENTER inserts the name of the file into the command line while ENTER opens the file. How is it implemented? I tried to look into sources, but they are totally unreadable for me.
mc (midnight commander) doesn't use ncurses for input, but may use it for output. Essentially it is looking for specific character sequences.
mc makes little use of the terminfo database, essentially only to check on the xterm mouse- and alternate-screen features.
In principle, it could read the user-defined capabilities from the ncurses terminfo database (see for example ncurses trapping extended keys (Control-left, Shift-Function etc)), but it does not.
Since you are looking at sources, see the source of mc, in lib/tty/key.c, which contains tables which mc uses as a set of predefined keys. Doing it that way "works" when mc is configured to use slang, for instance, though it has the drawback that it's hardcoded and may not actually match your terminal.
However - as I said, mc does its own input. Further down in key.c, you may see a chunk in get_modifier() ifdef'd with HAVE_TEXTMODE_X11_SUPPORT. Inside that is a call which ultimately goes to XQueryPointer, which mc uses to find the current state of the modifier keys — if it happens to be running in an X display, and if the feature is enabled. You are probably seeing that.

Interact with console program via batch script

I want to start a console program via batch script. Starting the console program works fine. I am starting it via call xxx.exe para para. The problem is that the console program wants an input like that after it is started.
call xxx.exe para para
please type in password:_
Is it possible to make the input of the password from the batch script.
Whether you are using batch or bash, as it seemed originally, you could try this simple piping:
echo YourPassword| program.exe parameters...
Note that if it is indeed a batch script, it is vital to make sure there's no extra space between your password and the |, or it will be passed along with the password, as part of the password. In bash, if I'm not much mistaken, such a space would be disregarded (or maybe it would only be so if you enclosed the echoed string in quotation marks, I'm not entirely sure).
Anyway, the above doesn't always work, as some programs implement password reading in a way that disregards the input stream piped from another command.
You tagged your question "Windows" and "Batch" and asked about "batch" in the question. The answer to that question is: Yes, use set like this:
set /p password=please type in password:
If you're really asking about 'bash' shell, you should re-tag your question (and change the text).

Windows 'choice' command messing up Ruby 'gets' method

Open up irb and
type gets. It should work fine.
Then try system("choice /c YN") It should work as expected.
Now try gets again, it behaves oddly.
Can someone tell me why this is?
EDIT: For some clarification on the "odd" behavior, it allows me to type for gets, but doesn't show me the characters and I have to press the enter key twice.
Terminal input-output handling is dark and mysterious art. Anyone trying to make colorized output of bash work in windows PowerShell via ssh knows that. (And various shortcutting habits like Ctrl+Backspace only make things worse.)
One of the possible reasons for your problem is special characters handling. Every terminal out there can type characters in number of different modes, and it parses its own output in search for certain character sequences in order to toggle states.
F.e. here one can find ANSI escape code sequences, one of possible supported standards among different kind of terminals.
See there Esc[5;45m? That will make all the following output to blink on magenta background. And there is significantly more stuff like that out there.
So, the answer to your question taken literally is — your choice command messes something with output modes using special escape sequences, and ruby's gets breaks in that quirk special mode of terminal operation.
But more useful will be the link to HighLine gem documentation. Why one might want to implement platform-specific and obtrusive behavior when it is possible to implement the same with about 12 LOC? All the respect for the Gist goes to botimer, I've only stumbled into his code using search.

How would one implement bash-like tab completion?

I'm trying to determine how the system prints characters to standard input -- that is, how it prints characters which the user can delete and which are considered input if the user hits "Enter."
I happen to be using C, but I would be very surprised if the solution were language-dependent.
Thanks for any insights! : D
As iny says, bash uses readline for its input. The source is available here, and there's a file called complete.c.
To answer your question, I don't think they're actually printed to standard input. Readline contains some kind of buffer for the contents of the line the user is editing, and completion prints into this. When the user presses enter, the contents of the buffer are sent to whatever program wanted to read a line, and in the case of bash, passed along into standard input. (Readline doesn't do this - other programs which use readline might simply store the value into a string for later use.)
Several people have pointed out that bash uses readline, which is true, but I think what you're really asking is how is it able to see what you've typed before you hit enter.
The answer is that ttys (ie: terminals) can be switched into "raw mode", where input processing of the terminal is disabled, and then you'll see every character as it comes in. This also disables automatic echoing of typed characters.
See this guide on Reading a single character from a file or a terminal for more info.
It uses readline library to handle the input and readline provides the history and the completion.
To actually implement completion, access to the keyboard input handling is needed. The completion must be able to modify the buffer used by it. After that it is just about looking at the current input and checking what completions is found. The actual completion logic can work in many ways.
Here's a C snippet that implements tab completion via readline:
http://github.com/rupa/el

Resources