I'm writing an Expect script and am having trouble dealing with the shell prompt (on Linux). My Expect script spawns rlogin and the remote system is using ksh. The prompt on the remote system contains the current directory followed by " > " (space greater-than space). A script snippet might be:
send "some command here\r"
expect " > "
This works for simple commands, but things start to go wrong when the command I'm sending exceeds the width of the terminal (or more precisely, what ksh thinks is the width of the terminal). In that case, ksh does some weird horizontal scrolling of the interactive command line, which seems to rewrite the prompt and stick an extra " > " in the output. Naturally this causes the Expect script to get confused and out of sync when there appears to be more than one prompt in the output after executing a command (my script contains several send/expect pairs).
I've tried changing PS1 on the remote system to something more distinctive like "prompt> " but a similar problem arises which indicates to me that's not the right way to solve this.
What I'm thinking might help is the ability for the script to tell Expect that "I know I'm properly synchronised with the remote system at this point, so flush the input buffer now." The expect statement has the -notransfer flag which doesn't discard the input buffer even if the pattern does match, so I think I need the opposite of that.
Are there any other useful techniques that I can use to make the remote shell behave more predictably? I understand that Expect goes through a lot of work to make sure that the spawned session appears to be interactive to the remote system, but I'd rather that some of the more annoying interactive features (such as the horizontal scrolling of ksh) be turned off.
If you want to throw away all output Expect has seen so far, try
expect -re $
This is a regexp match on $ which means the end of the input buffer, so it will just skip everything received so far. More details at the Expect man page.
You could try "set -o multiline" or COLUMNS=1000000 (or some other suitably large value).
I have had difficulty with ksh and Expect in the past. My solution was to use something other than
ksh for a login shell.
If you can change the remote login to other than ksh (using the chsh command or editing /etc/passwd) then you might try this with /bin/sh as the shell.
Another alternative is to tell KSH that the terminal is a dumb terminal - disallow it from doing any special processing.
$ export TERM=""
might do the trick.
Related
As part of a bigger script I'm using print -z ls to have zsh's input buffer show the ls command. This requires me to manually press enter to actually execute the command. Is there a way to have ZSH execute the command?
To clarify, the objective is to have a command run, keep it in history, and in case another command is running it shouldn't run in parallel or something like that.
The solution I've found is:
python -c "import fcntl, sys, termios; fcntl.ioctl(sys.stdin, termios.TIOCSTI, '\n');
I'm not sure why, but sometimes you might need to repeat the command 2 times for the actual command to be executed. In my case this is happening because I send a process to the background, although this still doesn't make much sense because that process is sending a signal back to the original shell (triggering a trap) which actually calls this code.
In case anyone is interested, this was my goal:
https://gist.github.com/alexmipego/89c59a5e3abe34faeaee0b07b23b56eb
I want to send commands in the ADB shell itself as if i had done the following in cmd.
>adb shell
shell#:/ <command>
I am using python 3.4 on a windows 7 OS 64bit machine. I can send one-line shell commands simply using subprocess.getoutput such as:
subprocess.getoutput ('adb pull /storage/sdcard0/file.txt')
as long as the adb commands themselves are recognized by ADB specifically, such as pull and push, however there are other commands such as grep that need to be run IN the shell, like above, since they are not recognized by adb. for example, the following line will not work:
subprocess.getoutput ('adb shell ls -l | grep ...')
To enter the commands in the shell I thought I needed some kind of expect library as that is what 'everyone' suggests, however pexpect, wexpect, and winexpect all failed to work. they were written for python 2 and after being ported to python 3 and my going through the .py files by hand, even those tweaked for windows, nothing was working - each of them for different reasons.
how can i send the input i want to the adb shell directly?
If none of the already recommended shortcuts work for you you can still go the 'regular' way using 'subprocess.Popen' for entering commands in the adb shell with Popen:
cmd1 = 'adb shell'
cmd2 = 'ls -l | grep ...'
p = subprocess.Popen(cmd1.split(), stdin=PIPE)
time.sleep(1)
p.stdin.write(cmd2.encode('utf-8'))
p.stdin.write('\n'.encode('utf-8'))
p.stdin.flush()
time.sleep(3)
p.kill()
Some things to remember:
even though you import subprocess you still need to invoke subprocess.Popen
sending cmd1 as a string or as items in a list should work too but '.split()' does the trick and is easier on the eyes
since you only specidfied you want to enter input to the shell you only need stdin=PIPE. stdout would only be necessary if you wanted to receive output from the shell
time.sleep(1) isn't really necessary, however since many complained about input issues being faster or slower in python 2 vs 3 consider maybe using it. 'they' might have been using versions of 'expect' that need the shell's reply first. this code also worked when i tested it with simply swapping out and in the process with time.sleep(0)
stdin.write will return an error if the input is not encoded properly. python's default is unicode. entering by binary did not work for me in my tests like this "b\ls ..." but .encode() worked. dont forget the endline!
if you use .encode() there is a worry that the line might not get sent properly, so to be sure it might be good to include a flush().
time.sleep(3) is completely uneccesary, but if your command takes a long time to execute (eg a regressive search through the entire device piped out to a txt file on the memory card) maybe give it some extra time before killing anyhting.
remember to kill. if you didnt kill it, the pipe may remain open, and even after exiting the test app on the console the next commend still went to the shell even though the prompt appearsed to be my regular cmd prompt.
Amichai, I have to start with pointing out that your own "solution" is pretty awful. And your explanation makes it even worse. Doing all those unnecessary things just because you do not understand how shell (here I mean your PC's OS shell, not adb) command parsing works.
When all you needed was just this one command:
subprocess.check_output(['adb', 'shell', 'ls /storage/sdcard0 | grep ...']).decode('utf-8')
I ask because I recently made a change to a KornShell (ksh) script that was executing. A short while after I saved my changes, the executing process failed. Judging from the error message, it looked as though the running process had seen some -- but not all -- of my changes. This strongly suggests that when a shell script is invoked, the entire script is not read into memory.
If this conclusion is correct, it suggests that one should avoid making changes to scripts that are running.
$ uname -a
SunOS blahblah 5.9 Generic_122300-61 sun4u sparc SUNW,Sun-Fire-15000
No. Shell scripts are read either line-by-line, or command-by-command followed by ;s, with the exception of blocks such as if ... fi blocks which are interpreted as a chunk:
A shell script is a text file containing shell commands. When such a
file is used as the first non-option argument when invoking Bash, and
neither the -c nor -s option is supplied (see Invoking Bash), Bash
reads and executes commands from the file, then exits. This mode of
operation creates a non-interactive shell.
You can demonstrate that the shell waits for the fi of an if block to execute commands by typing them manually on the command line.
http://www.gnu.org/software/bash/manual/bashref.html#Executing-Commands
http://www.gnu.org/software/bash/manual/bashref.html#Shell-Scripts
It's funny that most OS'es I know, do NOT read the entire content of any script in memory, and run it from disk. Doing otherwise would allow making changes to the script, while running. I don't understand why that is done, given the fact :
scripts are usually very small (and don't take many memory anyway)
at some point, and shown in this thread, people would start making changes to a script that is already running anyway
But, acknowledging this, here's something to think about: If you decided that a script is not running OK (because you are writing/changing/debugging), do you care on the rest of the running of that script ? you can go ahead making the changes, save them, and ignore all output and actions, done by the current run.
But .. Sometimes, and that depends on the script in question, a subsequent run of the same script (modified or not), can become a problem since the current/previous run is doing an abnormal run. It would typically skip some stuff, or sudenly jump to parts in the script, it shouldn't. And THAT may be a problem. It may leave "things" in a bad state; particularly if file manipulation/creation is involved.
So, as a general rule : even if the OS supports the feature or not, it's best to let the current run finish, and THEN save the updated script. You can change it already, but don't save it.
It's not like in the old days of DOS, where you actually have only one screen in front of you (one DOS screen), so you can't say you need to wait on run completion, before you can open a file again.
No they are not and there are many good reasons for that.
One of the things you should keep in mind is that a shell is not an interpreter even if there are some similarities. Shells are designed to work with a stream of commands. Either from the TTY ,a PIPE, FIFO or even a socket.
The shell reads from its resource line by line until a EOF is returned by the kernel.
The most shells have no extra support for interpreting files. they work with a file as they would work with a terminal.
In fact this is considered to be a nice feature because you can do interesting stuff like this How do Linux binary installers (.bin, .sh) work?
You can use a binary file and prepend shell scripts. You can't do this with an interpreter. because it parses the whole file or at least it would try it and fail. A shell would just interpret it line by line and doesnt care about the garbage at the end of the file. You just have to make sure the execution of the script gets terminated before it reaches the binary part.
I'm thinking a hypothetical CMDOUTPUT would be useful:
locate -r 'regexp...' # locate finds a file: /myfile.
# Shell puts `/myfile' string into CMDOUTPUT
vim $CMDOUTPUT # No need to run locate again as with: vim `!!`
The locate command above is just an example. I want the output saved for all commands that I run so that if I need it I can access it quickly. (The output should still be printed by the command to stdout.) I don't want to do
CMDOUTPUT="$(...)"
or
command | tee /tmp/cmdoutput
or anything else that I have to do because that's more typing for me at the prompt for everything that I run: I want the shell to do it all in the background. Again, to make it clear: I am casually typing commands away and decide "Oh, I want to use the output of that last command in this command, let me just retrieve it...". Can I tell the shell to store the output somehow so that I can retrieve it.
If there's no option for it, is there some way that I can implement it that is as close to invisible as it can be, meaning exit codes from the command are not lost (...and that's all I can think of, but I'm sure there are other subtleties) etc. I'm primarily thinking of zsh, but answers for any shell would be useful.
I found a solution, not sure if this is exactly what you're looking for. But it should provide a start :)
zsh | tee log >&1
I want to write a program that can be run as a backgrounded task (i.e. my-thing &), and that will beautifully provide relevant output when necessary.
However, with background tasks that print to the terminal, there's always this annoying little progression:
bash-3.2$ my-thing &▏
bash-3.2$ my-thing &
bash-3.2$ -- Hello from my-thing!
-- now let me start doing what I do best ...
▏
… notice that we now have no prompt before the cursor, not to mention that there's an ugly erraneous prompt where my program spat out output while a prompt had already been printed.
Thus, I want three things, only one of which I don't have any idea how to go about doing:
Detect whether my program is being run in the background, and alter my output/operation accordingly (i.e. print out less useless information; queue further information to only be output when the program is foregrounded again; etceteras)
Clear out the prompt (if it already exists) when suddenly printing information to a terminal we've been backgrounded in
Inform the terminal after I've finished whatever I'm printing to the terminal for, somehow forcing it to re-draw a new prompt below the content I've added
The third is the only one I have absolutely no idea how to go about; though I've got some inklings on the other two, any links or suggestions there would be appreciated as well.
Equally appreciated, even, would be links to any programs (any language/environment) which already have modes that operate this; as I could attempt to reverse-engineer/spelunk their implementation.
This works for me:
echo 'sleep 1; echo; echo hi' > /tmp/hi
chmod +x /tmp/hi
zsh
/tmp/hi &
asdfsadf # type whatever, here.
After the 'hi' gets echo'd, zsh knows to redraw your prompt with the 'asdfasdf' intact.
So, no big surprise: zsh being better than bash.