I have a Matlab program that runs different unix commands fairly often. For this question let's assume that what I'm doing is:
unix('ls test')
It happens to me quite frequently that I accidentally press a key(like enter or the arrow keys) f.e. when I'm waking up my display from standby. In theory this shouldn't interfere with the unix command. Though unfortunately, Matlab will take this input and forward it right into the execution of the command. The above command then becomes something like this:
unix('ls te^[0Ast')
(Side note: ^[0A is the hex representation of the linefeed character)
Obviously, this will produce an error.
Does anyone have an idea how to work around this issue?
I was thinking that there might be a way to start Matlab with my script in a way that doesn't forward any user input from within the unix shell.
#!/bin/bash
matlab -nodisplay -nosplash -r "runMyScript();"
Can I somehow pipe the user-input somewhere else and isolate Matlab from any sort of input?
That is not very specific question, but let me try. I can see several options. I am assuming that matlab is text terminal application.
There is nohup(1) command. Since you use linux, chances are that there is non-posix version if it which says in it's man page: If standard input is a terminal, redirect it from /dev/null.
$ nohup matlab -nodisplay -nosplash -r "runMyScript();"
You can redirect /dev/null yourself
$ matlab -nodisplay -nosplash -r "runMyScript();" < /dev/null
But matlab can actually re-open it's stdin ignoring what you piped into it (for example ssh does that, you can't use echo password | ssh somewhere.
if you are running in graphics environment you may want to minimise the window, so that it does not receive any input. Probably not your case, you would figure out yourself :)
you may try to wake up by hitting "Ctrl", similar key or mouse
You may run matlab in screen(1) command disconnect from the screen or switch to different window. Screen is a program allowing you to create virtual terminals (similar to virtual desktops in GUI). If you haven't heard of screen, I suggest you to look at some tutorials. Googling for gnu screen tutorial seems to offer quite a few.
Related
One may want to use Bash on Windows in Task Scheduler or maybe as version-control hook scripts. Is it possible or supported?
If not, why? Is it a bug or a measure to prevent some issues?
Use #3d1t0r's solution, but also pipe to cat
wsl bash -c "man bash | cat" # noninteractive; streams the entire manpage to the terminal
wsl bash -c "man bash" # shows me the first page, and lets me scroll around; need to hit `q` to exit
If interactive mode is fine, bash -c is often superfluous
wsl man bash # same behavior as `wsl bash -c "man bash"`
Context (what in the world is "interactive" vs "non-interactive" invocation?):
The example above might not make it entirely clear, but man is changing its behavior based on what it's connected to.
In "interactive mode", man lets me scroll around, so that I can read the page at a comfortable reading pace.
In noninteractive mode, man dumps the entire manpage to the console, giving me no time to read anything.
"But wait," I hear you ask, "isn't man catting the man page because you asked it to? I see it right there--man bash | cat"
No, man has no idea what cat is. It just gets hints about whether STDOUT is connected to an interactive terminal.
Here's a different example, that consistently cats:
wsl bash -c "echo hey | grep --color e" # colors 'e' red
wsl bash -c "echo hey | grep --color e | cat" # colors disappear, what gives?
Now both examples are streaming their output, but the second one is defiantly ignoring my --color flag.
The common thread here is man and grep both behave appropriately depending on whether they think their output is going to be read by a human piped away somewhere.
Other common commands that auto-detect interactivity include ls and git. Usually the behavior change will involve output paging or colors (other variations exist).
paging is nice for humans, because humans generally can't read at the speed of streamed output.
paging is bad for robots, because paging is a lot of protocol overhead when you can just consume buffered streams. I mean seriously, why are humans so slow and chatty?
colors are nice for humans, because we like additional visual cues to aid visual distinction.
colors are bad for streaming to file, because your file will be full of ansi color code garbage, that most text editors don't display nicely.
Automatic behavior switching based on whether STDOUT is connected to an interactive terminal makes all these use cases usually "just work".
Restating the Original Question
In my use case and #bahrep's use case, interactive mode can be especially bad for unsupervised scripts (e.g. as launched by Task Scheduler). I am guessing #bahrep's scheduled runs hung on less getting invoked and waiting for human input.
For some reason, wsl-driven scripts launched from the task-scheduler give underlying scripts the wrong hints--they hint that the final output is attached to an interactive terminal.
Ideally, wsl would know from the windows side of the execution environment whether it is getting invoked interactively or not, and pass along the proper hint. Then I could just run wsl [command]. Until that happens, I'll need to use wsl bash -c "[command] | cat" as a workaround.
If I'm understanding your question correctly, the -c option is what you're looking for. It allows you to directly invoke a Linux command.
For example, to open the man page for bash (perhaps in order to find out about the -c option):
bash -c "man bash"
Note: You can leave off the quotes if you escape any spaces (e.g. bash -c man\ bash), but it's often easier to just use the quotes, as the first unescaped space will lose the rest of your command.
e.g. bash -c man bash will be interpreted the same as bash -c man.
I run my Matlab scripts from bash in the following way:
matlab -nodesktop -nosplash -nodisplay -r "matlabfun()" &> log
The resulting log file starts and ends with a strange character sequence that in less appears as: ESC[?1hESC=. Do you know what this is caused by?
I can reproduce your error. From this table I would assume, that Matlab forces the cursor to be in the application.
I have now idea where else it should be in a bash session, maybe it is a leftover from the graphical version or other platforms. You can just ignore it.
Do not have the chance to verify it myself, but this website suggests that it is because bash tries to help you.
The solution is to set the TERM to invalid entry:
TERM=vt444
This question already has answers here:
How to trick an application into thinking its stdout is a terminal, not a pipe
(9 answers)
Closed 5 years ago.
Various bash commands I use -- fancy diffs, build scripts, etc, produce lots of color output.
When I redirect this output to a file, and then cat or less the file later, the colorization is gone -- presumably b/c the act of redirecting the output stripped out the color codes that tell the terminal to change colors.
Is there a way to capture colorized output, including the colorization?
One way to capture colorized output is with the script command. Running script will start a bash session where all of the raw output is captured to a file (named typescript by default).
Redirecting doesn't strip colors, but many commands will detect when they are sending output to a terminal, and will not produce colors by default if not. For example, on Linux ls --color=auto (which is aliased to plain ls in a lot of places) will not produce color codes if outputting to a pipe or file, but ls --color will. Many other tools have similar override flags to get them to save colorized output to a file, but it's all specific to the individual tool.
Even once you have the color codes in a file, to see them you need to use a tool that leaves them intact. less has a -r flag to show file data in "raw" mode; this displays color codes. edit: Slightly newer versions also have a -R flag which is specifically aware of color codes and displays them properly, with better support for things like line wrapping/trimming than raw mode because less can tell which things are control codes and which are actually characters going to the screen.
Inspired by the other answers, I started using script. I had to use -c to get it working though. All other answers, including tee, different script examples did not work for me.
Context:
Ubuntu 16.04
running behavior tests with behave and starting shell command during the test with python's subprocess.check_call()
Solution:
script --flush --quiet --return /tmp/ansible-output.txt --command "my-ansible-command"
Explanation for the switches:
--flush was needed, because otherwise the output is not well live-observable, coming in big chunks
--quiet supresses the own output of the script tool
-c, --command directly provides the command to execute, piping from my command to script did not work for me (no colors)
--return to make script propagate the exit code of my command so I know if my command has failed
I found that using script to preserve colors when piping to less doesn't really work (less is all messed up and on exit, bash is all messed up) because less is interactive. script seems to really mess up input coming from stdin even after exiting.
So instead of running:
script -q /dev/null cargo build | less -R
I redirect /dev/null to it before piping to less:
script -q /dev/null cargo build < /dev/null | less -R
So now script doesn't mess with stdin and gets me exactly what I want. It's the equivalent of command | less but it preserves colors while also continuing to read new content appended to the file (other methods I tried wouldn't do that).
some programs remove colorization when they realize the output is not a TTY (i.e. when you redirect them into another program). You can tell some of those to use color forcefully, and tell the pager to turn on colorization, for example use less -R
This question over on superuser helped me when my other answer (involving tee) didn't work. It involves using unbuffer to make the command think it's running from a shell.
I installed it using sudo apt install expect tcl rather than sudo apt-get install expect-dev.
I needed to use this method when redirecting the output of apt, ironically.
I use tee: pipe the command's output to teefilename and it'll keep the colour. And if you don't want to see the output on the screen (which is what tee is for: showing and redirecting output at the same time) then just send the output of tee to /dev/null:
command| teefilename> /dev/null
I'm working on a project that uses several languages:
SQL for querying a database
Perl/Ruby for quick-and-dirty processing of the data from the database and some other bookkeeping
Matlab for matrix-oriented computations
Various statistics languages (SAS/R/SPSS) for processing the Matlab output
Each language fits its niche well and we already have a fair amount of code in each. Right now, there's a lot of manual work to run all these steps that would be much better scripted. I've already done this on Linux, and it works relatively well.
On Linux:
matlab -nosplash -nodesktop -r "command"
or
echo "command" | matlab -nosplash -nodesktop
...opens Matlab in a "command line" mode. (That is, no windows are created -- it just reads from STDIN, executes, and outputs to STDOUT/STDERR.) My problem is that on Windows (XP and 7), this same code opens up a window and doesn't read from / write to the command line. It just stares me blankly in the face, totally ignoring STDIN and STDOUT.
How can I script running Matlab commands on Windows? I basically want something that will do:
ruby database_query.rb
perl legacy_code.pl
ruby other_stuff.rb
matlab processing_step_1.m
matlab processing_step_2.m
# etc, etc.
I've found out that Matlab has an -automation flag on Windows to start an "automation server". That sounds like overkill for my purposes, and I'd like something that works on both platforms.
What options do I have for automating Matlab in this workflow?
matlab -nosplash -nodesktop -r "command"
works on Windows as well. Yes, it opens another window, but it's not a problem. I run it in a batch mode from Java wrapper on Tomcat server and there were no issues. Put all commands into a script file, don't forget to close the session with exit command, and use -r flag. Also you can find -noFigureWindows and -wait parameters useful. And it works on both Windows and Unix. You can use platform-specific flags, if some are not supported they will be ignored.
Start MATLAB program (Windows platforms)
There is also a way to hide matlab window with C library. See engSetVisible.
I'm writing an Expect script and am having trouble dealing with the shell prompt (on Linux). My Expect script spawns rlogin and the remote system is using ksh. The prompt on the remote system contains the current directory followed by " > " (space greater-than space). A script snippet might be:
send "some command here\r"
expect " > "
This works for simple commands, but things start to go wrong when the command I'm sending exceeds the width of the terminal (or more precisely, what ksh thinks is the width of the terminal). In that case, ksh does some weird horizontal scrolling of the interactive command line, which seems to rewrite the prompt and stick an extra " > " in the output. Naturally this causes the Expect script to get confused and out of sync when there appears to be more than one prompt in the output after executing a command (my script contains several send/expect pairs).
I've tried changing PS1 on the remote system to something more distinctive like "prompt> " but a similar problem arises which indicates to me that's not the right way to solve this.
What I'm thinking might help is the ability for the script to tell Expect that "I know I'm properly synchronised with the remote system at this point, so flush the input buffer now." The expect statement has the -notransfer flag which doesn't discard the input buffer even if the pattern does match, so I think I need the opposite of that.
Are there any other useful techniques that I can use to make the remote shell behave more predictably? I understand that Expect goes through a lot of work to make sure that the spawned session appears to be interactive to the remote system, but I'd rather that some of the more annoying interactive features (such as the horizontal scrolling of ksh) be turned off.
If you want to throw away all output Expect has seen so far, try
expect -re $
This is a regexp match on $ which means the end of the input buffer, so it will just skip everything received so far. More details at the Expect man page.
You could try "set -o multiline" or COLUMNS=1000000 (or some other suitably large value).
I have had difficulty with ksh and Expect in the past. My solution was to use something other than
ksh for a login shell.
If you can change the remote login to other than ksh (using the chsh command or editing /etc/passwd) then you might try this with /bin/sh as the shell.
Another alternative is to tell KSH that the terminal is a dumb terminal - disallow it from doing any special processing.
$ export TERM=""
might do the trick.