I want to run gdb (GNU debugger) in Screen virtual terminal and grep the output in real time on adjacent Screen view.
How do I implement this arrangement? Normal pipe just redirects the output. I'm also curious how to bind Screen view (^A + c) to an existing process for IO.
EDIT:
I came up with the following solution. Created a named pipe with mkfifo pipe and executed gdb program | tee pipe in pty1. This will duplicate the output to a pipe. In pty2 I executed less -f pipe | grep foo to print the lines of interest.
I'm sure there have to be simpler way for such a trivial task though.
EDIT2:
The method mentioned above seems somewhat buggy. Gdb doesn't print anything to it's console unless something actually reads from the FIFO. Why is that? Also, when I try this method with my own program, which simply printf HelloWorld to stdout, nothing is printed in neither view.
EDIT3:
I figured out it's intentional that Tee blocks if nobody actually reads from the pipe. A matter of synchronization. Still I wonder, how is the original program able to read the input from keyboard even Tee controls now the terminal window. Or is it so that terminal input goes to stdin of original program and output to stdout of Tee?
You don't have to start your program out of gdb. Just start it in one screen pane, and determine the pid (use top, pgrep, ps).
In the other pane you start the gdb session:
gdb <path_to_program> <pid>
This way you have a terminal to control gdb and a terminal to use the program you are debugging, both have their own input and outputs.
The only condition is that the program runs long enough for you to attach the debugger to the process. An easy way to do that is to make it wait for input at the beginning. You could also make it print its pid.
Related
My bash script looks something like this
mpiexec ./fortran_bin |& tee text_file
wait
./process_output_files
My MPI-based Fortran program makes several synchronous system calls with call exec_cmd(cmd,wait=.true.).
My problem is that handle_output_files only waits for fortran_bin to finish, but some system commands (cmd) are not yet done, and this messes up my output files.
How do I make process_output_files wait for cmd to finish?
NOTES
I'm not sure where best to solve this problem (if there is a solution):
within Fortran, with MPI, within Bash ...
cmd is of the form cat out_{1..n} > out && rm -f out_{1..n}.
I would like it to run synchronously (wait=.false.), because cmd can be time-consuming, and unrelated to the rest of the Fortran program.
The wait line in the bash script seems to have no effect.
I suppose you could ask the equivalent question for a C/C++ program that calls system(some_script).
But I can only find question about waiting within a C/C++ program, if the same program needs the result of the called command (e.g., here and here).
From the notes above, looks like the sub command inherit the stdout/stderr on the calling process, and do not leave any background processes behind.
If those assumptions are true, you can impose a wait until there is no more output coming from the fortran_bin and it's children by piping the output into cat (or similar). The cat program will not terminate until all 'fortran_bin' children (that did not redirect stderr) will finish
mpiexec ./fortran_bin 2>&1 3>&1 | cat
Possible to use tee (or other similar programs) instead of cat
I want to invoke mpg123 from PHP (using exec) and monitor the diagnostic output generated by the program while it is running.
I have been searching the Internet and cannot find any way to see the redirected output of a command line program while it is running.
Instead, the output file is always written out AFTER the process finishes, but I need to access the output while it still running, hence my question.
Testing with:
mpg123.exe http://148.251.184.14:8192/stream | tee.exe streaming.txt
... file streaming.txt` is always empty while running the exe.
[Editors note: and so it would be, mpg123 sends diagnostic output to stderr].
Also, I tested this:
mpg123.exe http://148.251.184.14:8192/stream > streaming.txt
... and still no luck, because again, file streaming.txt is always empty while mpg123 is still running.
[Editor's note: of course, for the same reason as above, the command should be:
mpg123.exe http://148.251.184.14:8192/stream 2> streaming.txt
But still you see nothing in file streaming.txt until the program terminates.
end note]
Is there a way to do this? Seems to be a hard nut or not even possible...
Thank you for any help.
PS:
Using static binary from: https://mpg123.de/download/win64/1.25.10/
Tee.exe: https://sourceforge.net/projects/unxutils/files/unxutils/current/
You could, for example, get tail from GnuWin32 (it's in package coreutils). Then:
In one command prompt window run tail -F output-file. This will initially sit there because there is no output-file yet. Let it sit.
In another command prompt window run your-command > output.file.
In the first command prompt window tail will display the contents of output-file as it is generated.
Note 1: The program your-command may buffer its output, so that it written in chunks. Some programs have options to minimize output buffering, for example sed -u or grep --line-buffered.
Note 2: tail works as fast as it can, but console output is quite slow on Windows. It is perfectly possible for a program to generate output much faster than tail can display it.
I have tested this procedure with dir /s C:\ > Ls-lR.txt and tail Ls-lR.txt.
The quirks of MPG123
The specific program which the querent wants to monitor is MPG123. This program:
Does not normally write to standard output, and it actually closes stdandard output unless it wants to write WAV data.
Writes diagnostic messages to standard error, but only if standard error is not redirected or the option -v is given.
So...
Open a command prompt window and type tail -F mpg123.out. Since there is no file named mpg123.out, tail will sit and wait. Let it wait.
C> tail -F MPG123.out
Open a second command prompt window, and run mpg123
Redirecting stdandard error to mpg123.out, and
With the option -v.
C> mpg123.exe 2>MPG123.out -v "\path\to\the\music\file.mp3"
In the first window, watch the diagnostic messages of MPG123.
I have decided to delete my original answer and post a new one, because although the old one was factually correct it didn't answer the question very well. Now that I understand what the OP is actually doing, I can answer this properly.
The issue is actually very simple. Most programs, especially command line programs, on most platforms contain logic to detect if stdout or stderr has been redirected to a file (> file) or a pipe (e.g. | tee). This logic is usually actually buried in the runtime library so programs get it for free, which is why they pretty much all do it, and I'm sure that's true of mpg123 which is a relatively simple beast. What I say below will apply to almost any program.
Now, what this logic does is to decide whether or not to buffer output to stdout / stderr (it may make a different decision for each one). If output is going directly to the console (or, in Unix, the terminal) then it is not buffered at all (or maybe just on a per-line basis). Everything is sent out pretty much as soon as the program generates it.
If, on the other hand, output is redirected then mpg123 detects this and writes the data out in chunks (often 4k chunks), and if the total amount of output generated while the program is running is smaller than the size of the buffer then you won't see anything in the output file or pipe until the program terminates, at which point the buffer is flushed and the file closed (so you see it then, as the OP noted).
Now, knowing all that, we can explain the behaviour that the OP observes when running mpg123. This is not in fact down to any intricate juggling that mpg123 might do with file handles and the change in behaviour when you add in -v is just a side-effect. What you see is a direct result of the different buffer strategy used when the output is redirected.
So, using the binary linked to by the OP, this command:
mpg123 http://148.251.184.14:8192/stream
Generates the following output on the console straightaway (because nothing is buffered):
High Performance MPEG 1.0/2.0/2.5 Audio Player for Layers 1, 2 and 3
version 1.25.10; written and copyright by Michael Hipp and others
free software (LGPL) without any warranty but with best wishes
Directory: http://148.251.184.14:8192/
Playing MPEG stream 1 of 1: stream ...
ICY-NAME: Chroma Metal
ICY-URL: http://chromaradio.com
MPEG 1.0 L III cbr128 44100 j-s
ICY-META: StreamTitle='Avantasia - The Seven Angels';
It then goes on to play the stream though the sound card, which takes quite a while. The above information is written to stdout (and mpg123 always writes diagostic information to stdout).
This command, however, behaves differently, because the output is buffered (note the redirection of stdout):
mpg123 http://148.251.184.14:8192/stream 2>x.txt
As noted by the OP, this just creates a zero length file while the stream is playing, because the total amount of diagnostic output fits in mpg123s internal buffer so it just stays there until the program terminates, at which point the output duly turns up in the file for the reason given above.
And finally, this command, with the -v parameter added in:
mpg123 -v http://148.251.184.14:8192/stream 2>x.txt
does generate some output in x.txt while the program is running because the buffer fills up with the extra diagnostic information that the -v flag generates and at that point mpg123 has to write it to disk. The -v flag means verbose. That's where the extra output comes from.
Please note though that when you do this the data in the file is still always some way behind (because the next buffer-full is building up and won't be output until it's full), so while adding -v might get you what you want (or at least some of it), it hasn't changed the underlying problem. You can see this quite clearly if you run the above command in one console window and tail -F x.txt in another. When you do that, nothing shows up for the first 5 seconds or so. Then some (partial) output appears, and so it goes on.
So I hope that clears things up. Windows and Unix behave pretty much the same in this regard. I will edit the OP's question to make it a little less confusing. It's a bit untidy at the moment.
Perhaps the "tee" already on the machine could be used. I do not have you mpg123.exe executable, so I cannot test it.
powershell -NoProfile -Command "& mpg123.exe [StreamURL] | Tee-Object -FilePath .\streaming.txt"
Edit
Based on the information from #AlexP that mpg123.exe is writing to stderr, I would try:
powershell -NoProfile -Command "& mpg123.exe [StreamURL] 2>&1 | Tee-Object -FilePath .\streaming.txt"
Usually if I want to print the output of a command and in addition capture that output in a file, tee is the solution. But I'm making a script using a utility which seems to have a special behaviour. It's the wps wireless assessment tool bully.
If I run a bully command as normal (without tee), the output is shown in a standard way, step by step. But If I put the pipe at the end to log like this | tee "/path/to/my/logfile" the output on the screen freezes. It shows nothing until the command ends. And after ending, it shows everything in one shot (not step by step) and of course it puts the output also in the log tee file.
An example of bully command: bully wlan0mon -b 00:11:22:33:44:55 -c 8 -L -F -B -v 3 -p 12345670 | tee /root/Desktop/log.txt
Why? Not sure if it only happens with bully or if there are other programs with the same behaviour.
Is there another way to capture the output into a file having the output on screen in real time?
What you're seeing is full buffering vs line buffering. By default, when stdout is writing to a tty (i.e. interactive) you'll have line buffering, vs the default of being fully buffered otherwise. You can see the setvbuf(3) man page for a more detailed explanation.
Some commands offer an option to force line buffering (e.g. GNU grep has --line-buffered). But that sort of option is not widely available.
Another option is to use something like expect's unbuffer command if you want to be able to see the output more interactively (at the cost of depending on expect, of course).
is there a way to make stdin and stderr visible in unix bash/zsh/whatever?
Maybe turn the stderr output to red or something like that.
It is always a pain if you are figuring out why you can't parse the output from command x. This often drives me crazy. Then I do the 2> thing but at this point 30 minutes are already gone...
-Timo
I usually just do
command | grep '.*'
I have set $GREP_COLORS to
ms=01;32:mc=01;32:sl=:cx=:fn=35:ln=32:bn=33:se=36
which means the stdout is green while stderr stays black (or white).
Here is my solution a tiny little dylib from github: https://github.com/sickill/stderred
You can use stderred to automagically colorize stderr. It's a shared library that intercepts certain stream functions, so works with any program that uses those functions to write to your terminal.
I have a program (that I did not write) which is not designed to read in commands from a file. Entering commands on STDIN is pretty tedious, so I'd like to be able to automate it by writing the commands in a file for re-use. Trouble is, if the program hits EOF, it loops infinitely trying to read in the next command dropping an endless torrent of menu options on the screen.
What I'd like to be able to do is cat a file containing the commands into the program via a pipe, then use some sort of shell magic to have it switch from the file to STDIN when it hits the file's EOF.
Note: I've already considered using cat with the '-' for STDIN. Unfortunately (I didn't know this before), piped commands wait for the first program's output to terminate before starting the second program -- they do not run in parallel. If there's some way to get the programs to run in parallel with that kind of piping action, that would work!
Any thoughts? Thanks for any assistance!
EDIT:
I should note that my goal is not only to prevent the system from hitting the end of the commands file. I would like to be able to continue typing in commands from the keyboard when the file hits EOF.
I would do something like
(cat your_file_with_commands; cat) | sh your_script
That way, when the file with commands is done, the second cat will feed your script with whatever you type on stdin afterwards.
Same as Idelic answer with more simple syntax ;)
cat your_file_with_commands - | sh your_script
I would think expect would work for this.
Have you tried using something like tail -f commandfile | command I think that should pipe the lines of the file to command without closing the file descriptor afterwards. Use -n to specify the number of lines to be piped if tail -f doesn't catch all of them.