format command line/shell output for readability - shell

Example: man -k ls
Output: A LOT of text, so much that I can only read the last 20 lines.
I don't want information on how to scroll up through the output.
I would like to know, if possible, how to format/control the output so that only the first 20 lines are shown, then, when I press enter/scroll down, the next 20 lines are shown.
This way I can read all the output at my own pace. The output waits for me to tell it to continue. Is there a simple command for this?
Notice: This isn't a text file I'm outputting (I think), its just standard output, and way too much, so much so that it is unreadable except for the last 20 lines.

Can you just pipe the output to less or more? Or redirect the output to files and then go through them after the output is generated?
E.g. To redirect stdout to a file:
prompt> command -args > output.txt
More information on redirecting stdout and stderr can be found here:
http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-3.html

man -k ls | less
Found the answer, literally, right after I posted this question...
Apparently, the "| less" uses pipelines to make any command have scrolling output. I got this info from another site through a Google search.

Related

Redirecting output of command "systat -ifstat" to a file adding special chars in FreeBsd

I am trying to redirect outputs of systat -ifstat and systat -vmstat to a file and When I open that file, lot of special chars are added to a file like below
(B)0[?1049h[1;39r[m[4l[H[2J[1;21H/0 /1 /2 /3 /4 /5 /6 /7 /8 /9 /10
[68DLoad Average
[11DInterface[4;27HTraffic[4;49HPeak[4;69HTotal[2;21H[5;13H1/1 in[6C0.000 KB/s[5;46H0.000 KB/s[5;66H260.087 MB[6;18Hout 0.000 KB/s[6;46H0.000 KB/s[6;66H205.319
The command I am using to redirect to a file is below:
systat -ifstat 1 > text.txt
Can someone guide me to get rid of these special chars.Help is appreciated.
Keep in mind that systat(from the man page) display system statistics in a screen-oriented fashion using the curser screen display library, therefore in the try to get something like a screenshot, this partially works:
systat -ifstat | tee /tmp/output
to see the output like when using the command you will need to do cat /tmp/output, otherwise you will see all the shell escape characters.
You could also try script:
$ script
Script started, output file is typescript
$ systat -ifstat
next, you exit systat ctrl+c
$ exit
exit
Script done, output file is typescript
This will create a file named typescript but the output is not as clear as when using tee. (still, haven't found a way of how to render properly typescript within csh)

tee output not appearing until cmd finishes

Usually if I want to print the output of a command and in addition capture that output in a file, tee is the solution. But I'm making a script using a utility which seems to have a special behaviour. It's the wps wireless assessment tool bully.
If I run a bully command as normal (without tee), the output is shown in a standard way, step by step. But If I put the pipe at the end to log like this | tee "/path/to/my/logfile" the output on the screen freezes. It shows nothing until the command ends. And after ending, it shows everything in one shot (not step by step) and of course it puts the output also in the log tee file.
An example of bully command: bully wlan0mon -b 00:11:22:33:44:55 -c 8 -L -F -B -v 3 -p 12345670 | tee /root/Desktop/log.txt
Why? Not sure if it only happens with bully or if there are other programs with the same behaviour.
Is there another way to capture the output into a file having the output on screen in real time?
What you're seeing is full buffering vs line buffering. By default, when stdout is writing to a tty (i.e. interactive) you'll have line buffering, vs the default of being fully buffered otherwise. You can see the setvbuf(3) man page for a more detailed explanation.
Some commands offer an option to force line buffering (e.g. GNU grep has --line-buffered). But that sort of option is not widely available.
Another option is to use something like expect's unbuffer command if you want to be able to see the output more interactively (at the cost of depending on expect, of course).

How to get rid of bash control characters by evaluating them?

I have an output file (namely a log from screen) containing several control characters. Inside the screen, I have programs running that use control characters to refresh certain lines (examples would be top or anything printing progress bars).
I would like to output a tail of this file using PHP. If I simply read in that file and echo its contents (either using PHP functions or through calling tail, the output is messy and much more than these last lines as it also includes things that have been overwritten. If I instead run tail in the command line, it returns just what I want because the terminal evaluates the control characters.
So my question is: Is there a way to evaluate the control characters, getting the output that a terminal would show me, in a way that I could then use elsewhere (e.g., write to a file)?
#5gon12eder's answer got rid of some control characters (thanks for that!) but it did not handle the carriage return part that was even more important to me.
I figured out that I could just delete anything from the beginning of a line to the last carriage return inside that line and simply keep everything after that, so here is my sed command accomplishing that:
sed 's/^.*\r\([^\r]\+\)\r\?$/\1\r/g'
The output can then be further cleaned using #5gon12eder's answer:
cat screenlog.0 | sed 's/^.*\r\([^\r]\+\)\r\?$/\1\r/g' | sed 's,\x1B\[[0-9?;]*[a-zA-Z],,g'
Combined, this looks exactly like I wanted.
I'm not sure what you mean by “evaluating” the control characters but you could remove them easily.
Here is an example using sed but if you are already using PHP, its internal regex processing functionality seems more appropriate. The command
$ sed 's,\x1B\[[0-9?;]*[a-zA-Z],,g' file.dat
will dump the contents of file.dat to standard output with all ANSI escape sequences removed. (And I'm pretty sure that nothing else is removed except if your file contains invalid escape sequences in which case the operation is ill-defined anyway.)
Here is a little demo:
$ echo -e "This is\033[31m a \033[umessy \033[46mstring.\033[0m" > file.dat
$ cat file.dat
# The output of the above command is not shown to protect small children
# that might be browsing this site.
$ reset # your terminal
$ sed 's,\x1B\[[0-9?;]*[a-zA-Z],,g' file.dat
This is a messy string.
The less program has some more advanced logic built in to selectively replace some escape sequences. Read the man page for the relevant options.

Switch from file contents to STDIN in piped command? (Linux Shell)

I have a program (that I did not write) which is not designed to read in commands from a file. Entering commands on STDIN is pretty tedious, so I'd like to be able to automate it by writing the commands in a file for re-use. Trouble is, if the program hits EOF, it loops infinitely trying to read in the next command dropping an endless torrent of menu options on the screen.
What I'd like to be able to do is cat a file containing the commands into the program via a pipe, then use some sort of shell magic to have it switch from the file to STDIN when it hits the file's EOF.
Note: I've already considered using cat with the '-' for STDIN. Unfortunately (I didn't know this before), piped commands wait for the first program's output to terminate before starting the second program -- they do not run in parallel. If there's some way to get the programs to run in parallel with that kind of piping action, that would work!
Any thoughts? Thanks for any assistance!
EDIT:
I should note that my goal is not only to prevent the system from hitting the end of the commands file. I would like to be able to continue typing in commands from the keyboard when the file hits EOF.
I would do something like
(cat your_file_with_commands; cat) | sh your_script
That way, when the file with commands is done, the second cat will feed your script with whatever you type on stdin afterwards.
Same as Idelic answer with more simple syntax ;)
cat your_file_with_commands - | sh your_script
I would think expect would work for this.
Have you tried using something like tail -f commandfile | command I think that should pipe the lines of the file to command without closing the file descriptor afterwards. Use -n to specify the number of lines to be piped if tail -f doesn't catch all of them.

How do I jump to the first line of shell output? (shell equivalent of emacs comint-show-output)

I recently discovered 'comint-show-output' in emacs shell mode, which jumps to the first line of shell output, which I find incredibly handy when looking at shell output that exceeds a screen length. The advantages of this command over scrolling with 'page up' are A) you don't have to scan with your eyes for the first line of the output B) you only have to hit the key combo once (instead of 'page up' a number of times which probably is not known beforehand).
I thought about ending all my commands with '| more' but actually this is not what I want since most of the time, I want to retain all output in the terminal buffer, and I usually want to see the end of the shell output first.
I use OSX. Is there a terminal app (on os x) and shell (on remote linux) combination equivalent (so I can do something similar without using emacs all the time - I know, crazy talk)? I normally use bash, but would be fine with switching shells just for this feature.
The way I do this sort of thing is by sending my output to a file and then watching the file as it is written. You still get the results of the command dumped to terminal history in real time and can still inspect the output's actual contents further after the fact (or in another terminal, etc...)
command > output &
tail -f output
head output
You could always do something in bash like this:
alias foo='!! | more'
which would make foo run the previous command with more. I'm not sure of any way to do exactly what you are suggesting.
If you're expecting a lot of output and don't want to run your command twice, you can use tee(1) to fork the output:
my-command | tee /tmp/my-command.log | less
This will pipe the output to a paginator (less), while simultaneously logging the output to a file (in this case, a file named /tmp/my-command.log). If you need to review the output after you've quit from less, you can just cat the log file instead of re-running the command.

Resources