I am using GDAL. in command prompt, i am doing
$ gdalinfo (my file location)
It works but because it is a huge file the command gives a lot of information. I am only interested in seeing what's near the beginning. The command prompt only allows scrolling up to the last 1000 or so lines of info (it must give about 100,000 lines or so). How can I do this?
This will depend on the OS and utilities it provides. I am assuming you are using a POSIX OS which support pipes and provides utilities such as less/more. The command in this case would be:
$ gdalinfo file.tif | less
If less is unavailable, you may have the more command installed. You can also save the output from gdalinfo into a file and look at the file later.
$gdalinfo tile.tif > output.txt
n
[Appended]
On Windows, I get a truncated response like this:
C:\Users\elijah>gdalinfo "C:\xData\GreeneCountyMo\ortho_1-1_1n_s_mo077_2010_1.sid" | more
(Use ENTER/RETURN to advance to the next line, and CTRL+C to "escape" when you're finished.)
Or I can do the outfile as well:
C:\Users\elijah>gdalinfo "C:\xData\ortho_1-1_1n_s_mo077_2010_1.sid" > "C:\xData\gdalinfo.txt"
If you are on a windows machine... What type of file are you using? Perhaps it contains a lot of ground control points, which you can skip using the -nogcp flag, or skip the metadata using the -nomd flag (see http://www.gdal.org/gdalinfo.html). Also, see --help-general; you might have the --debug flag set to on?
Related
I want to invoke mpg123 from PHP (using exec) and monitor the diagnostic output generated by the program while it is running.
I have been searching the Internet and cannot find any way to see the redirected output of a command line program while it is running.
Instead, the output file is always written out AFTER the process finishes, but I need to access the output while it still running, hence my question.
Testing with:
mpg123.exe http://148.251.184.14:8192/stream | tee.exe streaming.txt
... file streaming.txt` is always empty while running the exe.
[Editors note: and so it would be, mpg123 sends diagnostic output to stderr].
Also, I tested this:
mpg123.exe http://148.251.184.14:8192/stream > streaming.txt
... and still no luck, because again, file streaming.txt is always empty while mpg123 is still running.
[Editor's note: of course, for the same reason as above, the command should be:
mpg123.exe http://148.251.184.14:8192/stream 2> streaming.txt
But still you see nothing in file streaming.txt until the program terminates.
end note]
Is there a way to do this? Seems to be a hard nut or not even possible...
Thank you for any help.
PS:
Using static binary from: https://mpg123.de/download/win64/1.25.10/
Tee.exe: https://sourceforge.net/projects/unxutils/files/unxutils/current/
You could, for example, get tail from GnuWin32 (it's in package coreutils). Then:
In one command prompt window run tail -F output-file. This will initially sit there because there is no output-file yet. Let it sit.
In another command prompt window run your-command > output.file.
In the first command prompt window tail will display the contents of output-file as it is generated.
Note 1: The program your-command may buffer its output, so that it written in chunks. Some programs have options to minimize output buffering, for example sed -u or grep --line-buffered.
Note 2: tail works as fast as it can, but console output is quite slow on Windows. It is perfectly possible for a program to generate output much faster than tail can display it.
I have tested this procedure with dir /s C:\ > Ls-lR.txt and tail Ls-lR.txt.
The quirks of MPG123
The specific program which the querent wants to monitor is MPG123. This program:
Does not normally write to standard output, and it actually closes stdandard output unless it wants to write WAV data.
Writes diagnostic messages to standard error, but only if standard error is not redirected or the option -v is given.
So...
Open a command prompt window and type tail -F mpg123.out. Since there is no file named mpg123.out, tail will sit and wait. Let it wait.
C> tail -F MPG123.out
Open a second command prompt window, and run mpg123
Redirecting stdandard error to mpg123.out, and
With the option -v.
C> mpg123.exe 2>MPG123.out -v "\path\to\the\music\file.mp3"
In the first window, watch the diagnostic messages of MPG123.
I have decided to delete my original answer and post a new one, because although the old one was factually correct it didn't answer the question very well. Now that I understand what the OP is actually doing, I can answer this properly.
The issue is actually very simple. Most programs, especially command line programs, on most platforms contain logic to detect if stdout or stderr has been redirected to a file (> file) or a pipe (e.g. | tee). This logic is usually actually buried in the runtime library so programs get it for free, which is why they pretty much all do it, and I'm sure that's true of mpg123 which is a relatively simple beast. What I say below will apply to almost any program.
Now, what this logic does is to decide whether or not to buffer output to stdout / stderr (it may make a different decision for each one). If output is going directly to the console (or, in Unix, the terminal) then it is not buffered at all (or maybe just on a per-line basis). Everything is sent out pretty much as soon as the program generates it.
If, on the other hand, output is redirected then mpg123 detects this and writes the data out in chunks (often 4k chunks), and if the total amount of output generated while the program is running is smaller than the size of the buffer then you won't see anything in the output file or pipe until the program terminates, at which point the buffer is flushed and the file closed (so you see it then, as the OP noted).
Now, knowing all that, we can explain the behaviour that the OP observes when running mpg123. This is not in fact down to any intricate juggling that mpg123 might do with file handles and the change in behaviour when you add in -v is just a side-effect. What you see is a direct result of the different buffer strategy used when the output is redirected.
So, using the binary linked to by the OP, this command:
mpg123 http://148.251.184.14:8192/stream
Generates the following output on the console straightaway (because nothing is buffered):
High Performance MPEG 1.0/2.0/2.5 Audio Player for Layers 1, 2 and 3
version 1.25.10; written and copyright by Michael Hipp and others
free software (LGPL) without any warranty but with best wishes
Directory: http://148.251.184.14:8192/
Playing MPEG stream 1 of 1: stream ...
ICY-NAME: Chroma Metal
ICY-URL: http://chromaradio.com
MPEG 1.0 L III cbr128 44100 j-s
ICY-META: StreamTitle='Avantasia - The Seven Angels';
It then goes on to play the stream though the sound card, which takes quite a while. The above information is written to stdout (and mpg123 always writes diagostic information to stdout).
This command, however, behaves differently, because the output is buffered (note the redirection of stdout):
mpg123 http://148.251.184.14:8192/stream 2>x.txt
As noted by the OP, this just creates a zero length file while the stream is playing, because the total amount of diagnostic output fits in mpg123s internal buffer so it just stays there until the program terminates, at which point the output duly turns up in the file for the reason given above.
And finally, this command, with the -v parameter added in:
mpg123 -v http://148.251.184.14:8192/stream 2>x.txt
does generate some output in x.txt while the program is running because the buffer fills up with the extra diagnostic information that the -v flag generates and at that point mpg123 has to write it to disk. The -v flag means verbose. That's where the extra output comes from.
Please note though that when you do this the data in the file is still always some way behind (because the next buffer-full is building up and won't be output until it's full), so while adding -v might get you what you want (or at least some of it), it hasn't changed the underlying problem. You can see this quite clearly if you run the above command in one console window and tail -F x.txt in another. When you do that, nothing shows up for the first 5 seconds or so. Then some (partial) output appears, and so it goes on.
So I hope that clears things up. Windows and Unix behave pretty much the same in this regard. I will edit the OP's question to make it a little less confusing. It's a bit untidy at the moment.
Perhaps the "tee" already on the machine could be used. I do not have you mpg123.exe executable, so I cannot test it.
powershell -NoProfile -Command "& mpg123.exe [StreamURL] | Tee-Object -FilePath .\streaming.txt"
Edit
Based on the information from #AlexP that mpg123.exe is writing to stderr, I would try:
powershell -NoProfile -Command "& mpg123.exe [StreamURL] 2>&1 | Tee-Object -FilePath .\streaming.txt"
I am using Kubuntu and its Konsole terminal. I have annoying behavior using different applications from console such as gdb or octave. The problem is following: the program outputs one screen of text and than stops, prompting me with double colon and waiting for me to press any key so that it can continue with the output. So if the output is big (for example 10000 iterations in octave which print some values) I have to click Enter numerous times until I get to the end of the output. I would like not to be prompt at all. I don't know if this is due to bash, Konsole application or a program in question (gdb or octave for example).
For the specific case of GNU Octave, you can turn off this behavior by running more off at the Octave prompt. You may want to add that line to your .octaverc file.
If you need to repeat an input automatically, then yes is your solution :
$ yes | any_app_waiting_input
As default behaviour, yes print a "y". You can put an argument to yes if you want anything else, like newlines : yes ""
Similar to Jantio's answer, you can use can use cat as your pager by setting the PAGER environment variable:
For all commands:
export PAGER=cat
For a single command:
PAGER=cat git log
For GDB, you can use "set pagination off" to change this behavior. See GDB Manual: Screen Size. I tend to toggle pagination on and off quite often as I use GDB.
I think the output is being presented through a pager (ie. probably less or more).
One simple way to bypass this is to redirect the output using cat:
my_command | cat
Hope this helps =)
I am a newbie to unix scripting, I want to do following and I have little clue how to proceed.
I want to log the input and output of certain set of commands, given on the terminal, to a trace file. I should be able to switch it on and off.
E.g.
switch trace on
user:echo Hello World
user:Hello World
switch trace off
Then the trace log file, e.g. trace.log, it's content should be
echo Hello World
Hello World
One thing that I can think to do is to use set -x, redirecting its output to some file, but couldn't find a way to do that. I did man set, or man -x but I found no entry. Maybe I am being too naive, but some guidance will be very helpful.
I am using bash shell.
See script(1), "make typescript of terminal session". To start a new transcript in file xyz: script xyz. To add on to an existing transcript in file xyz: script -a xyz.
There will be a few overhead lines, like Script started on ... and Script done on ... which you could use awk or sed to filter out on printout. The -t switch allows a realtime playback.
I think there might have been a recent question regarding how to display a transcript in less, and although I can't find it, this question and this one address some of the same issues of viewing a file that contains control characters. (Captured transcripts often contain ANSI control sequences and usually contain Returns as well as Linefeeds.)
Update 1 A Perl program script-declutter is available to remove special characters from script logs.
The program is about 45 lines of code found near the middle of the link. Save those lines of code in a file called script-declutter, in a subdirectory that's on your PATH (for example, $HOME/bin if that's on your search path, else (eg) /usr/local/bin) and make the file executable. After that, a command like
script-declutter typescript > out
will remove most special characters from file typescript,
while directing the result to file out.
I'm trying to write a script that contains this
screen -S demo -d -m which should start a new screen session named demo and detach it.
Putting screen -S demo -d -m in the command line works.
If I put it in a file named boot.sh, and run it ./boot.sh I get
Error: Unknown option m
Why does this work in the command line but not as a shell script?
This file was transferred from windows and had ctrl-M characters.
Running "screen" on my Linux machine, a bad option (Screen version 4.00.03jw4 (FAU) 2-May-06) gives the error,
Error: Unknown option -z"
while your description includes no dash before the offending option. I'd check that the characters in your script file are what you expect them to be. There are many characters that look like a dash but which are not.
cat -v boot.sh
may show something interesting as it'll show codes for non-ascii characters.
This may seem a little like the "make sure your printer is plugged in" kind of help, but anyway:
have you tried to check if the screen you're invoking from the script is the same as the one invoked from the command line ?
I'm thinking you may change the PATH variable inside your script somewhere and perhaps screen from the script would be something else (a different version, perhaps ?).
I recently discovered 'comint-show-output' in emacs shell mode, which jumps to the first line of shell output, which I find incredibly handy when looking at shell output that exceeds a screen length. The advantages of this command over scrolling with 'page up' are A) you don't have to scan with your eyes for the first line of the output B) you only have to hit the key combo once (instead of 'page up' a number of times which probably is not known beforehand).
I thought about ending all my commands with '| more' but actually this is not what I want since most of the time, I want to retain all output in the terminal buffer, and I usually want to see the end of the shell output first.
I use OSX. Is there a terminal app (on os x) and shell (on remote linux) combination equivalent (so I can do something similar without using emacs all the time - I know, crazy talk)? I normally use bash, but would be fine with switching shells just for this feature.
The way I do this sort of thing is by sending my output to a file and then watching the file as it is written. You still get the results of the command dumped to terminal history in real time and can still inspect the output's actual contents further after the fact (or in another terminal, etc...)
command > output &
tail -f output
head output
You could always do something in bash like this:
alias foo='!! | more'
which would make foo run the previous command with more. I'm not sure of any way to do exactly what you are suggesting.
If you're expecting a lot of output and don't want to run your command twice, you can use tee(1) to fork the output:
my-command | tee /tmp/my-command.log | less
This will pipe the output to a paginator (less), while simultaneously logging the output to a file (in this case, a file named /tmp/my-command.log). If you need to review the output after you've quit from less, you can just cat the log file instead of re-running the command.