i know what they are, but i dunno when i should use them. Are they useful? I think yes, but I want you to tell me in which situations a file descriptor could be useful. Thanks :D
The most obvious case which springs to mind is:
myProgram >myProgram.output_and_error 2>&1
which sends both standard output and error to the same file.
I've also used:
myProgram 2>&1 | less
which will allow me to page through the output and error in sequence (rather than having error got to the terminal in "arbitrary" places in the output).
Basically, any time when you need to get at an already existing file descriptor, you'll find yourself using this.
Related
I need to start up a Golang web server and leave it running in the background from a bash script. If the script in question in syntactically correct (as it will be most of the time) this is simply a matter of issuing a
go run /path/to/index.go &
However, I have to allow for the possibility that index.go is somehow erroneous. I should explain that in Golang this for something as "trival" as importing a module that you then fail to use. In this case the go run /path/to/index.go bit will return an error message. In the terminal this would be something along the lines of
index.go:4:10: expected...
What I need to be able to do is to somehow change that command above so I can funnel any error messages into a file for examination at a later stage. I tried variants on go run /path/to/index.go >> errors.txt with the terminating & in different positions but to no avail.
I suspect that there is a bash way to do this by altering the priority of evaluation of the command via some judiciously used braces/brackets etc. However, that is way beyond my bash capabilities. I would be most obliged to anyone who might be able to help.
Update
A few minutes later... After a few more experiments I have found that this works
go run /path/to/index.go &> errors.txt &
Quite apart from the fact that I don't in fact understand why it works there remains the issue that it produces a 0 byte errors.txt file when the command goes to completion without Golang throwing up any error messages. Can someone shed light on what is going on and how it might be improved?
Taken from man bash.
Redirecting Standard Output and Standard Error
This construct allows both the standard output (file descriptor 1) and the standard error output (file descriptor 2) to be redirected to the file whose name is the expansion of word.
There are two formats for redirecting standard output and standard error:
&>word
and
>&word
Of the two forms, the first is preferred. This is semantically equivalent to
>word 2>&1
Appending Standard Output and Standard Error
This construct allows both the standard output (file descriptor 1) and the standard error output (file descriptor 2) to be appended to the file whose name is the expansion of word.
The format for appending standard output and standard error is:
&>>word
This is semantically equivalent to
>>word 2>&1
Narūnas K's answer covers why the &> redirection works.
The reason why the file is created anyway is because the shell creates the file before it even runs the command in question.
You can see this by trying no-such-command > file.out and seeing that even though the shell errors because no-such-command doesn't exist the file gets created (using &> on that test will get the shell's error in the file).
This is why you can't do things like sed 'pattern' file > file to edit a file in place.
I'm making some command-line tools for some research I'm doing. I'd like these tools to follow commonly used conventions regarding command line programs in Unix.
Should I use flags or just list parameters?
program one two three
program -a one -b two -c three
Where in the list of commands does the input file normally go, or is it better to < it into the program?
What about the output filename?
Should I specify the file extension for the output format, or have my program automatically put the correct extension on?
When the user enters an invalid command, is there a prototypical "correct usage" message?
Is "--help" or "-h" required?
Also, is there some sort of header file I can include that would help with managing these?
If you're looking for a "standard", then you could do worse than look at GNU's Standards for Command Line Interfaces. Other standards are available.
As far as coding for this goes, take a look at boost::program_options. Not only will this save you rolling a lot of your own code, but it does a good job of formatting the options for presenting to the user (the prototypical "correct usage" message, you asked for).
In answer to your specific questions:
Where in the list of commands does the input file normally go, or is it better to < it into the program?
I would expect these to come at the end of a command line. Like in GNU grep. If you are only processing one file and would like to make stdin available as an input source, that would not surprise most users.
If your command processes lots of files, then it would be unusual to have to specify a switch before the filenames. Think cat.
What about the output filename?
A -o or --output option is fairly common. If your file takes exactly one input and one output, then program inputfile outputfile would not surprise many users. If no output file is specified, perhaps you'll output to stdout; that would not be unusual behaviour and would allow your users to pipe the output through other commands (such as grep, less, etc...), They could also redirect stdout to a file using >.
Should I specify the file extension for the output format, or have my program automatically put the correct extension on?
This is probably a matter for debate. If I specified an output filename, I would expect to find that file created (or replaced, after a prompt) without the program changing the name.
When the user enters an invalid command, is there a prototypical "correct usage" message?
Using GNU grep as an example again:
grep: unrecognized option '--incorrect'
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
This wouldn't surprise too many users and points them in the right direction if they've made a typo without swamping them with information.
Is "--help" or "-h" required?
That depends on your customer! I find it frustrating when this option isn't available.
Usually speaking, flags are there for providing options and parameter are for passing information. If you have input,output file as command line argument, use flags like -i -o, so sequence will not matter. -h is required if you want to (and need to) give documentation.
Does anyone happen to know how to direct STDOUT in Terminal to Cache? Sometimes I would like to copy text from STDOUT somewhere else, e.g. my mail program, and it seems always a bit inconvenient to me to either copy the output manually or create a new temporary file.
Is there an easy way to do this?
Thanks a lot!
Alex
It's not clear exactly what you're asking. But if you're talking about capturing stdout to file whilst still being able to see it on the console, then you can use tee (assuming you're using *nix):
./myApp | tee stdout.txt
I want to put my program's output into a file. I keyed in the following :
./prog > log 2>&1
But there is nothing in the file "log". I am using the Ubuntu 11.10 and the default shell is bash.
Anybody know the cause of this AND how I can debug this?
There are many possible causes:
The program reads the input from log file while you try to redirect into it with truncation (see Why doesn't "sort file1 > file1" work?)
The output is buffered so that you don't see data in the file until the output buffer is flushed. You can manually call fflush or output std::flush if using C++ I/O stream etc.
The program is smart enough and disables output if the output stream is not a terminal.
You look at the wrong file (i.e. in another directory).
You try to dump file's contents incorrectly.
Your program outputs '\0' as the first character so the output appears to be empty, even though there is some data.
Name your own.
Your best bet is to run this application under a debugger (like gdb) or use strace or ptrace (or both) and see what the program is doing. I mean, really, output redirection works for the last like 40 years, so the problem must be somewhere else.
I am a newbie to unix scripting, I want to do following and I have little clue how to proceed.
I want to log the input and output of certain set of commands, given on the terminal, to a trace file. I should be able to switch it on and off.
E.g.
switch trace on
user:echo Hello World
user:Hello World
switch trace off
Then the trace log file, e.g. trace.log, it's content should be
echo Hello World
Hello World
One thing that I can think to do is to use set -x, redirecting its output to some file, but couldn't find a way to do that. I did man set, or man -x but I found no entry. Maybe I am being too naive, but some guidance will be very helpful.
I am using bash shell.
See script(1), "make typescript of terminal session". To start a new transcript in file xyz: script xyz. To add on to an existing transcript in file xyz: script -a xyz.
There will be a few overhead lines, like Script started on ... and Script done on ... which you could use awk or sed to filter out on printout. The -t switch allows a realtime playback.
I think there might have been a recent question regarding how to display a transcript in less, and although I can't find it, this question and this one address some of the same issues of viewing a file that contains control characters. (Captured transcripts often contain ANSI control sequences and usually contain Returns as well as Linefeeds.)
Update 1 A Perl program script-declutter is available to remove special characters from script logs.
The program is about 45 lines of code found near the middle of the link. Save those lines of code in a file called script-declutter, in a subdirectory that's on your PATH (for example, $HOME/bin if that's on your search path, else (eg) /usr/local/bin) and make the file executable. After that, a command like
script-declutter typescript > out
will remove most special characters from file typescript,
while directing the result to file out.