I'm combining sox and lame to generate a new music file, but in order to do everything on one line using pipes, it seems necessary to 'mark' the output and input boundaries with a - character. I've inherited this code, so let me show.
sox $DIRNAME/$BASENAME -e signed-integer -r 8000 -c 2 -t wav - trim $POSITIONS | lame -v -V4 --resample 8 - $DIRNAME/${NOEXT}.mp3
The - between wav and trim is the output file, and the - between --resample 8 and $DIRNAME/${NOEXT}.mp3 is the input file.
I'm trying to find further information on this, like whether any character can be used, or if - is special in this way. What is this called, and what makes it work?
Many Unix command-line utilities use "-" as a shorthand to mean "don't use a real file here, use stdin (or stdout) instead". Sox is one of these programs:
This is from the sox manpage
SoX can be used in simple pipeline operations by using the special
filename '-' which, if used in place of an input filename, will cause
SoX will read audio data from 'standard input' (stdin), and which, if
used in place of the output filename, will cause SoX will send audio
data to 'standard output' (stdout). Note that when using this option,
the file-type (see -t below) must also be given.
By convention, unix and friends use - to represent stdin and stdout.
It's not 100% universal, but it's a pretty widely used.
In your example, it's the same thing as
/dev/stdin
Try to replace your - with it, you will see.
"-" is often use as a convention instead of a file name to say "use the standard input (instead or reading for the file) or standard output (instead of writing to the file)". This is not a feature of the command shell (i.e. bash), so in that sense, it is not a special character. It is a feature of some commands (like in your case "sox" and "lame") and is very useful to chain these commands through pipes.
Related
Examples:
Create an ISO image and burn it directly to a CD.
mkisofs -V Photos -r /home/vivek/photos | cdrecord -v dev=/dev/dvdrw -
Change to the previous directory.
cd -
Listen on port 12345 and untar data sent to it.
nc -l -p 12345 | tar xvzf -
What is the purpose of the dash and how do I use it?
If you mean the naked - at the end of the tar command, that's common on many commands that want to use a file.
It allows you to specify standard input or output rather than an actual file name.
That's the case for your first and third example. For example, the cdrecord command is taking standard input (the ISO image stream produced by mkisofs) and writing it directly to /dev/dvdrw.
With the cd command, every time you change directory, it stores the directory you came from. If you do cd with the special - "directory name", it uses that remembered directory instead of a real one. You can easily switch between two directories quite quickly by using that.
Other commands may treat - as a different special value.
It's not magic. Some commands interpret - as the user wanting to read from stdin or write to stdout; there is nothing special about it to the shell.
- means exactly what each command wants it to mean. There are several common conventions, and you've seen examples of most of them in other answers, but none of them are 100% universal.
There is nothing magic about the - character as far as the shell is concerned (except that the shell itself, and some of its built-in commands like cd and echo, use it in conventional ways). Some characters, like \, ', and ", are "magical", having special meanings wherever they appear. These are "shell metacharacters". - is not like that.
To see how a given command uses -, read the documentation for that command.
It means to use the program's standard input stream.
In the case of cd, it means something different: change to the prior working directory.
The magic is in the convention. For millennia, people have used '-' to distinguish options from arguments, and have used '-' in a filename to mean either stdin or stdout, as appropriate. Do not underestimate the power of convention!
Examples:
Create an ISO image and burn it directly to a CD.
mkisofs -V Photos -r /home/vivek/photos | cdrecord -v dev=/dev/dvdrw -
Change to the previous directory.
cd -
Listen on port 12345 and untar data sent to it.
nc -l -p 12345 | tar xvzf -
What is the purpose of the dash and how do I use it?
If you mean the naked - at the end of the tar command, that's common on many commands that want to use a file.
It allows you to specify standard input or output rather than an actual file name.
That's the case for your first and third example. For example, the cdrecord command is taking standard input (the ISO image stream produced by mkisofs) and writing it directly to /dev/dvdrw.
With the cd command, every time you change directory, it stores the directory you came from. If you do cd with the special - "directory name", it uses that remembered directory instead of a real one. You can easily switch between two directories quite quickly by using that.
Other commands may treat - as a different special value.
It's not magic. Some commands interpret - as the user wanting to read from stdin or write to stdout; there is nothing special about it to the shell.
- means exactly what each command wants it to mean. There are several common conventions, and you've seen examples of most of them in other answers, but none of them are 100% universal.
There is nothing magic about the - character as far as the shell is concerned (except that the shell itself, and some of its built-in commands like cd and echo, use it in conventional ways). Some characters, like \, ', and ", are "magical", having special meanings wherever they appear. These are "shell metacharacters". - is not like that.
To see how a given command uses -, read the documentation for that command.
It means to use the program's standard input stream.
In the case of cd, it means something different: change to the prior working directory.
The magic is in the convention. For millennia, people have used '-' to distinguish options from arguments, and have used '-' in a filename to mean either stdin or stdout, as appropriate. Do not underestimate the power of convention!
Examples:
Create an ISO image and burn it directly to a CD.
mkisofs -V Photos -r /home/vivek/photos | cdrecord -v dev=/dev/dvdrw -
Change to the previous directory.
cd -
Listen on port 12345 and untar data sent to it.
nc -l -p 12345 | tar xvzf -
What is the purpose of the dash and how do I use it?
If you mean the naked - at the end of the tar command, that's common on many commands that want to use a file.
It allows you to specify standard input or output rather than an actual file name.
That's the case for your first and third example. For example, the cdrecord command is taking standard input (the ISO image stream produced by mkisofs) and writing it directly to /dev/dvdrw.
With the cd command, every time you change directory, it stores the directory you came from. If you do cd with the special - "directory name", it uses that remembered directory instead of a real one. You can easily switch between two directories quite quickly by using that.
Other commands may treat - as a different special value.
It's not magic. Some commands interpret - as the user wanting to read from stdin or write to stdout; there is nothing special about it to the shell.
- means exactly what each command wants it to mean. There are several common conventions, and you've seen examples of most of them in other answers, but none of them are 100% universal.
There is nothing magic about the - character as far as the shell is concerned (except that the shell itself, and some of its built-in commands like cd and echo, use it in conventional ways). Some characters, like \, ', and ", are "magical", having special meanings wherever they appear. These are "shell metacharacters". - is not like that.
To see how a given command uses -, read the documentation for that command.
It means to use the program's standard input stream.
In the case of cd, it means something different: change to the prior working directory.
The magic is in the convention. For millennia, people have used '-' to distinguish options from arguments, and have used '-' in a filename to mean either stdin or stdout, as appropriate. Do not underestimate the power of convention!
mongodump -db somedb --collection somecollection --out - | gzip > dump.gz
What is the dash mean in above picture? I seems something to do with tunnel. But after looking up bash reference, i still can't get what't the meaning.
It means output goes to the stdout, so the pipe would pass it to the gzip. And it's not about bash(1), it's up to the program mongodump. In unix and linux, the dash - related to input or output, usually stands for stdin or stdout.
oxnz is right on target. To expand upon that a little bit, some commands want a file as an argument typically at the end. In the command you describe, mongodump is hoping to output a file.
That's great, but your intent is to just compress it into a file dump_xyz. So instead of mongodump take a file as the final argument, the - indicates standard output (depending on the context, - can mean standard input). The output of mongodump directly goes to the standard output and pipes right into the waiting gzip. Gzip takes mongodump's output, compresses it on the fly and stores the compressed information into dump_xyz file.
Here's some light reading about - special character: http://tldp.org/LDP/abs/html/special-chars.html#DASHREF2
I'm writing a bash script to make DVD authoring more automated (but, mainly, so that I can learn some more bash scripting) and I'm trying to find out if it's possible to control how an exterrnal command presents its output.
For instance, the output from ffmpeg is a load of (to me) irrelevant cruft about options, libraries, streams, progress and so on.
What I really want is to be able to select for display only the lines with the input and output filenames and then to display the progress on the same line each time. Similarly for mkisofs and wodim.
I've tried Googling for this and am beginning to suspect that either it's not possible or nobody's thought of it before (or, possibly, that it's so obvious that nobody thinks it necessary to say how :-) ).
Many thanks, in advance,
David Shaw
You want to use grep and pipes. They are your friends. You want to pipe the output of the ffmpeg into grep and have it output only lines containing the text you want.
Assuming you have the input and output file names as command lines arguments $1 and $2 to your shell script, you might try something like
ffmpeg .... | grep "$1\|$2"
^ ^
| +-- escape and OR character
+--pipe character
The '\|' is an escape and an OR character for regular expressions. The OR '|' is also the pipe character so you have to escape that.
This will output only output lines that contain the files you are looking for.
This assumes all output is via stdout. If ffmpeg is outputting text via stderr then you will need to add some redirects at the end of ffmpeg line to redirect those back to stdout.
EDIT: I used the wrong quotes in the first example. Use double quotes or it won't expand the parameters $1 and $2