Suppose I have three files file1 file2 file3 having some content
Now when I do this on shell prompt cat file1 > file2 >file3
Content of file1 is copied to file3 and file2 becomes empty
Similarly when I do cat > file1 > file2 > file3
It ask for input and this input is stored in file3 and both file1 and file2 are empty
and also for cat > file1 > file2 < file3 contents of file3 is copied to file2 and file1 is empty.
Can someone please explain to me what is happening I am new to UNIX. Also any website where I can learn more about these redirection operators.
Thanks
Consider how the shell processes each part of the command as it parses it:
cat file1 > file2 >file3
cat file1: prepare a new process with the cat program image with argument file1. ( given 1 or more arguments, cat will read from each argument as a file and write to its output file descriptor)
> file2: change the new process' output file descriptor to write to file2 instead of the current output sink (initially the console for an interactive shell) - create `file2 if necessary.
> file3: change the new process' output file descriptor to write to file3 instead of the current output sink (was file2) - create file3 if necessary
End of command: Spawn the new process
So in the end, file2 is created, but unused. file3 gets the data.
cat > file1 > file2 > file3
cat: prepare a new process with the cat program/image with no arguments. (given no arguments, cat will read from its input file descriptor and write to its output file descriptor)
> file1: change the new process' output file descriptor to write to file1 instead of the current output sink (initially the console for an interactive shell) - create file1 if necessary.
> file2: change the new process' output file descriptor to write to file2 instead of the current output sink (was file1) - create file2 if necessary.
> file3: change the new process' output file descriptor to write to file3 instead of the current output sink - (was file2) create file3 if necessary
End of command: Spawn the new process
So in the end, file1 and file2 are created, but unused. file3 gets the data. cat waits for input on its input device (the console device as default for an interactive shell). Any input that cat receives will go to its output device (which ends up being file3 by the time the shell finished processing the command and invoked cat).
cat > file1 > file2 < file3
cat: prepare a new process with the cat program/image with no arguments. (given no arguments, cat will read from its input file descriptor and write to its output file descriptor)
> file1: change the new process' output file descriptor to write to file1 instead of the current output sink (initially the console for an interactive shell) - create file1 if necessary.
> file2: change the new process' output file descriptor to write to file2 instead of the current output sink (was file1) - create file2 if necessary.
< file3: change the new process' input file descriptor to read from file3 instead of the current input source (initially the console for an interactive shell)
End of command: Spawn the new process
So in the end, file1 is created, but unused. file2 gets the data. cat waits for input on its input device (which as set to file3 by the time the shell finished processing the command and invoked cat). Any input that cat receives will go to its output device (which ends up being file2 by the time the shell finished processing the command and invoked cat).
--
Note that in the first example, cat is the one who processes/opens file1. The shell simply passed the word file1 to the program as an argument. However, the shell opened/created file2 and file3. cat knew nothing about file3 and has no idea where the stuff it was writing to its standard output was going.
In the other 2 examples, the shell opened all the files. cat knew nothing about any files. cat had no idea where its standard input was coming from and where its standard output was going to.
Per #Sorpigal comment - the BASH manual has some good descriptions of what the different redirection operators do. Much of it is the same across different Unix shells to varying degrees, but consult your specific shell manual/manpage to confirm. Thanks #Sorpigal.
http://gnu.org/software/bash/manual/html_node/Redirections.html
You can redirect the standard input < standard output 1> or > error output 2> or both outputs &> but you can only redirect 1:1, you can't redirect one output into two different files.
What you are looking for is the tee utility.
If you don't want to lose original content, you should use redirect and append >> or << operators instead. You can read more here.
Related
I am looking for a way to run a command like smartctl on a file containing device names like /dev/sda; (one per line). The ansible playbook should be able to read each line and make it an arg to the command.
Are you looking for something like this?
<file_with_smartctl_args xargs -n1 smartctl
Replace file_with_smartctl_args with the file (complete path!) that contains the names of the files (arguments) you want to pass to smartctl. This will run "smartctl" one time for EACH of the lines (arguments) in the file.
Example:
If the file /usr/me/smartctl_args contains the following text:
file1
file2
file3
The command:
</usr/me/smartctl_args xargs -n1 smartctl
Will run smartctl 3 times (since the file has 3 lines in it), like this:
smartctl file1
smartctl file2
smartctl file3
The initial < tells the Unix shell that your "standard input" is going to come from the filename that follows (/usr/me/smartctl_args). Then, xargs will convert the "standard input" to command arguments, the -n1 option causes xargs to execute the command (smartctl) once for each argument.
I am new to unix scripting. I have a requirement that I need to check a string from a csv file against a set of strings in another csv file. My file1 will have values like below,
file1
task desc
1 network error in there in line one.
2 data error is there in line three.
3 connection is missing from device.
4 new error is there
And file2 will have the standard lookup data
file2
Network error
data error
connection is
The file2 is static and my file1 changes daily. My requirement is to check each line in the file1 and check against the strings in file2. if there are any new error entry in the file1, then that entire row needs to be write into a new file called file3. this kind of daily standard error checking from a log file and report the new error. For example from my above example, the output file3 needs to look like below.
file3
4 new error is there
tail + grep solution:
tail -n +2 file1 | grep -ivf file2 > file3
> cat file3
4 new error is there
tail -n +2 file1 - output the last part of file1 starting from the 2nd line
grep options:
-f - obtain patterns from file
-i - caseinsensitive matching
-v - invert the sense of matching, to select non-matching lines
Why echo a > file1 > file2 creates both files but only write to file2? (file1 is empty.)
Because I/O redirections are processed from left to right. The sequence of actions is:
Open file1 for writing (creating it if it doesn't exist).
Redirect stdout to file1.
Open file2 for writing (creating it if it doesn't exist).
Redirect stdout to file2.
Run echo a.
I have a directory that contains a file called oracle.txt. I need to copy the file name to a flat file I tried using the CP command but it tells me that I am copying a directory. I tried the cat command but that copies the contents not the name. Is there a command to copy Oracle.txt (the name not the contents) to a flat file? So after it completes there will be a file called process.txt and its contents will be oracle.txt
you can get the names of the files in directory using the ls command.
Then you have to redirect that output to a file using >
The command required is shown below.
ls > process.txt
cp file1 file2 will copy the contents of file1 over file2. cat file1 > file2 does the same thing. echo file1 > file2 will replace the contents of file2 with the string file1 (followed by a newline).
The Bash Guide has much more.
If you know the file names and their location on your machine then you can use following command, ls /path/to/your/file > flat_file_name This command will list all the files from your expected directory and will put those names in your flat file.
The sign '>' will overwrite all contents from flat file. If you want to append in the file use '>>' instead of '>'.
Please explain the output of this shell command:
ls >file1 >file2
Why does the output go to file2 instead of file1?
bash only allows one redirection per file descriptor. If multiple redirections are provided, like in your example, they are processed from left to right, with the last one being the only one that takes effect. (Notice, though, that each file will still be created, or truncated if already in existence; the others just won't be used by the process.)
Some shells (like zsh) have an option to allow multiple redirections. In bash, you can simulate this with a series of calls to tee:
ls | tee file1 file2 > /dev/null
Each call to tee writes its input to the named file(s) and its standard output.
If the shell finds multiple redirections of any output, it will redirect it to the last file given, in your case file2, since redirections are evaluated from left to right.
While it works, you should not do something like that!
You first redirected the STDOUT stream to go to file1 but then immediately redirected it to go to file2. So, the output goes to file2.
Every process initially has three file descriptors - 0 for STDIN, 1 for STDOUT and 2 for STDERR. >file1 means that open the file1 and assign its descriptor the id 1. Note that the process which is about to execute doesn't care about what is the end point where its STDOUT goes. It just writes to whatever is described by file descriptor 1.
For a more technical description of how this works, see this answer.
The redirect operator is short for a stdout redirection(1>). Since the command is evaluated left to right, the last stdout redirection is used for the running of ls.
ls 1> file1 1>file2
is equivalent to
ls >file1 >file2
If you're trying to redirection stderr, use
ls > file1 2> file2
0 = stdin
1 = stdout
2 = stderr
try this, you'll notice that file2 will receive the stderr message.
ls ------ > file1 2> file2
then these, in both cases output will be in stdout and will go to file1.
ls >file1 2>file2
ls 1>file1 2>file2
Because first redirection gets overridden by the second. Note though, that an empty file1 is still created when file1 was opened for output.