cat command: unexpected output - bash

I tried the following command $cat < text > text where text is a non-empty file.
There was no output to stdout and the file text became blank. I expected cat command to read the file text and output the contents to the same file.
However when I try $cat < text > newtext it works! newtext is a copy of text.
Another doubt, When I try $cat < text >>text where >> usually appends to a file. My terminal gets stuck in an infinite loop and file text is repeatedly appended to itself. Why does this happen?


You cannot use the same file as stdin and stdout. Why? Because all commands are executed at the same time, and the shell prepares redirections before executing the commands. Hence, the output file is emptied before executing.
Instead, you have to use temporary files.
A workaround could be your solution or also:
cat text > newtext && mv newtext text


When you redirect your output to a file with
echo "something" > file.txt
the first thing that happens, is that your file is (re)created. If it exists, it will be emptied and that's exactly what you see.
You can show the contents of the file by simply invoking
cat file.txt
To achieve what you've tried, you could do
cat file.txt > temp.txt && mv temp.txt file.txt
but I don't see a reason why you would want to do that.


If you think about it the shell has to do the redirections first. If the shell actually executed the command from left to right as you expect then cat file would display the files contents to the terminal then the shell would see > file and have to back track i.e. clear the output from the terminal and rerun the command with the stdout redirected.
This obviously doesn't make sense so when the shell parses your command the redirections must be done first and since > overwrites the contents your file is clobbered before it is read i.e it's empty!

Related

What’s difference between `cat <file` with `cat file`?

Those look same. Maybe I’m misunderstaning about < operator.
Why those two commands give same results?

With cat < file, cat reads from its standard input; the shell opens the file and connects the file handle to cat. With cat file, cat itself opens the file without any shell involvement. The end result is indeed the same for both: cat reads the contents of file and outputs them to standard output.

Append output from both stdout and stderr of every command of a bash script to file

Here
https://stackoverflow.com/a/876267/1579327
I learned how to do that for a single command cmd appending output to file.txt
cmd >>file.txt 2>&1
But my script contains many statements and commands.
And I would like to avoid appending >>file.txt 2>&1 to every line.
Is there a directive to let me to do that by default for every subsequent command?
Side note: I'm looking for a solution suitable also for MacOs X's bash

On top of your script you can use exec like this:
#!/bin/bash
# append stdout/stderr to a file
exec >> file.log 2>&1
# script starts here

Why does input redirection work differently when in command substitution?

Consider the following case:
$ echo "abc" > file
$ var=$(< file)
$ echo "$var"
abc
Inside the command substitution, we use a redirect and a file, and the content of the file is correctly captured by the variable.
However, all the following examples produce no output:
$ < file
$ < file | cat
$ < file > file2
$ cat file2
In all these cases the content of the command is not redirected to the output.
So why is there a difference when the redirect is placed inside the command substitution or not? Does the redirect have a different function when inside vs outside a command substitution block?

$(< file) is not a redirection; it is just a special case of a command substitution that uses the same syntax as an input redirection.
In general, an input redirection must be associated with a command. There is one case that arguably could be considered an exception, which is
$ > file
It's not technically a redirection, since nothing is redirected to the file, but file is still opened in write mode, which truncates it to 0 bytes.

strings not appended onto the file in shell scripting

I was trying a simple shell program as below to append data at the end of the file,
path="/root/dir"
secure="*(rw,..)"
echo "$path $secure" >> a.txt
is not appending the string to a.txt

Just a guess but your script may be in DOS format that you're actually trying to write output to a.txt\r instead. Try to run one of the following to your code and try again:
sed -i 's|\r||' file
dos2unix file

Why reading and writing the same file through I/O redirection results in an empty file in Unix?

If I redirect output of a command to same file it reads from, its contents is erased.
sed 's/abd/def/g' a.txt > a.txt
Can anyone explain why?

The first thing the redirection does is to open the file for writing, thus clearing any existing contents. sed then tries to read this empty file you have just created, and does nothing. The file is then closed, containing nothing.

The redirection operations <, >, etc. are handled by the shell. When you give a command to the shell that includes redirection, the shell will first open the file. In the case of > the file will be opened for writing, which means it gets truncated to zero size. After the redirection files have been opened, the shell starts a new process, binding its standard input, output, and error to any possible redirected files, and only then executes the command you gave. So when the sed command in your example begins execution, a.txt has already been truncated by the shell.
Incidentally, and somewhat tangentially, this is also the reason why you cannot use redirection directly with sudo because it is the shell that needs the permissions to open the redirection file, not the command being executed.

You need to use the -i option to edit the file in place:
sed -i .bck 's/abd/def/g' a.txt
EDIT: as noted by neil, the redirection first opens the file for writing thus clears it.
EDIT2: it might be interesting for some reader
On OSX, if you want to use -i with an empty extension to prevent the creation of a backup file, you need to use the -eswitch as well otherwise it fails to parse the arguments correctly:
sed -i -e 's/abc/def/g' a.txt

stdout and stderr will first prepared and then stdin and then the command execute. so a.txt would be clear for stdout first and then when the comamnd execute no content could be found.
try
sed -i 's/abd/def/g' a.txt
or
sed 's/abd/def/g' a.txt 1<> a.txt

Resources