When I try to grep pattern and write to a file, sometimes it complains the file bar.txt already exist, so I have to use >> instead of > to overwrite it.
grep 'pattern' foo.txt >> bar.txt
But if the file didn't exist, using >> it will complain about no such file or directory. Is there a way for shell to automatically make it's own decision? If not exist, create a file. If exists, overwrite.
Extracted from the tcsh man page:
> name
...
If the shell variable noclobber is set, then the file must not
exist or be a character special file (e.g., a terminal or
`/dev/null') or an error results. This helps prevent acciden-
tal destruction of files.
...
>> name
...
Like `>', but appends output to the end of name. If the shell
variable noclobber is set, then it is an error for the file not
to exist, unless one of the `!' forms is given.
So ... it sounds to me as if you have "noclobber" set.
% set noclobber
% echo foo >> bar
bar: No such file or directory.
% echo foo > bar
% echo foo > bar
bar: File exists.
% unset noclobber
% echo foo > bar
If this special shell variable was set in your .tcshrc or .cshrc or .login, you can unset it. Or, if it is on by default or set in a system-wide shell startup file, simply append a line to your rc file:
unset noclobber
And you should be good to go.
Related
I designed a custom script to grep a concatenated list of .bash_history backup files. In my script, I am creating a temporary file with mktemp and saving it to a variable temp. Next, I am redirecting output to that file using the cat command.
Is there a means to create a temporary file (using mktemp), redirect output to it, then store it in a variable in one command, while preserving newline characters?
The below snippet of code works just fine, but I have a feeling there is a more terse and canonical way to achieve this in one line – maybe using process substitution or something of the like.
# Concatenate all .bash_history files into a temporary file `temp`.
temp="$(mktemp)"
cat "$HOME/.bash_history."* > $temp
trap 'rm -f $temp' 0
# Set `HISTFILE` shell variable to the `temp` file.
HISTFILE="$temp"
keyword="$1"
# Search for `keyword` using the `history` command
if [[ "$keyword" ]]; then
# Enable history
set -o history
history | grep "$keyword"
# Disable history
set +o history
else
echo -e "usage: search <keyword>"
exit 0
fi
If you're comfortable with the side effect of making the assignment conditional on tempfile not previously having a nonempty value, this is straightforward via the ${var:=value} expansion:
cat "$HOME/.bash_history" >"${tempfile:=$(mktemp)}"
cat myfile.txt | f=`mktemp` && cat > "${f}"
I guess there is more than one way to do it. I found following to be working for me:
cat myfile.txt > $(echo "$(mktemp)")
Also don't forget about tee:
cat myfile.txt | tee "$(mktemp)" > /dev/null
My environments:
CentOS 6.5
bash 4.1.2(1)
Sometimes when I intend to add something to a file,
instead of
$ echo "xxx" >> mymemo.txt
I type mistakenly
$ echo "xxx" > mymemo.txt
resulting in loosing memos in mymemo.txt.
I am wondering if there is a way to prohibit to use redirection (>), but allow to use redirection (>>)?
You can use set -o noclobber in your .bashrc or .profile
If set, bash prevents you from overwriting existing files when redirecting.
mint#mint ~ $ echo "foo" > test
bash: test: cannot overwrite existing file
What is the easiest way to append text to a file in Linux?
I had a look at this question, but the accepted answer uses an additional program (sed) I'm sure there should be an easier way with echo or similar.
How about:
echo "hello" >> <filename>
Using the >> operator will append data at the end of the file, while using the > will overwrite the contents of the file if already existing.
You could also use printf in the same way:
printf "hello" >> <filename>
Note that it can be dangerous to use the above. For instance if you already have a file and you need to append data to the end of the file and you forget to add the last > all data in the file will be destroyed. You can change this behavior by setting the noclobber variable in your .bashrc:
set -o noclobber
Now when you try to do echo "hello" > file.txt you will get a warning saying cannot overwrite existing file.
To force writing to the file you must now use the special syntax:
echo "hello" >| <filename>
You should also know that by default echo adds a trailing new-line character which can be suppressed by using the -n flag:
echo -n "hello" >> <filename>
References
echo(1) - Linux man page
noclobber variable
I/O Redirection
cat >> filename
This is text, perhaps pasted in from some other source.
Or else entered at the keyboard, doesn't matter.
^D
Essentially, you can dump any text you want into the file. CTRL-D sends an end-of-file signal, which terminates input and returns you to the shell.
Other possible way is:
echo "text" | tee -a filename >/dev/null
The -a will append at the end of the file.
If needing sudo, use:
echo "text" | sudo tee -a filename >/dev/null
Follow up to accepted answer.
You need something other than CTRL-D to designate the end if using this in a script. Try this instead:
cat << EOF >> filename
This is text entered via the keyboard or via a script.
EOF
This will append text to the stated file (not including "EOF").
It utilizes a here document (or heredoc).
However if you need sudo to append to the stated file, you will run into trouble utilizing a heredoc due to I/O redirection if you're typing directly on the command line.
This variation will work when you are typing directly on the command line:
sudo sh -c 'cat << EOF >> filename
This is text entered via the keyboard.
EOF'
Or you can use tee instead to avoid the command line sudo issue seen when using the heredoc with cat:
tee -a filename << EOF
This is text entered via the keyboard or via a script.
EOF
I'd like to create a script A which creates a script B. Script B creates a directory. So I created a file with this content, grant x permission to it, then execute it. Unfortunately it doesn't run as I expect. It makes directory first then create an empty file. Why?
#!/bin/bash
batch=`mkdir /home/hieund/bpl`
echo $batch > newfile
Update:
After trying your solution, I have:
#!/bin/bash
$myPath=$HOME/bpl
batch='mkdir ' $myPath
echo $batch > newfile
It doesn't work as well. Same unexpected behavior.
Update:
#!/bin/bash
$myPath=$HOME/bpl
batch="mkdir $myPath"
echo $batch > newfile
It doesn't work too. Same unexpected behavior.
bash: /home/hieund/bpl=/home/hieund/bpl: No such file or directory
It makes a directory at the moment of assignment, because you said this with "command substitution"
batch=`mkdir /home/hieund/bpl`
The flow of execution
mkdir /home/... - creates the directory - because of backticks - command substitution
the mkdir returns nothing, therefore
the assignment is like batch= (it assigns nothing)
the echo $batch echoes the "nothing" so: echo > newfile
you should to use
batch='mkdir /home/hieund/bpl'
for embedding variable use double quotes
batch="mkdir $myPath"
You always can use the bash -x script - to show what is executing. E.g. having a script myscript.sh
#!/bin/bash
MYDIR="./somedir"
batch="mkdir $MYDIR"
echo "$batch" > newfile
the command
bash -x myscript.sh
will show the execution of command and arguments. (note, not shown redirections)
+ MYDIR=./somedir
+ batch='mkdir ./somedir'
+ echo 'mkdir ./somedir'
One comment: You should generally assign things to variables with double quotes, because you can avoid problems with spaces. Note
myvar=$VAR/some
and
myvar="$VAR/some"
makes a big difference when the $VAR contains spaces.
Because backticks are not used for strings.
batch="mkdir /home/hieund/bpl"
I tried to use a copied script, which includes the following command
echo "rc $2" > $WORKDIR/out.dat
I can guess it tries to output some contents to file out.dat. But what does "rc $2" mean?
It also includes
echo "PWD" >> $WORKDIR/env.txt
Why it uses >> here instead of >
"rc" means nothing here and neither does "PWD". They are just strings. They presumably mean something in out.dat and env.txt, though. The "$2" is a reference to the second arg used to call the script.
>> means append to a file rather than overwriting it like > will do.
$2 is the 2nd incoming var when the script is executed.
example:
./script.sh foo bar # $2 would be bar
>> means append to file instead of overwrite file entirely.