Write to file, but overwrite it if it exists - bash

echo "text" >> 'Users/Name/Desktop/TheAccount.txt'
How do I make it so it creates the file if it doesn't exist, but overwrites it if it already exists. Right now this script just appends.

The >> redirection operator will append lines to the end of the specified file, where-as the single greater than > will empty and overwrite the file.
echo "text" > 'Users/Name/Desktop/TheAccount.txt'

In Bash, if you have set noclobber a la set -o noclobber, then you use the syntax >|
For example:
echo "some text" >| existing_file
This also works if the file doesn't exist yet
Check if noclobber is set with: set -o | grep noclobber
For a more detailed explanation on this special type of operator, see this post
For a more exhaustive list of redirection operators, refer to this post

Despite NylonSmile's answer, which is "sort of" correct.. I was unable to overwrite files, in this manner..
echo "i know about Pipes, girlfriend" > thatAnswer
zsh: file exists: thatAnswer
to solve my issues.. I had to use... >!, á la..
[[ $FORCE_IT == 'YES' ]] && echo "$#" >! "$X" || echo "$#" > "$X"
Obviously, be careful with this...

If your environment doesn't allow overwriting with >, use pipe | and tee instead as follows:
echo "text" | tee 'Users/Name/Desktop/TheAccount.txt'
Note this will also print to the stdout. In case this is unwanted, you can redirect the output to /dev/null as follows:
echo "text" | tee 'Users/Name/Desktop/TheAccount.txt' > /dev/null

#!/bin/bash
cat <<EOF > SampleFile
Put Some text here
Put some text here
Put some text here
EOF

Just noting that if you wish to redirect both stderr and stdout to a file while you have noclobber set (i.e. set -o noclobber), you can use the code:
cmd >| file.txt 2>&1
More information about this can be seen at https://stackoverflow.com/a/876242.
Also this answer's #TuBui's question on the answer #BrDaHa provided above at Aug 9 '18 at 9:34.

To overwrite one file's content to another file you use the single greater than sign, using two will append.
echo "this is foo" > foobar.txt
cat foobar.txt
> this is foo
echo "this is bar" > foobar.txt
cat foobar.txt
> this is bar
echo "this is foo, again" >> foobar.txt
cat foobar.txt
> this is bar
> this is foo, again
As mentioned in other answers, if you have noclobber set then use the >| operator.

If you have output that can have errors, you may want to use an ampersand and a greater than, as follows:
my_task &> 'Users/Name/Desktop/task_output.log' this will redirect both stderr and stdout to the log file (instead of stdout only).

Related

Shell script to append file count to the same file [duplicate]

How do I append the output of a command to the end of a text file?
Use >> instead of > when directing output to a file:
your_command >> file_to_append_to
If file_to_append_to does not exist, it will be created.
Example:
$ echo "hello" > file
$ echo "world" >> file
$ cat file
hello
world
To append a file use >>
echo "hello world" >> read.txt
cat read.txt
echo "hello siva" >> read.txt
cat read.txt
then the output should be
hello world # from 1st echo command
hello world # from 2nd echo command
hello siva
To overwrite a file use >
echo "hello tom" > read.txt
cat read.txt
then the out put is
hello tom
You can use the >> operator. This will append data from a command to the end of a text file.
To test this try running:
echo "Hi this is a test" >> textfile.txt
Do this a couple of times and then run:
cat textfile.txt
You'll see your text has been appended several times to the textfile.txt file.
Use command >> file_to_append_to to append to a file.
For example echo "Hello" >> testFile.txt
CAUTION: if you only use a single > you will overwrite the contents of the file. To ensure that doesn't ever happen, you can add set -o noclobber to your .bashrc.
This ensures that if you accidentally type command > file_to_append_to to an existing file, it will alert you that the file exists already. Sample error message: file exists: testFile.txt
Thus, when you use > it will only allow you to create a new file, not overwrite an existing file.
Using tee with option -a (--append) allows you to append to multiple files at once and also to use sudo (very useful when appending to protected files). Besides that, it is interesting if you need to use other shells besides bash, as not all shells support the > and >> operators
echo "hello world" | sudo tee -a output.txt
This thread has good answers about tee
Use the >> operator to append text to a file.
I often confuse the two. Better to remember through their output:
> for Overwrite
$ touch someFile.txt
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" > someFile.txt
$ cat someFile.txt
>
>> for Append
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" >> someFile.txt
$ cat someFile.txt
>>
for the whole question:
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
this will append 720 lines (30*24) into o.txt and after will rename the file based on the current date.
Run the above with the cron every hour, or
while :
do
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
sleep 3600
done
I would use printf instead of echo because it's more reliable and processes formatting such as new line \n properly.
This example produces an output similar to echo in previous examples:
printf "hello world" >> read.txt
cat read.txt
hello world
However if you were to replace printf with echo in this example, echo would treat \n as a string, thus ignoring the intent
printf "hello\nworld" >> read.txt
cat read.txt
hello
world
I'd suggest you do two things:
Use >> in your shell script to append contents to particular file. The filename can be fixed or using some pattern.
Setup a hourly cronjob to trigger the shell script
For example your file contains :
1. mangesh#001:~$ cat output.txt
1
2
EOF
if u want to append at end of file then ---->remember spaces between 'text' >> 'filename'
2. mangesh#001:~$ echo somthing to append >> output.txt|cat output.txt
1
2
EOF
somthing to append
And to overwrite contents of file :
3. mangesh#001:~$ echo 'somthing new to write' > output.tx|cat output.tx
somthing new to write
In Linux, You can use cat command to append file content to another file
cat fileName_1.txt >> fileName_2.txt
In the previous command you will append content of fileName_1.txt to fileName_2.txt.
In Windows OS you can use type command
type fileName_1.txt >> fileName_2.txt
See this gif image:
While all of these answers are technically correct that appending to a file with >> is generally the way to go, note that if you use this in a loop when for example parsing/processing a file and append each line to the resulting file, this might be much slower then you would expect.
A faster alternative might be this:
stringBuilder=""
while read -r line; do
# $'\n' prints a newline so we don't have to know what special chars the string contains
stringBuilder+="$line"$'\n'
done < "myFile.txt"
echo "$stringBuilder" > $file
WARNING: you are reading all lines into memory; memory is a limited resource, so don't go doing this for gigantic files.

How to continuously dump dumpsys meminfo into a file? [duplicate]

How do I append the output of a command to the end of a text file?
Use >> instead of > when directing output to a file:
your_command >> file_to_append_to
If file_to_append_to does not exist, it will be created.
Example:
$ echo "hello" > file
$ echo "world" >> file
$ cat file
hello
world
To append a file use >>
echo "hello world" >> read.txt
cat read.txt
echo "hello siva" >> read.txt
cat read.txt
then the output should be
hello world # from 1st echo command
hello world # from 2nd echo command
hello siva
To overwrite a file use >
echo "hello tom" > read.txt
cat read.txt
then the out put is
hello tom
You can use the >> operator. This will append data from a command to the end of a text file.
To test this try running:
echo "Hi this is a test" >> textfile.txt
Do this a couple of times and then run:
cat textfile.txt
You'll see your text has been appended several times to the textfile.txt file.
Use command >> file_to_append_to to append to a file.
For example echo "Hello" >> testFile.txt
CAUTION: if you only use a single > you will overwrite the contents of the file. To ensure that doesn't ever happen, you can add set -o noclobber to your .bashrc.
This ensures that if you accidentally type command > file_to_append_to to an existing file, it will alert you that the file exists already. Sample error message: file exists: testFile.txt
Thus, when you use > it will only allow you to create a new file, not overwrite an existing file.
Using tee with option -a (--append) allows you to append to multiple files at once and also to use sudo (very useful when appending to protected files). Besides that, it is interesting if you need to use other shells besides bash, as not all shells support the > and >> operators
echo "hello world" | sudo tee -a output.txt
This thread has good answers about tee
Use the >> operator to append text to a file.
I often confuse the two. Better to remember through their output:
> for Overwrite
$ touch someFile.txt
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" > someFile.txt
$ cat someFile.txt
>
>> for Append
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" >> someFile.txt
$ cat someFile.txt
>>
for the whole question:
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
this will append 720 lines (30*24) into o.txt and after will rename the file based on the current date.
Run the above with the cron every hour, or
while :
do
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
sleep 3600
done
I would use printf instead of echo because it's more reliable and processes formatting such as new line \n properly.
This example produces an output similar to echo in previous examples:
printf "hello world" >> read.txt
cat read.txt
hello world
However if you were to replace printf with echo in this example, echo would treat \n as a string, thus ignoring the intent
printf "hello\nworld" >> read.txt
cat read.txt
hello
world
I'd suggest you do two things:
Use >> in your shell script to append contents to particular file. The filename can be fixed or using some pattern.
Setup a hourly cronjob to trigger the shell script
For example your file contains :
1. mangesh#001:~$ cat output.txt
1
2
EOF
if u want to append at end of file then ---->remember spaces between 'text' >> 'filename'
2. mangesh#001:~$ echo somthing to append >> output.txt|cat output.txt
1
2
EOF
somthing to append
And to overwrite contents of file :
3. mangesh#001:~$ echo 'somthing new to write' > output.tx|cat output.tx
somthing new to write
In Linux, You can use cat command to append file content to another file
cat fileName_1.txt >> fileName_2.txt
In the previous command you will append content of fileName_1.txt to fileName_2.txt.
In Windows OS you can use type command
type fileName_1.txt >> fileName_2.txt
See this gif image:
While all of these answers are technically correct that appending to a file with >> is generally the way to go, note that if you use this in a loop when for example parsing/processing a file and append each line to the resulting file, this might be much slower then you would expect.
A faster alternative might be this:
stringBuilder=""
while read -r line; do
# $'\n' prints a newline so we don't have to know what special chars the string contains
stringBuilder+="$line"$'\n'
done < "myFile.txt"
echo "$stringBuilder" > $file
WARNING: you are reading all lines into memory; memory is a limited resource, so don't go doing this for gigantic files.

Create temporary file and redirect output to it in one command

I designed a custom script to grep a concatenated list of .bash_history backup files. In my script, I am creating a temporary file with mktemp and saving it to a variable temp. Next, I am redirecting output to that file using the cat command.
Is there a means to create a temporary file (using mktemp), redirect output to it, then store it in a variable in one command, while preserving newline characters?
The below snippet of code works just fine, but I have a feeling there is a more terse and canonical way to achieve this in one line – maybe using process substitution or something of the like.
# Concatenate all .bash_history files into a temporary file `temp`.
temp="$(mktemp)"
cat "$HOME/.bash_history."* > $temp
trap 'rm -f $temp' 0
# Set `HISTFILE` shell variable to the `temp` file.
HISTFILE="$temp"
keyword="$1"
# Search for `keyword` using the `history` command
if [[ "$keyword" ]]; then
# Enable history
set -o history
history | grep "$keyword"
# Disable history
set +o history
else
echo -e "usage: search <keyword>"
exit 0
fi
If you're comfortable with the side effect of making the assignment conditional on tempfile not previously having a nonempty value, this is straightforward via the ${var:=value} expansion:
cat "$HOME/.bash_history" >"${tempfile:=$(mktemp)}"
cat myfile.txt | f=`mktemp` && cat > "${f}"
I guess there is more than one way to do it. I found following to be working for me:
cat myfile.txt > $(echo "$(mktemp)")
Also don't forget about tee:
cat myfile.txt | tee "$(mktemp)" > /dev/null

Bash script - stdout file descriptor?

I have the following in my script:
OUTFILE=./output.log
echo "foo" >> ${OUTFILE}
It works just fine when OUTFILE is an actual file path. However, sometimes I'd like to see the output on stdout by modifying OUTFILE but it doesn't work. I tried with 1 and &1, both quoted and unquoted, as well as leaving it empty.
It just keeps telling me this:
./foo.sh: line 2: ${OUTFILE}: ambiguous redirect
Use /dev/stdout as your filename for this. (See the Portability of “> /dev/stdout”.)
You can't use &1 in the variable because of parsing order issues. The redirection tokens are searched for before variable expansion is performed. So when you use:
$ o='&1'
$ echo >$o
the shell scans for redirection operators and sees the > redirection operator and not the >& operator. (And there isn't a >>& operator to begin with anyway so your appending example wouldn't work regardless. Though newer versions of bash do have an &>> operator for >> file 2>&1 use.)
Im guessing you want to do one of these
Print to file
OUTFILE=./output.log
echo "foo" >> "${OUTFILE}"
Print to stdout
OUTFILE=/dev/stdout
echo "foo" >> "${OUTFILE}"
or just
echo "foo"
Print to file and stdout
OUTFILE=./output.log
echo "foo" | tee "${OUTFILE}"

Unexpected output and error redirection in bash script

My Script:
#!/bin/bash
generic()
{
echo $1;
$1 > temp.txt 2>&1
}
generic "echo asd > /dev/null 2>&1; echo temp"
Expected Result:
"asd" should go to /dev/null
"temp" should go into temp.txt file.
Actual result:
"asd > /dev/null 2>&1; echo temp" goes into temp.txt file.
Why only first echo statement runs and its output is getting redirect to temp.txt. Also, why is the second command treated as string?
You need to read the spec
(This is the POSIX sh spec, because bash has no real spec. bash tries to be compatible to the POSIX sh, but adds a crapload of unnecessary complexity. You should use #!/bin/sh instead as a beginner and whenever you want to write a sane script)
In particular, you can't store anything other than a single command in a variable. If you execute from a variable as you do by saying $1, then this is interpreted as a single command:
Splitting occurrs on whitespace or whatever is inside the IFS variable. In you example, this result in the following token list:
echo
asd
>
/dev/null
2>&1;
echo
temp
This list is then executed as a process, equivalent to writing in C execlp("echo", "echo", "asd", ">", "/dev/null", "2>&1;", "echo", "temp", NULL);.
Note that if you had written "$1" instead, this would result in the shell trying to launch a program called echo asd > /dev/null 2>&1; echo temp with no arguments.
If you instead want to re-evaluate a string as a shell command line, you need to use eval: eval "$1". But be aware of the security implications! Eval is by many considered evil.

Resources