Shell script to append file count to the same file [duplicate] - bash

How do I append the output of a command to the end of a text file?

Use >> instead of > when directing output to a file:
your_command >> file_to_append_to
If file_to_append_to does not exist, it will be created.
Example:
$ echo "hello" > file
$ echo "world" >> file
$ cat file
hello
world

To append a file use >>
echo "hello world" >> read.txt
cat read.txt
echo "hello siva" >> read.txt
cat read.txt
then the output should be
hello world # from 1st echo command
hello world # from 2nd echo command
hello siva
To overwrite a file use >
echo "hello tom" > read.txt
cat read.txt
then the out put is
hello tom

You can use the >> operator. This will append data from a command to the end of a text file.
To test this try running:
echo "Hi this is a test" >> textfile.txt
Do this a couple of times and then run:
cat textfile.txt
You'll see your text has been appended several times to the textfile.txt file.

Use command >> file_to_append_to to append to a file.
For example echo "Hello" >> testFile.txt
CAUTION: if you only use a single > you will overwrite the contents of the file. To ensure that doesn't ever happen, you can add set -o noclobber to your .bashrc.
This ensures that if you accidentally type command > file_to_append_to to an existing file, it will alert you that the file exists already. Sample error message: file exists: testFile.txt
Thus, when you use > it will only allow you to create a new file, not overwrite an existing file.

Using tee with option -a (--append) allows you to append to multiple files at once and also to use sudo (very useful when appending to protected files). Besides that, it is interesting if you need to use other shells besides bash, as not all shells support the > and >> operators
echo "hello world" | sudo tee -a output.txt
This thread has good answers about tee

Use the >> operator to append text to a file.

I often confuse the two. Better to remember through their output:
> for Overwrite
$ touch someFile.txt
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" > someFile.txt
$ cat someFile.txt
>
>> for Append
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" >> someFile.txt
$ cat someFile.txt
>>

for the whole question:
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
this will append 720 lines (30*24) into o.txt and after will rename the file based on the current date.
Run the above with the cron every hour, or
while :
do
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
sleep 3600
done

I would use printf instead of echo because it's more reliable and processes formatting such as new line \n properly.
This example produces an output similar to echo in previous examples:
printf "hello world" >> read.txt
cat read.txt
hello world
However if you were to replace printf with echo in this example, echo would treat \n as a string, thus ignoring the intent
printf "hello\nworld" >> read.txt
cat read.txt
hello
world

I'd suggest you do two things:
Use >> in your shell script to append contents to particular file. The filename can be fixed or using some pattern.
Setup a hourly cronjob to trigger the shell script

For example your file contains :
1. mangesh#001:~$ cat output.txt
1
2
EOF
if u want to append at end of file then ---->remember spaces between 'text' >> 'filename'
2. mangesh#001:~$ echo somthing to append >> output.txt|cat output.txt
1
2
EOF
somthing to append
And to overwrite contents of file :
3. mangesh#001:~$ echo 'somthing new to write' > output.tx|cat output.tx
somthing new to write

In Linux, You can use cat command to append file content to another file
cat fileName_1.txt >> fileName_2.txt
In the previous command you will append content of fileName_1.txt to fileName_2.txt.
In Windows OS you can use type command
type fileName_1.txt >> fileName_2.txt
See this gif image:

While all of these answers are technically correct that appending to a file with >> is generally the way to go, note that if you use this in a loop when for example parsing/processing a file and append each line to the resulting file, this might be much slower then you would expect.
A faster alternative might be this:
stringBuilder=""
while read -r line; do
# $'\n' prints a newline so we don't have to know what special chars the string contains
stringBuilder+="$line"$'\n'
done < "myFile.txt"
echo "$stringBuilder" > $file
WARNING: you are reading all lines into memory; memory is a limited resource, so don't go doing this for gigantic files.

Related

How to continuously dump dumpsys meminfo into a file? [duplicate]

How do I append the output of a command to the end of a text file?
Use >> instead of > when directing output to a file:
your_command >> file_to_append_to
If file_to_append_to does not exist, it will be created.
Example:
$ echo "hello" > file
$ echo "world" >> file
$ cat file
hello
world
To append a file use >>
echo "hello world" >> read.txt
cat read.txt
echo "hello siva" >> read.txt
cat read.txt
then the output should be
hello world # from 1st echo command
hello world # from 2nd echo command
hello siva
To overwrite a file use >
echo "hello tom" > read.txt
cat read.txt
then the out put is
hello tom
You can use the >> operator. This will append data from a command to the end of a text file.
To test this try running:
echo "Hi this is a test" >> textfile.txt
Do this a couple of times and then run:
cat textfile.txt
You'll see your text has been appended several times to the textfile.txt file.
Use command >> file_to_append_to to append to a file.
For example echo "Hello" >> testFile.txt
CAUTION: if you only use a single > you will overwrite the contents of the file. To ensure that doesn't ever happen, you can add set -o noclobber to your .bashrc.
This ensures that if you accidentally type command > file_to_append_to to an existing file, it will alert you that the file exists already. Sample error message: file exists: testFile.txt
Thus, when you use > it will only allow you to create a new file, not overwrite an existing file.
Using tee with option -a (--append) allows you to append to multiple files at once and also to use sudo (very useful when appending to protected files). Besides that, it is interesting if you need to use other shells besides bash, as not all shells support the > and >> operators
echo "hello world" | sudo tee -a output.txt
This thread has good answers about tee
Use the >> operator to append text to a file.
I often confuse the two. Better to remember through their output:
> for Overwrite
$ touch someFile.txt
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" > someFile.txt
$ cat someFile.txt
>
>> for Append
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" >> someFile.txt
$ cat someFile.txt
>>
for the whole question:
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
this will append 720 lines (30*24) into o.txt and after will rename the file based on the current date.
Run the above with the cron every hour, or
while :
do
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
sleep 3600
done
I would use printf instead of echo because it's more reliable and processes formatting such as new line \n properly.
This example produces an output similar to echo in previous examples:
printf "hello world" >> read.txt
cat read.txt
hello world
However if you were to replace printf with echo in this example, echo would treat \n as a string, thus ignoring the intent
printf "hello\nworld" >> read.txt
cat read.txt
hello
world
I'd suggest you do two things:
Use >> in your shell script to append contents to particular file. The filename can be fixed or using some pattern.
Setup a hourly cronjob to trigger the shell script
For example your file contains :
1. mangesh#001:~$ cat output.txt
1
2
EOF
if u want to append at end of file then ---->remember spaces between 'text' >> 'filename'
2. mangesh#001:~$ echo somthing to append >> output.txt|cat output.txt
1
2
EOF
somthing to append
And to overwrite contents of file :
3. mangesh#001:~$ echo 'somthing new to write' > output.tx|cat output.tx
somthing new to write
In Linux, You can use cat command to append file content to another file
cat fileName_1.txt >> fileName_2.txt
In the previous command you will append content of fileName_1.txt to fileName_2.txt.
In Windows OS you can use type command
type fileName_1.txt >> fileName_2.txt
See this gif image:
While all of these answers are technically correct that appending to a file with >> is generally the way to go, note that if you use this in a loop when for example parsing/processing a file and append each line to the resulting file, this might be much slower then you would expect.
A faster alternative might be this:
stringBuilder=""
while read -r line; do
# $'\n' prints a newline so we don't have to know what special chars the string contains
stringBuilder+="$line"$'\n'
done < "myFile.txt"
echo "$stringBuilder" > $file
WARNING: you are reading all lines into memory; memory is a limited resource, so don't go doing this for gigantic files.

Bourne Shell - Read and print tab from file

I am trying, so far unsuccessfully to read and print a tab character from a file in a Bourne shell script.
For example, here is my file, in.txt (stackoverflow won't let me write a tab, so replace [tabcharacter] with a tab):
[tabcharacter]Hello World!
My script as as follows:
#!/bin/sh
while read line
do
echo -e "${line}" >> out.txt
/bin/echo -e "${line}" >> out.txt
done < "./in.txt"
The out.txt I get is:
-e hello!
hello!
Whereas I would expect from one of these the output to be the same as in.txt.
I think it's a problem with the way I use the read command. But I'm not sure how I can get it to read tabs.
Any help much appreciated.
#!/bin/sh
export IFS=
while read line
do
echo -e "$line" >> out.txt
/bin/echo -e "$line" >> out.txt
done < "./in.txt"
I seted the IFS variable to a empty string, now its working, please test it!

How to add literal text to the Unix 'cat' command

I'm trying to cat some files together, while at the same time adding some text between files. I'm a Unix newbie and I don't have the hang of the syntax.
Here's my failed attempt:
cat echo "# Final version (reflecting my edits)\n\n" final.md echo "\n\n# The changes I made\n\n" edit.md echo "\n\n#Your original version\n\n" original.md > combined.md
How do I fix this? Should I be using pipes or something?
A process substitution seems to work:
$ cat <(echo 'FOO') foo.txt <(echo 'BAR') bar.txt
FOO
foo
BAR
bar
You can also use command substitution inside a here-document.
$ cat <<EOF
FOO
$(< foo.txt)
BAR
$(< bar.txt)
EOF
Use a command group to merge the output into one stream:
{
echo -e "# Final version (reflecting my edits)\n\n"
cat final.md
echo -e "\n\n# The changes I made\n\n"
cat edit.md
echo -e "\n\n#Your original version\n\n"
cat original.md
} > combined.md
There are tricks you can play with process substitution and command substitution (see Lev Levitsky's answer) to do it all with one command (instead of the separate cat processes used here), but this should be efficient enough with so few files.
If I understand you, it should be something like:
echo "# Final version (reflecting my edits)\n\n" >> combined.md
cat final.md >> combined.md
echo "\n\n# The changes I made\n\n" >> combined.md
cat edit.md >> combined.md
And so on.

Open and write data to text file using Bash?

How can I write data to a text file automatically by shell scripting in Linux?
I was able to open the file. However, I don't know how to write data to it.
The short answer:
echo "some data for the file" >> fileName
However, echo doesn't deal with end of line characters (EOFs) in an ideal way. So, if you're going to append more than one line, do it with printf:
printf "some data for the file\nAnd a new line" >> fileName
The >> and > operators are very useful for redirecting output of commands, they work with multiple other bash commands.
#!/bin/sh
FILE="/path/to/file"
/bin/cat <<EOM >$FILE
text1
text2 # This comment will be inside of the file.
The keyword EOM can be any text, but it must start the line and be alone.
EOM # This will be also inside of the file, see the space in front of EOM.
EOM # No comments and spaces around here, or it will not work.
text4
EOM
You can redirect the output of a command to a file:
$ cat file > copy_file
or append to it
$ cat file >> copy_file
If you want to write directly the command is echo 'text'
$ echo 'Hello World' > file
#!/bin/bash
cat > FILE.txt <<EOF
info code info
info code info
info code info
EOF
I know this is a damn old question, but as the OP is about scripting, and for the fact that google brought me here, opening file descriptors for reading and writing at the same time should also be mentioned.
#!/bin/bash
# Open file descriptor (fd) 3 for read/write on a text file.
exec 3<> poem.txt
# Let's print some text to fd 3
echo "Roses are red" >&3
echo "Violets are blue" >&3
echo "Poems are cute" >&3
echo "And so are you" >&3
# Close fd 3
exec 3>&-
Then cat the file on terminal
$ cat poem.txt
Roses are red
Violets are blue
Poems are cute
And so are you
This example causes file poem.txt to be open for reading and writing on file descriptor 3. It also shows that *nix boxes know more fd's then just stdin, stdout and stderr (fd 0,1,2). It actually holds a lot. Usually the max number of file descriptors the kernel can allocate can be found in /proc/sys/file-max or /proc/sys/fs/file-max but using any fd above 9 is dangerous as it could conflict with fd's used by the shell internally. So don't bother and only use fd's 0-9. If you need more the 9 file descriptors in a bash script you should use a different language anyways :)
Anyhow, fd's can be used in a lot of interesting ways.
I like this answer:
cat > FILE.txt <<EOF
info code info
...
EOF
but would suggest cat >> FILE.txt << EOF if you want just add something to the end of the file without wiping out what is already exists
Like this:
cat >> FILE.txt <<EOF
info code info
...
EOF
Moving my comment as an answer, as requested by #lycono
If you need to do this with root privileges, do it this way:
sudo sh -c 'echo "some data for the file" >> fileName'
For environments where here documents are unavailable (Makefile, Dockerfile, etc) you can often use printf for a reasonably legible and efficient solution.
printf '%s\n' '#!/bin/sh' '# Second line' \
'# Third line' \
'# Conveniently mix single and double quotes, too' \
"# Generated $(date)" \
'# ^ the date command executes when the file is generated' \
'for file in *; do' \
' echo "Found $file"' \
'done' >outputfile
I thought there were a few perfectly fine answers, but no concise summary of all possibilities; thus:
The core principal behind most answers here is redirection. Two are important redirection operators for writing to files:
Redirecting Output:
echo 'text to completely overwrite contents of myfile' > myfile
Appending Redirected Output
echo 'text to add to end of myfile' >> myfile
Here Documents
Others mentioned, rather than from a fixed input source like echo 'text', you could also interactively write to files via a "Here Document", which are also detailed in the link to the bash manual above. Those answers, e.g.
cat > FILE.txt <<EOF` or `cat >> FILE.txt <<EOF
make use of the same redirection operators, but add another layer via "Here Documents". In the above syntax, you write to the FILE.txt via the output of cat. The writing only takes place after the interactive input is given some specific string, in this case 'EOF', but this could be any string, e.g.:
cat > FILE.txt <<'StopEverything'` or `cat >> FILE.txt <<'StopEverything'
would work just as well. Here Documents also look for various delimiters and other interesting parsing characters, so have a look at the docs for further info on that.
Here Strings
A bit convoluted, and more of an exercise in understanding both redirection and Here Documents syntax, but you could combine Here Document style syntax with standard redirect operators to become a Here String:
Redirecting Output of cat Input
cat > myfile <<<'text to completely overwrite contents of myfile'
Appending Redirected Output of cat Input
cat >> myfile <<<'text to completely overwrite contents of myfile'
This approach works and is the best
cat > (filename) <<EOF
Text1...
Text2...
EOF
Basically the text will search for keyword "EOF" till it terminates writing/appending the file
If you are using variables, you can use
first_var="Hello"
second_var="How are you"
If you want to concat both string and write it to file, then use below
echo "${first_var} - ${second_var}" > ./file_name.txt
Your file_name.txt content will be "Hello - How are you"
Can also use here document and vi, the below script generates a FILE.txt with 3 lines and variable interpolation
VAR=Test
vi FILE.txt <<EOFXX
i
#This is my var in text file
var = $VAR
#Thats end of text file
^[
ZZ
EOFXX
Then file will have 3 lines as below. "i" is to start vi insert mode and similarly to close the file with Esc and ZZ.
#This is my var in text file
var = Test
#Thats end of text file

Write to file, but overwrite it if it exists

echo "text" >> 'Users/Name/Desktop/TheAccount.txt'
How do I make it so it creates the file if it doesn't exist, but overwrites it if it already exists. Right now this script just appends.
The >> redirection operator will append lines to the end of the specified file, where-as the single greater than > will empty and overwrite the file.
echo "text" > 'Users/Name/Desktop/TheAccount.txt'
In Bash, if you have set noclobber a la set -o noclobber, then you use the syntax >|
For example:
echo "some text" >| existing_file
This also works if the file doesn't exist yet
Check if noclobber is set with: set -o | grep noclobber
For a more detailed explanation on this special type of operator, see this post
For a more exhaustive list of redirection operators, refer to this post
Despite NylonSmile's answer, which is "sort of" correct.. I was unable to overwrite files, in this manner..
echo "i know about Pipes, girlfriend" > thatAnswer
zsh: file exists: thatAnswer
to solve my issues.. I had to use... >!, รก la..
[[ $FORCE_IT == 'YES' ]] && echo "$#" >! "$X" || echo "$#" > "$X"
Obviously, be careful with this...
If your environment doesn't allow overwriting with >, use pipe | and tee instead as follows:
echo "text" | tee 'Users/Name/Desktop/TheAccount.txt'
Note this will also print to the stdout. In case this is unwanted, you can redirect the output to /dev/null as follows:
echo "text" | tee 'Users/Name/Desktop/TheAccount.txt' > /dev/null
#!/bin/bash
cat <<EOF > SampleFile
Put Some text here
Put some text here
Put some text here
EOF
Just noting that if you wish to redirect both stderr and stdout to a file while you have noclobber set (i.e. set -o noclobber), you can use the code:
cmd >| file.txt 2>&1
More information about this can be seen at https://stackoverflow.com/a/876242.
Also this answer's #TuBui's question on the answer #BrDaHa provided above at Aug 9 '18 at 9:34.
To overwrite one file's content to another file you use the single greater than sign, using two will append.
echo "this is foo" > foobar.txt
cat foobar.txt
> this is foo
echo "this is bar" > foobar.txt
cat foobar.txt
> this is bar
echo "this is foo, again" >> foobar.txt
cat foobar.txt
> this is bar
> this is foo, again
As mentioned in other answers, if you have noclobber set then use the >| operator.
If you have output that can have errors, you may want to use an ampersand and a greater than, as follows:
my_task &> 'Users/Name/Desktop/task_output.log' this will redirect both stderr and stdout to the log file (instead of stdout only).

Resources