What is the easiest way to append text to a file in Linux?
I had a look at this question, but the accepted answer uses an additional program (sed) I'm sure there should be an easier way with echo or similar.
How about:
echo "hello" >> <filename>
Using the >> operator will append data at the end of the file, while using the > will overwrite the contents of the file if already existing.
You could also use printf in the same way:
printf "hello" >> <filename>
Note that it can be dangerous to use the above. For instance if you already have a file and you need to append data to the end of the file and you forget to add the last > all data in the file will be destroyed. You can change this behavior by setting the noclobber variable in your .bashrc:
set -o noclobber
Now when you try to do echo "hello" > file.txt you will get a warning saying cannot overwrite existing file.
To force writing to the file you must now use the special syntax:
echo "hello" >| <filename>
You should also know that by default echo adds a trailing new-line character which can be suppressed by using the -n flag:
echo -n "hello" >> <filename>
References
echo(1) - Linux man page
noclobber variable
I/O Redirection
cat >> filename
This is text, perhaps pasted in from some other source.
Or else entered at the keyboard, doesn't matter.
^D
Essentially, you can dump any text you want into the file. CTRL-D sends an end-of-file signal, which terminates input and returns you to the shell.
Other possible way is:
echo "text" | tee -a filename >/dev/null
The -a will append at the end of the file.
If needing sudo, use:
echo "text" | sudo tee -a filename >/dev/null
Follow up to accepted answer.
You need something other than CTRL-D to designate the end if using this in a script. Try this instead:
cat << EOF >> filename
This is text entered via the keyboard or via a script.
EOF
This will append text to the stated file (not including "EOF").
It utilizes a here document (or heredoc).
However if you need sudo to append to the stated file, you will run into trouble utilizing a heredoc due to I/O redirection if you're typing directly on the command line.
This variation will work when you are typing directly on the command line:
sudo sh -c 'cat << EOF >> filename
This is text entered via the keyboard.
EOF'
Or you can use tee instead to avoid the command line sudo issue seen when using the heredoc with cat:
tee -a filename << EOF
This is text entered via the keyboard or via a script.
EOF
Related
I'm trying to write a shell which would append a '.txt' file with some data(stored in a variable). This i'm trying to do using 'vi'. I know there are other tools too to append the file...but i need to use vi only
I tried below command, but unfortunately this command is not inserting the data to the end of the file.:
echo $'i{$var}\E:x\n' |vi file.txt
Using vi/vim does not allow you to do a command line edit of the file in-place. Instead you could use its command line equivalent tool ex(vi-summary.doc) which should be available in any POSIX compliant shell.
cat file
foo
bar
Now use the ex utility in the command line as
var=dude
printf '%s\n' '$a' "$var" '.' x | ex file
This would edit the file in-place and add the text dude at the last line of the file.
cat file
foo
bar
dude
I think this work too
var="value"
printf "$(cat file.txt)\n$var" > newfile.txt
I have this code
TOKEN=$(cat ./config/token)
echo "$TOKEN"
cat > variables.env <<EOF
TOKEN=`echo "$TOKEN"`
EOF
I am trying to get the content of a file and output it in a new file prefixed by some text. The first echo in the console echoes the output I want, keeping the whitespaces and newlines.
However, in the new file the output is just the first line of the original string, while I'd like the same output I can see in the console with the first echo.
Use printf %q (in ksh or bash) to escape content in such a way that it will always evaluate back to its literal value:
printf 'TOKEN=%q\n' "$(<./config/token)" >variables.env
$(<file) is a ksh and bash extension which acts as a more efficient replacement for $(cat file) (as the regular command substitution needs to fork off a subprocess, set up a FIFO, and spawn an external copy of /bin/cat, whereas the $(<file) form simply tells the shell to read the file directly).
This way a taken containing an otherwise-hostile string such as $(rm -rf ~) or content that could simply be expanded as a variable ($$) will be emitted as literal content.
Providing an explicit example of how this behaves:
printf '%s\n' "first line" "second line" >token # write two lines to the file "token"
printf 'TOKEN=%q\n' "$(<token)" >variables.env # write a shell command which assigns those
# two lines to a variable to variables.env
source variables.env # execute variables.env in the current shell
echo "$TOKEN" # emit the value of TOKEN, as given in the current shell
...when run with bash, will emit the exact output:
first line
second line
...after writing the following (with bash 3.2.48; may vary with other releases) to variables.env:
TOKEN=$'first line\nsecond line'
Useless use of echo
This is what you could write:
cat > variables.env <<EOF
TOKEN=${TOKEN}
EOF
you are doing it in a very convoluted way, there are easier methods
sed '1s/./TOKEN=&/' file > newfile
will insert TOKEN= on the first line. This has an additional benefit of not modifying empty files (at least one char should exist in the original file). If that's not intended you can use unconditional insert.
You can do:
echo "TOKEN=" > newfile && cat ./config/token >> newfile
>> appends to a file.
I designed a custom script to grep a concatenated list of .bash_history backup files. In my script, I am creating a temporary file with mktemp and saving it to a variable temp. Next, I am redirecting output to that file using the cat command.
Is there a means to create a temporary file (using mktemp), redirect output to it, then store it in a variable in one command, while preserving newline characters?
The below snippet of code works just fine, but I have a feeling there is a more terse and canonical way to achieve this in one line – maybe using process substitution or something of the like.
# Concatenate all .bash_history files into a temporary file `temp`.
temp="$(mktemp)"
cat "$HOME/.bash_history."* > $temp
trap 'rm -f $temp' 0
# Set `HISTFILE` shell variable to the `temp` file.
HISTFILE="$temp"
keyword="$1"
# Search for `keyword` using the `history` command
if [[ "$keyword" ]]; then
# Enable history
set -o history
history | grep "$keyword"
# Disable history
set +o history
else
echo -e "usage: search <keyword>"
exit 0
fi
If you're comfortable with the side effect of making the assignment conditional on tempfile not previously having a nonempty value, this is straightforward via the ${var:=value} expansion:
cat "$HOME/.bash_history" >"${tempfile:=$(mktemp)}"
cat myfile.txt | f=`mktemp` && cat > "${f}"
I guess there is more than one way to do it. I found following to be working for me:
cat myfile.txt > $(echo "$(mktemp)")
Also don't forget about tee:
cat myfile.txt | tee "$(mktemp)" > /dev/null
I have multiple files to edit. I have to make certain changes, like copying some lines based on patterns and then I have to insert some text at different places. I am planning to use vim to automate my task. I have written this script
gg
i
some text to add
some more text to add
<esc>
:%s/text/TEXT/g
:wq
But it just opens up the vim editor and inserts even the commands in the file and then I have to manually remove the following text
<esc>
:%s/text/TEXT/g
:wq
and save the file.
I am invoking the following command:
vi -s vimscript mytextfile
I have used vim scripting earlier to do other things than inserting text like searching and replacing or copy-pasting patterns etc.
Alternatives
Unless you really need special Vim capabilities, you're probably better off using non-interactive tools like sed, awk, or Perl / Python / Ruby / your favorite scripting language here.
That said, you can use Vim non-interactively:
Silent Batch Mode
For very simple text processing (i.e. using Vim like an enhanced 'sed' or 'awk', maybe just benefitting from the enhanced regular expressions in a :substitute command), use Ex-mode.
REM Windows
call vim -N -u NONE -n -es -S "commands.ex" "filespec"
Note: silent batch mode (:help -s-ex) messes up the Windows console, so you may have to do a cls to clean up after the Vim run.
# Unix
vim -T dumb --noplugin -n -es -S "commands.ex" "filespec"
Attention: Vim will hang waiting for input if the "commands.ex" file doesn't exist; better check beforehand for its existence! Alternatively, Vim can read the commands from stdin. You can also fill a new buffer with text read from stdin, and read commands from stderr if you use the - argument.
Full Automation
For more advanced processing involving multiple windows, and real automation of Vim (where you might interact with the user or leave Vim running to let the user take over), use:
vim -N -u NONE -n -c "set nomore" -S "commands.vim" "filespec"
Here's a summary of the used arguments:
-T dumb Avoids errors in case the terminal detection goes wrong.
-N -u NONE Do not load vimrc and plugins, alternatively:
--noplugin Do not load plugins.
-n No swapfile.
-es Ex mode + silent batch mode -s-ex
Attention: Must be given in that order!
-S ... Source script.
-c 'set nomore' Suppress the more-prompt when the screen is filled
with messages or output to avoid blocking.
From :help -s:
-s {scriptin} The script file "scriptin" is read. The characters in the
file are interpreted as if you had typed them. The same can
be done with the command ":source! {scriptin}". If the end
of the file is reached before the editor exits, further
characters are read from the keyboard. Only works when not
started in Ex mode, see |-s-ex|. See also |complex-repeat|.
{not in Vi}
Think of it as a macro. This is how your vimscript file should look:
Osome text to add^Msome more text to add^[:%s/text/TEXT^M:wq^M
The ^M special character is a literal <CR> and is obtained with <C-v><CR>.
The ^[ special character is a literal <Esc> and is obtained with <C-v><Esc>.
Why using vim to automate that kind of task whereas there are many commands or shell languages which are excellent and more efficient to do it ? You could use other tools instead of vim to automate your task.
You could try to use sed for exemple :
[ ~]$ cat file
Here is a file
[ ~]$ echo "some text to add" >> file
[ ~]$ echo "some more text to add" >> file
[ ~]$ cat file
Here is a file
some text to add
some more text to add
[ ~]$ sed -i "s/text/TEXT/g" file
[ ~]$ cat file
Here is a file
some TEXT to add
some more TEXT to add
EDIT : There are different ways to insert some text at the top of a file.
Using a temporary file :
echo "some text to add" > tmp.txt
cat file.text >> tmp.txt
mv tmp.text file.txt
Using sed :
sed -i "1isome text to add" file.text # or
sed -i "1s/^/some text to add\n/" file.txt
Using subshell :
echo -e "some text to add\n$(cat file.txt)" > file.txt
I find it more natural to use a standard Vim script, not a normal mode script. You can invoke it using:
vim -c "source vimscript" mytextfile
Your vim script will need a little update to work using this approach. It will look something like:
1
i
some text to add
some more text to add
.
%s/text/TEXT/g
wq
How can I write data to a text file automatically by shell scripting in Linux?
I was able to open the file. However, I don't know how to write data to it.
The short answer:
echo "some data for the file" >> fileName
However, echo doesn't deal with end of line characters (EOFs) in an ideal way. So, if you're going to append more than one line, do it with printf:
printf "some data for the file\nAnd a new line" >> fileName
The >> and > operators are very useful for redirecting output of commands, they work with multiple other bash commands.
#!/bin/sh
FILE="/path/to/file"
/bin/cat <<EOM >$FILE
text1
text2 # This comment will be inside of the file.
The keyword EOM can be any text, but it must start the line and be alone.
EOM # This will be also inside of the file, see the space in front of EOM.
EOM # No comments and spaces around here, or it will not work.
text4
EOM
You can redirect the output of a command to a file:
$ cat file > copy_file
or append to it
$ cat file >> copy_file
If you want to write directly the command is echo 'text'
$ echo 'Hello World' > file
#!/bin/bash
cat > FILE.txt <<EOF
info code info
info code info
info code info
EOF
I know this is a damn old question, but as the OP is about scripting, and for the fact that google brought me here, opening file descriptors for reading and writing at the same time should also be mentioned.
#!/bin/bash
# Open file descriptor (fd) 3 for read/write on a text file.
exec 3<> poem.txt
# Let's print some text to fd 3
echo "Roses are red" >&3
echo "Violets are blue" >&3
echo "Poems are cute" >&3
echo "And so are you" >&3
# Close fd 3
exec 3>&-
Then cat the file on terminal
$ cat poem.txt
Roses are red
Violets are blue
Poems are cute
And so are you
This example causes file poem.txt to be open for reading and writing on file descriptor 3. It also shows that *nix boxes know more fd's then just stdin, stdout and stderr (fd 0,1,2). It actually holds a lot. Usually the max number of file descriptors the kernel can allocate can be found in /proc/sys/file-max or /proc/sys/fs/file-max but using any fd above 9 is dangerous as it could conflict with fd's used by the shell internally. So don't bother and only use fd's 0-9. If you need more the 9 file descriptors in a bash script you should use a different language anyways :)
Anyhow, fd's can be used in a lot of interesting ways.
I like this answer:
cat > FILE.txt <<EOF
info code info
...
EOF
but would suggest cat >> FILE.txt << EOF if you want just add something to the end of the file without wiping out what is already exists
Like this:
cat >> FILE.txt <<EOF
info code info
...
EOF
Moving my comment as an answer, as requested by #lycono
If you need to do this with root privileges, do it this way:
sudo sh -c 'echo "some data for the file" >> fileName'
For environments where here documents are unavailable (Makefile, Dockerfile, etc) you can often use printf for a reasonably legible and efficient solution.
printf '%s\n' '#!/bin/sh' '# Second line' \
'# Third line' \
'# Conveniently mix single and double quotes, too' \
"# Generated $(date)" \
'# ^ the date command executes when the file is generated' \
'for file in *; do' \
' echo "Found $file"' \
'done' >outputfile
I thought there were a few perfectly fine answers, but no concise summary of all possibilities; thus:
The core principal behind most answers here is redirection. Two are important redirection operators for writing to files:
Redirecting Output:
echo 'text to completely overwrite contents of myfile' > myfile
Appending Redirected Output
echo 'text to add to end of myfile' >> myfile
Here Documents
Others mentioned, rather than from a fixed input source like echo 'text', you could also interactively write to files via a "Here Document", which are also detailed in the link to the bash manual above. Those answers, e.g.
cat > FILE.txt <<EOF` or `cat >> FILE.txt <<EOF
make use of the same redirection operators, but add another layer via "Here Documents". In the above syntax, you write to the FILE.txt via the output of cat. The writing only takes place after the interactive input is given some specific string, in this case 'EOF', but this could be any string, e.g.:
cat > FILE.txt <<'StopEverything'` or `cat >> FILE.txt <<'StopEverything'
would work just as well. Here Documents also look for various delimiters and other interesting parsing characters, so have a look at the docs for further info on that.
Here Strings
A bit convoluted, and more of an exercise in understanding both redirection and Here Documents syntax, but you could combine Here Document style syntax with standard redirect operators to become a Here String:
Redirecting Output of cat Input
cat > myfile <<<'text to completely overwrite contents of myfile'
Appending Redirected Output of cat Input
cat >> myfile <<<'text to completely overwrite contents of myfile'
This approach works and is the best
cat > (filename) <<EOF
Text1...
Text2...
EOF
Basically the text will search for keyword "EOF" till it terminates writing/appending the file
If you are using variables, you can use
first_var="Hello"
second_var="How are you"
If you want to concat both string and write it to file, then use below
echo "${first_var} - ${second_var}" > ./file_name.txt
Your file_name.txt content will be "Hello - How are you"
Can also use here document and vi, the below script generates a FILE.txt with 3 lines and variable interpolation
VAR=Test
vi FILE.txt <<EOFXX
i
#This is my var in text file
var = $VAR
#Thats end of text file
^[
ZZ
EOFXX
Then file will have 3 lines as below. "i" is to start vi insert mode and similarly to close the file with Esc and ZZ.
#This is my var in text file
var = Test
#Thats end of text file