I'm trying to cat some files together, while at the same time adding some text between files. I'm a Unix newbie and I don't have the hang of the syntax.
Here's my failed attempt:
cat echo "# Final version (reflecting my edits)\n\n" final.md echo "\n\n# The changes I made\n\n" edit.md echo "\n\n#Your original version\n\n" original.md > combined.md
How do I fix this? Should I be using pipes or something?
A process substitution seems to work:
$ cat <(echo 'FOO') foo.txt <(echo 'BAR') bar.txt
FOO
foo
BAR
bar
You can also use command substitution inside a here-document.
$ cat <<EOF
FOO
$(< foo.txt)
BAR
$(< bar.txt)
EOF
Use a command group to merge the output into one stream:
{
echo -e "# Final version (reflecting my edits)\n\n"
cat final.md
echo -e "\n\n# The changes I made\n\n"
cat edit.md
echo -e "\n\n#Your original version\n\n"
cat original.md
} > combined.md
There are tricks you can play with process substitution and command substitution (see Lev Levitsky's answer) to do it all with one command (instead of the separate cat processes used here), but this should be efficient enough with so few files.
If I understand you, it should be something like:
echo "# Final version (reflecting my edits)\n\n" >> combined.md
cat final.md >> combined.md
echo "\n\n# The changes I made\n\n" >> combined.md
cat edit.md >> combined.md
And so on.
Related
How do I append the output of a command to the end of a text file?
Use >> instead of > when directing output to a file:
your_command >> file_to_append_to
If file_to_append_to does not exist, it will be created.
Example:
$ echo "hello" > file
$ echo "world" >> file
$ cat file
hello
world
To append a file use >>
echo "hello world" >> read.txt
cat read.txt
echo "hello siva" >> read.txt
cat read.txt
then the output should be
hello world # from 1st echo command
hello world # from 2nd echo command
hello siva
To overwrite a file use >
echo "hello tom" > read.txt
cat read.txt
then the out put is
hello tom
You can use the >> operator. This will append data from a command to the end of a text file.
To test this try running:
echo "Hi this is a test" >> textfile.txt
Do this a couple of times and then run:
cat textfile.txt
You'll see your text has been appended several times to the textfile.txt file.
Use command >> file_to_append_to to append to a file.
For example echo "Hello" >> testFile.txt
CAUTION: if you only use a single > you will overwrite the contents of the file. To ensure that doesn't ever happen, you can add set -o noclobber to your .bashrc.
This ensures that if you accidentally type command > file_to_append_to to an existing file, it will alert you that the file exists already. Sample error message: file exists: testFile.txt
Thus, when you use > it will only allow you to create a new file, not overwrite an existing file.
Using tee with option -a (--append) allows you to append to multiple files at once and also to use sudo (very useful when appending to protected files). Besides that, it is interesting if you need to use other shells besides bash, as not all shells support the > and >> operators
echo "hello world" | sudo tee -a output.txt
This thread has good answers about tee
Use the >> operator to append text to a file.
I often confuse the two. Better to remember through their output:
> for Overwrite
$ touch someFile.txt
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" > someFile.txt
$ cat someFile.txt
>
>> for Append
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" >> someFile.txt
$ cat someFile.txt
>>
for the whole question:
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
this will append 720 lines (30*24) into o.txt and after will rename the file based on the current date.
Run the above with the cron every hour, or
while :
do
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
sleep 3600
done
I would use printf instead of echo because it's more reliable and processes formatting such as new line \n properly.
This example produces an output similar to echo in previous examples:
printf "hello world" >> read.txt
cat read.txt
hello world
However if you were to replace printf with echo in this example, echo would treat \n as a string, thus ignoring the intent
printf "hello\nworld" >> read.txt
cat read.txt
hello
world
I'd suggest you do two things:
Use >> in your shell script to append contents to particular file. The filename can be fixed or using some pattern.
Setup a hourly cronjob to trigger the shell script
For example your file contains :
1. mangesh#001:~$ cat output.txt
1
2
EOF
if u want to append at end of file then ---->remember spaces between 'text' >> 'filename'
2. mangesh#001:~$ echo somthing to append >> output.txt|cat output.txt
1
2
EOF
somthing to append
And to overwrite contents of file :
3. mangesh#001:~$ echo 'somthing new to write' > output.tx|cat output.tx
somthing new to write
In Linux, You can use cat command to append file content to another file
cat fileName_1.txt >> fileName_2.txt
In the previous command you will append content of fileName_1.txt to fileName_2.txt.
In Windows OS you can use type command
type fileName_1.txt >> fileName_2.txt
See this gif image:
While all of these answers are technically correct that appending to a file with >> is generally the way to go, note that if you use this in a loop when for example parsing/processing a file and append each line to the resulting file, this might be much slower then you would expect.
A faster alternative might be this:
stringBuilder=""
while read -r line; do
# $'\n' prints a newline so we don't have to know what special chars the string contains
stringBuilder+="$line"$'\n'
done < "myFile.txt"
echo "$stringBuilder" > $file
WARNING: you are reading all lines into memory; memory is a limited resource, so don't go doing this for gigantic files.
I know I can use bash's process substitution feature to specify a file parameter via process substitution and then use a here document to specify the data, like so:
foo --config <(cat <<EOF
# contents of config file
...
EOF
)
When I need the same here document at two different locations in the same script, then it would be more useful to store the here document in a variable.
How can I do something like this:
read -r -d '' MY_CONFIG <<EOF
# contents of config file
...
EOF
Then how can I call foo to pass the contents of $MY_CONFIG to the --config parameter?
Additionally, you can use a herestring inside of process substitution:
MY_CONFIG="hello world"
md5sum <( <<< "$MY_CONFIG" )
The solution occurred to my while writing this question. Rather then using <(cat ...), we can use <(echo ...) to write something from the process substitution. So we can simply use:
foo --config <(echo "$MY_CONFIG")
A simple example to test this would be this:
MY_CONFIG="hello world"
less -f <(echo "$MY_CONFIG")
This should open less and show hello world in the buffer.
I consider the solution you found yourself simpler, but for sake of completeness, here is how you could do it with here-docs.
To store a here-doc in a variable, you can use (> is the secondary prompt string):
$ var=$(cat <<EOF
> line1
> line2
> EOF
> )
$ declare -p var
declare -- var="line1
line2"
Then, to use a here-doc with the contents of that variable:
$ cat <<EOF
> $var
> EOF
line1
line2
I.e., for your case, something like this:
$ foo --config <(cat <<EOF
> $var
> EOF
> )
As I said, it's clearly simpler to just use <(echo "$var") or <(printf '%s\n' "$var") instead.
How do I append the output of a command to the end of a text file?
Use >> instead of > when directing output to a file:
your_command >> file_to_append_to
If file_to_append_to does not exist, it will be created.
Example:
$ echo "hello" > file
$ echo "world" >> file
$ cat file
hello
world
To append a file use >>
echo "hello world" >> read.txt
cat read.txt
echo "hello siva" >> read.txt
cat read.txt
then the output should be
hello world # from 1st echo command
hello world # from 2nd echo command
hello siva
To overwrite a file use >
echo "hello tom" > read.txt
cat read.txt
then the out put is
hello tom
You can use the >> operator. This will append data from a command to the end of a text file.
To test this try running:
echo "Hi this is a test" >> textfile.txt
Do this a couple of times and then run:
cat textfile.txt
You'll see your text has been appended several times to the textfile.txt file.
Use command >> file_to_append_to to append to a file.
For example echo "Hello" >> testFile.txt
CAUTION: if you only use a single > you will overwrite the contents of the file. To ensure that doesn't ever happen, you can add set -o noclobber to your .bashrc.
This ensures that if you accidentally type command > file_to_append_to to an existing file, it will alert you that the file exists already. Sample error message: file exists: testFile.txt
Thus, when you use > it will only allow you to create a new file, not overwrite an existing file.
Using tee with option -a (--append) allows you to append to multiple files at once and also to use sudo (very useful when appending to protected files). Besides that, it is interesting if you need to use other shells besides bash, as not all shells support the > and >> operators
echo "hello world" | sudo tee -a output.txt
This thread has good answers about tee
Use the >> operator to append text to a file.
I often confuse the two. Better to remember through their output:
> for Overwrite
$ touch someFile.txt
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" > someFile.txt
$ cat someFile.txt
>
>> for Append
$ echo ">" > someFile.txt
$ cat someFile.txt
>
$ echo ">" >> someFile.txt
$ cat someFile.txt
>>
for the whole question:
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
this will append 720 lines (30*24) into o.txt and after will rename the file based on the current date.
Run the above with the cron every hour, or
while :
do
cmd >> o.txt && [[ $(wc -l <o.txt) -eq 720 ]] && mv o.txt $(date +%F).o.txt
sleep 3600
done
I would use printf instead of echo because it's more reliable and processes formatting such as new line \n properly.
This example produces an output similar to echo in previous examples:
printf "hello world" >> read.txt
cat read.txt
hello world
However if you were to replace printf with echo in this example, echo would treat \n as a string, thus ignoring the intent
printf "hello\nworld" >> read.txt
cat read.txt
hello
world
I'd suggest you do two things:
Use >> in your shell script to append contents to particular file. The filename can be fixed or using some pattern.
Setup a hourly cronjob to trigger the shell script
For example your file contains :
1. mangesh#001:~$ cat output.txt
1
2
EOF
if u want to append at end of file then ---->remember spaces between 'text' >> 'filename'
2. mangesh#001:~$ echo somthing to append >> output.txt|cat output.txt
1
2
EOF
somthing to append
And to overwrite contents of file :
3. mangesh#001:~$ echo 'somthing new to write' > output.tx|cat output.tx
somthing new to write
In Linux, You can use cat command to append file content to another file
cat fileName_1.txt >> fileName_2.txt
In the previous command you will append content of fileName_1.txt to fileName_2.txt.
In Windows OS you can use type command
type fileName_1.txt >> fileName_2.txt
See this gif image:
While all of these answers are technically correct that appending to a file with >> is generally the way to go, note that if you use this in a loop when for example parsing/processing a file and append each line to the resulting file, this might be much slower then you would expect.
A faster alternative might be this:
stringBuilder=""
while read -r line; do
# $'\n' prints a newline so we don't have to know what special chars the string contains
stringBuilder+="$line"$'\n'
done < "myFile.txt"
echo "$stringBuilder" > $file
WARNING: you are reading all lines into memory; memory is a limited resource, so don't go doing this for gigantic files.
I'm trying to write a shell script that takes in a file(ex. file_1_2.txt) and replaces any "_" with "."(ex. file.1.2.txt). This is what I have but its giving me a blank output when I run it.
read $var
x= `echo $var | sed 's/\./_/g'`
echo $x
I'm trying to store the changed filename in the variable "x" and then output x to the console.
I am calling this script by writing
./script2.sh < file_1_2.txt
There is two problems. First, your code has some bugs:
read var
x=`echo $var | sed 's/_/\./g'`
echo $x
will work. You had an extra $ in read var, a space too much (as mentioned before) and you mixed up the replacement pattern in sed (it was doing the reverse of what you wanted).
Also if you want to replace the _ by . in the filename you should do
echo "file_1_2.txt" | ./script2.sh
If you use < this will read the content of `file_1_2.txt" into your script.
Another solution, with bash only:
$ x=file_1_2.txt; echo "${x//_/.}"
file.1.2.txt
(See “Parameter expansion” section in bash manual page for details)
And you can also do this with rename:
$ touch file_1_2.txt
$ ls file*
file_1_2.txt
$ rename 'y/_/\./' file_1_2.txt
$ ls file*
file.1.2.txt
Threre is not need for sed as bash supports variable replacement:
$ cat ./script2
#!/bin/bash
ofile=$1
nfile=${ofile//_/./}
echo mv "$ofile" "$nfile"
$ ./script2 file_1_2.txt
mv "file_1_2.txt" "file.1.2.txt"
Then just remove echo if you are satisfied with the result.
echo "text" >> 'Users/Name/Desktop/TheAccount.txt'
How do I make it so it creates the file if it doesn't exist, but overwrites it if it already exists. Right now this script just appends.
The >> redirection operator will append lines to the end of the specified file, where-as the single greater than > will empty and overwrite the file.
echo "text" > 'Users/Name/Desktop/TheAccount.txt'
In Bash, if you have set noclobber a la set -o noclobber, then you use the syntax >|
For example:
echo "some text" >| existing_file
This also works if the file doesn't exist yet
Check if noclobber is set with: set -o | grep noclobber
For a more detailed explanation on this special type of operator, see this post
For a more exhaustive list of redirection operators, refer to this post
Despite NylonSmile's answer, which is "sort of" correct.. I was unable to overwrite files, in this manner..
echo "i know about Pipes, girlfriend" > thatAnswer
zsh: file exists: thatAnswer
to solve my issues.. I had to use... >!, á la..
[[ $FORCE_IT == 'YES' ]] && echo "$#" >! "$X" || echo "$#" > "$X"
Obviously, be careful with this...
If your environment doesn't allow overwriting with >, use pipe | and tee instead as follows:
echo "text" | tee 'Users/Name/Desktop/TheAccount.txt'
Note this will also print to the stdout. In case this is unwanted, you can redirect the output to /dev/null as follows:
echo "text" | tee 'Users/Name/Desktop/TheAccount.txt' > /dev/null
#!/bin/bash
cat <<EOF > SampleFile
Put Some text here
Put some text here
Put some text here
EOF
Just noting that if you wish to redirect both stderr and stdout to a file while you have noclobber set (i.e. set -o noclobber), you can use the code:
cmd >| file.txt 2>&1
More information about this can be seen at https://stackoverflow.com/a/876242.
Also this answer's #TuBui's question on the answer #BrDaHa provided above at Aug 9 '18 at 9:34.
To overwrite one file's content to another file you use the single greater than sign, using two will append.
echo "this is foo" > foobar.txt
cat foobar.txt
> this is foo
echo "this is bar" > foobar.txt
cat foobar.txt
> this is bar
echo "this is foo, again" >> foobar.txt
cat foobar.txt
> this is bar
> this is foo, again
As mentioned in other answers, if you have noclobber set then use the >| operator.
If you have output that can have errors, you may want to use an ampersand and a greater than, as follows:
my_task &> 'Users/Name/Desktop/task_output.log' this will redirect both stderr and stdout to the log file (instead of stdout only).