I have a folder full of text files. I need to append the same block of text to each of them (and of course overwrite the original file).
I was wondering what the correct Bash shell syntax would be for this. Would I use cat?
I have done a few batch scripts but I'm not a Bash expert. Any suggestions appreciated.
Use append redirection.
for f in *.txt
do
cat footer >> "$f"
done
If you're needing to do this via a script, you can use echo and append redirection to get the extra text into the files.
FILES=pathto/*
for f in $FILES ; do
echo "#extra text" >> $f
done
sed -i.bak "$ a $(<file_block_of_text)" *.txt
Variant of kurumi's answer:
sed -i.bak "\$aTEXTTOINSERT" *.txt
For more details, see SED: insert something to the last line?
very simply one which worked well for me
#!/bin/sh
FILES="./files/*"
for f in $FILES
do
echo '0000000' | cat - $f > temp && mv temp $f
done
Related
I'm trying to change multiple file names with a for loop.
This works to send the output to the screen:
for i in *.gz; do echo $i | sed 's/\-//g'; done
However, when I try to overwrite the file name using sed -i, I get this error:
for i in *.gz; do echo $i | sed -i 's/\-//g'; done
sed: no input files
Any suggestions?
there is a command for this
$ rename - '' *.gz
NB. this is the standard one, not the advanced perl version.
Use Perl rename instead:
rename 's/-//g' *.gz
Or use simple parameter expansion:
for i in *.gz; do mv -- "$i" "${i//-}"; done
I have thousands of two set of files one with name.ext and another files name ending with name.ext.in, so for every name.ext there is a name.ext.in and now i have to pass this as argument to a script such as customise.pl name.ext name.ext.in. I am doing like
#!/bin/bash
FILE1=$.ext
FILE2=$.ext.in
customise.pl $FILE1 $FILE2
but no success. Any idea?
for i in *.ext; do
customise.pl "$i" "$i.in"
done
Judging from the comments to the other answers, you probably want something like this:
for file in *.ext; do
customise.pl "$file" "${file%.*}.new.psl"
done
The ${file%.*} syntax substitutes only the part of $file up until its last dot. You can check the manpages for Bash or Dash for more information on it if you need.
If you want to run customise.pl for each pair of files, you can do it like this in a bash script:
#!/bin/bash
for i in `ls *.ext`
do
customise.pl $i $i.in
done
I guess the simplest form is
for f in "*.ext"
do
customise.pl $f.ext $f.ext.in
done
OUTPUT:
another way.
ls *.ext |xargs -i customise.pl {} {}.in
I would like to traverse through some directories that contain xml files. To each file I would like to apply an awk command and write the output to the file again. The file that is written is empty, though. Why? I use the following script:
#!/bin/bash
for f in `find /myDir/ -iname "*.xml"`
do
awk '/<string name="my\/ attribute" optional="false">/,/<\/string>/ {sub(/<value>(.*)<\/value>/,"<value>none</value>")}1' $f >$f
done
You must use a temporary file:
awk ... "$f" > "$f".tmp; mv "$f".tmp "$f"
The problem is not in awk, the problem is in the redirection. It is not possible to write to the file you are reading.
I am using sed in a script to do a replace and I want to have the replaced file overwrite the file. Normally I think that you would use this:
% sed -i 's/cat/dog/' manipulate
sed: illegal option -- i
However as you can see my sed does not have that command.
I tried this:
% sed 's/cat/dog/' manipulate > manipulate
But this just turns manipulate into an empty file (makes sense).
This works:
% sed 's/cat/dog/' manipulate > tmp; mv tmp manipulate
But I was wondering if there was a standard way to redirect output into the same file that input was taken from.
I commonly use the 3rd way, but with an important change:
$ sed 's/cat/dog/' manipulate > tmp && mv tmp manipulate
I.e. change ; to && so the move only happens if sed is successful; otherwise you'll lose your original file as soon as you make a typo in your sed syntax.
Note! For those reading the title and missing the OP's constraint "my sed doesn't support -i": For most people, sed will support -i, so the best way to do this is:
$ sed -i 's/cat/dog/' manipulate
Yes, -i is also supported in FreeBSD/MacOSX sed, but needs the empty string as an argument to edit a file in-place.
sed -i "" 's/old/new/g' file # FreeBSD sed
If you don't want to move copies around, you could use ed:
ed file.txt <<EOF
%s/cat/dog/
wq
EOF
Kernighan and Pike in The Art of Unix Programming discuss this issue. Their solution is to write a script called overwrite, which allows one to do such things.
The usage is: overwrite file cmd file.
# overwrite: copy standard input to output after EOF
opath=$PATH
PATH=/bin:/usr/bin
case $# in
0|1) echo 'Usage: overwrite file cmd [args]' 1>&2; exit 2
esac
file=$1; shift
new=/tmp/overwr1.$$; old=/tmp/overwr2.$$
trap 'rm -f $new $old; exit 1' 1 2 15 # clean up
if PATH=$opath "$#" >$new
then
cp $file $old # save original
trap '' 1 2 15 # wr are commmitted
cp $new $file
else
echo "overwrite: $1 failed, $file unchanged" 1>&2
exit 1
fi
rm -f $new $old
Once you have the above script in your $PATH, you can do:
overwrite manipulate sed 's/cat/dog/' manipulate
To make your life easier, you can use replace script from the same book:
# replace: replace str1 in files with str2 in place
PATH=/bin:/usr/bin
case $# in
0|2) echo 'Usage: replace str1 str2 files' 1>&2; exit 1
esac
left="$1"; right="$2"; shift; shift
for i
do
overwrite $i sed "s#$left#$right#g" $i
done
Having replace in your $PATH too will allow you to say:
replace cat dog manipulate
You can use sponge from the moreutils.
sed "s/cat/dog/" manipulate | sponge manipulate
Perhaps -i is gnu sed, or just an old version of sed, but anyways. You're on the right track. The first option is probably the most common one, the third option is if you want it to work everywhere (including solaris machines)... :) These are the 'standard' ways of doing it.
To change multiple files (and saving a backup of each as *.bak):
perl -p -i -e "s/oldtext/newtext/g" *
replaces any occurence of oldtext by newtext in all files in the current folder. However you will have to escape all perl special characters within oldtext and newtext using the backslash
This is called a “Perl pie” (mnemonic: easy as a pie)
The -i flag tells it do do in-place replacement, and it should be ok to use single (“'”) as well as double (“””) quotes.
If using ./* instead of just *, you should be able to do it in all sub-directories
See man perlrun for more details, including how to take a backup file of the original.
using sed:
sed -i 's/old/new/g' ./* (used in GNU)
sed -i '' 's/old/new/g' ./* (used in FreeBSD)
-i option is not available in standard sed.
Your alternatives are your third way or perl.
A lot of answers, but none of them is correct. Here is the correct and simplest one:
$ echo "111 222 333" > file.txt
$ sed -i -s s/222/444/ file.txt
$ cat file.txt
111 444 333
$
Workaround using open file handles:
exec 3<manipulate
Prevent open file from being truncated:
rm manipulate
sed 's/cat/dog/' <&3 > manipulate
how to replace a set of lines in a file with another set of lines in unix ?
#!/usr/bin/ksh
export text1='log_file=$LOG_PATH${UNISON_JOB}".log"'
export text2='\. \$\{env_path\}set_test_log_file\.ksh'
export text3='log_file\=\$LOG_PATH\$\{UNISON_JOB\}\"\.log\"'
echo $text1
echo $text2
echo $text3
for file in `grep -il ${text1} SANDEEP`
do
sed 's/${text3}/${text2}/g' $file > /$file.tmp
mv /$file.tmp $file
echo $file
done
i tried the above code but its not working .
its kshell .here sandeep is the file name i m searching.
The reason why this doesn't work is because you use single qoutes for the sed command.
The sed command is trying to replace the literal text ${text3} with ${text2}. What you want it to to is to replace the value of $text3 with the value of $text2.
In order to use variables in the sed expression you need to use double quotes.
Another tip: if you do not need the temporary file you might as well use the -i option of sed to edit the file in place.
Hope this helps.
For sed with the quote, I like to write whole command into a file, then execute this file:
something like this:
echo "s/${text1}/${text2}/g" > a.sed
sed -f a.sed < inputFile > tmp && mv tmp inputFile
rm -f a.sed
that will save a lot of trouble to deal with those quote thing.