I am looking for a way to insert a line to a specific position in a file.
Kinda like this: using sed
But the problem is, i want to write the output into the same file where i get the input and i need to do it with more than one file.
sadly this does not work: sed '3iline 3' input.txt > input.txt
this would work: sed '3iline 3' input.txt > tmp.txt && cat tmp.txt > input.txt
but i doesn't work with find and exec anymore...
i hoped something like this would be possible:
find /usr/local/share/ -iname 'xyz.htm' -exec sed '19i<p>TEXT</p>' {} > {} \;
but it doesn't work like this, so i ended up writing a short script which worked. But it still bothers me, because i keep thinking it should be possible to do it short (maybe onliner) and easy.
I hope someone can point me in the right direction.
If I understood correctly, You need -i option.
Which will do the changes in original file. It is not require any other piping and redirection.
sed -i.bak '3iline 3' input.txt
This will take the backup of original file with .bak extension.
From man sed :
-i[SUFFIX], --in-place[=SUFFIX]
edit files in place (makes backup if extension supplied)
Related
I am on Mac Os 10.14.6 and have a directory that contains subdirectories that all contain text files. Altogether, there are many hundreds of text files.
I would like to go through the text files and check for any content in the first line that is in parentheses. If such content is found, then the parentheses (and content in the parentheses) should be removed.
Example:
Before removal:
The new world (82 edition)
After removal:
The new world
How would I do this?
Steps I have tried:
Google around, it seems SED would be best for this.
I have found this thread, which provides SED code for removing bracketed content.
sed -e 's/([^()]*)//g'
However, I am not sure how to adapt it to work on multiple files and also to limit it to the first line of those files. I found this thread which explains how to use SED on multiple files, but I am not sure how to adapt the example to work with parentheses content.
Please note: As long as the solution works on Mac OS terminal, then it does not need to use SED. However, from Googling, SED seems to be the most suited.
I managed to achieve what you're after simply by using a bash script and sed together, as so:
#!/bin/bash
for filename in $PWD/*.txt; do
sed -i '' '1 s/([^()]*)//g' $filename
done
The script simply iterates over all the .txt files in $PWD (the current working directory, so that you can add this script to your bin and run it anywhere), and then runs the command
sed -ie '1 s/([^()]*)//g' $filename
on the file. By starting the command with the number 1 we tell sed to only work on the first line of the file :)
Edit: Best Answer
The above works fine in a directory where all contained objects are files, and not including directories; in other words, the above does not perform recursive search through directories.
Therefore, after some research, this command should perform exactly what the question asks:
find . -name "*.txt" -exec sed -i '' '1 s/([^()]*)//g' {} \;
I must iterate, and reiterate, that you test this on a backup first to test it works. Otherwise, use the same command as above but change the '' in order to control the creation of backups. For example,
find . -name "*.txt" -exec sed -i '.bkp' '1 s/([^()]*)//g' {} \;
This command will perform the sed replace in the original file (keeping the filename) but will create a backup file for each with the appended .bkp, for example test1.txt becomes test1.txt.bkp. This a safer option, but choose what works best for you :)
Good try,
The command you where looking for single line:
sed -E '1s|\([^\)]+\)||'
The command to replace each input file first line:
sed -Ei '1s|\([^\)]+\)||' *.txt
example:
echo "The new world (82 edition)" |sed -E '1s|\([^\)]+\)||'
The new world
Explanation
sed -Ei E option: the extended RegExp syntax, i option: for in-place file replacement
sed -Ei '1s|match RegExp||' for first line only, replace first matched RegExp string with empty string
\([^\)]+\) RegExp matching: start with (, [^\)]any char not ), + - more than once, terminate with )
Try:
# create a temporary file
tmp=$(mktemp)
# for each something in _the current directory_
for i in *; do
# if it is not a file, don't parse it
if [ ! -f "$i" ]; then continue; fi
# remove parenthesis on first line, save the output in temporary file
sed '1s/([^)]*)//g' "$i" > "$tmp"
# move temporary file to the original file
mv "$tmp" "$i"
done
# remove temporary file
rm "$tmp"
I am on macOS High Sierra, and I've had a few issues with sed,
after spending a day on Google and this site, I've honestly tried everything I could think of and was suggested.
I have an example.txt file with 3 lines.
line1
line2
line3
The 3 lines can be anything, I do not know them upfront. (scenario)
And at some point I do know what the line is going to be.
All I wish to achieve is to use 'whatever' onliner that basically says:
in that file, replace line 2, with the new data.
sed -i "2s/.*/$NEW_DATA/" "$FILENAME"
This should work, but on macOS, this does not.
sed -ie "2s/.*/$NEW_DATA/" "$FILENAME"
Note the e? Someone on here suggested to add an e, and yes, that works.. but it means it adds the e behind the .txt. I end up with 2 files, example.txt and example.txte
sed -i'' "2s/.*/$NEW_DATA/" "$FILENAME"
Note the '' behind -i, and yes, without a space? This should work too.
It doesn't, it will complain that command c requires \
I could go on, but the request is quite simple:
on macOS, in a bash shell script, how do I simply replace a specified line of text in a .txt file, with a completely new line of text -- not knowing what the line of text was before?
If this can be a simple macOS one liner with awk or whatever, that's fine. It doesn't have to be sed. But when I search this site, it seems to be the recommended one to go with in this regards.
From the sed man page in macOS:
-i extension
Edit files in-place, saving backups with the specified extension.
If a zero-length extension is given, no backup will be saved.
Therefore this can be used to replace line 2 without keeping backup:
sed -i '' "2s/.*/$NEW_DATA/" testfile.txt
If the goal is just to replace contents of line 2 awk could also be used, for example:
awk 'NR==2{$0="your content"}1' testfile.txt > testfile.tmp && mv testfile.tmp testfile.txt
I want to find specific text in a directory, and then delete the lines from the files that include the specific text.
Now I have two questions:
How can I achieve the task?
What is wrong with What I have tried? I have tried the methods below, but failed. the details are following:
grep -rnw "./" -e "webdesign"
This searches the current directory with pattern "webdesign", and I get the result:
.//pages/index.html:1:{% load webdesign %}
.//pages/pricing.html:1:{% load webdesign %}
.//prototypes.py:16: 'django.contrib.webdesign',
Then I use sed to remove the lines from those files, which doesn't work, only get blank file ( I mean it deletes all my file content):
sed -i "/webdesign/d" ./pages/index.html
or
sed "/webdesign/d" ./pages/index.html > ./pages/index.html
My software environment is: OS X Yosemite, Mac Terminal, Bash
A loop in bash will do the trick provided that there are no filenames with spaces (in which case other solutions are possible, but this is the simplest)
for i in `grep -lrnw "yourdirectory/" -e "webdesign"`
do
sed "/webdesign/d" $i > $i.tmp
# safety to avoid destroying the file if problem arises (disk full?)
if [ $? = 0 ] ; then
mv -f $i.tmp $i
fi
done
note that you should not locate this script in the current directory because it contains webdesign and it will be modified as well :)
Thanks to choroba, I know that -i option doesn't work like wished. But it has another meaning or it would be rejected by the opt parser. It has something to do with suffixes, well, it doesn't matter now, but it's difficult to see the problem at first.
Without -i you cannot work on a file in-place. And redirecting output to the input just destroys the input file (!). That's why your solution did not work.
You can install GNU sed that supports the -i option, then
sed -i '/webdesign/d' files
should work. Note that it's safer to use -i~ to create a backup.
You cannot write to the same file you're reading from, that's why
sed /webdesign/d file > file
doesn't work (it overwrites the file before you can read anything from it). Create a temporary file
sed /webdesign/d file > file.tmp
mv file.tmp file
I'm attempting to remove a line from several hundred files. The following does exactly what I need but, it doesn't save changes (as expected).
$ grep -v meow src/files
I've seen that appending > to the end of a given command will specify where the output buffer should save but, does this work for multiple files?
So I'd like to know if there's an elegant way to mass edit via the terminal. All of the examples I've come across using awk or sed only provide solutions for editing one file at a time.
One way to do this is using the following Perl one-liner:
perl -i.bak -n -e 'print unless /meow/' src/files
This should do in-place editing of multiple files. The originals are saved in .bak files.
Another way to do it is to do a similar operation with sed:
sed -i .bak '/meow/d' src/files/*
Perl got its -i option from sed, after all. Note that to use no backup file, you need an explicit empty extension with at least some versions of sed:
sed -i '' '/meow/d' src/files/*
I have a huge csv file (on order of terabytes).
Now, I want to insert one row which is a header to the the top.
For example if input.csv looks like this:
1,2,3,4
22,3,23,1
I want it to look like
id1,id2,id3,id4
1,2,3,4
and so on
How do i do this from shell, terminal, awk, bash?/
In place, using sed:
sed -i 1i"id1,id2,id3,id4" file.csv
edit:
As #Ed Morton points out, using sed with the -i switch sed edits the file in place, and can therefore be dangerous when editing large files. If you supply a prefix after the -i option then sed creates a backup. So something like this would be safer:
sed -i.bak 1i"id1,id2,id3,id4" file.csv
The original file will then be located in file.csv.bak
This is that simple as :
{ echo "id1,id2,id3,id4"; cat file.csv; } > newfile.csv
using simple shell concatenation.
EDIT
after discussion thread below, I propose this :
create a file with your header, said head.txt
Then :
cat head.txt file.csv > newfile.csv
Edit. When I wrote this answer, I overlooked the "terabyte" part of the question. Hence, do not use the method presented here. I still leave this post, as it advertises the use of this wonderful tool, ed, the standard text editor.
As usual, ed is the standard text editor. The solution using sed -i doesn't, as it mentions, "edit the file in place". Instead, it outputs its content to a temporary file, and then renames this file to the original one. That's really not good for large files!
Using ed instead really edits the file. Something along the following lines:
#!/bin/bash
file="input.csv"
{
ed -s "$file" <<EOF
1
i
id1,id2,id3,id4
.
wq
EOF
} > /dev/null
Explanation: 1 goes to the first line, i goes into insert mode, then we insert id1,id2,id3,id4 then . to go back to normal mode, and wq to write and quit.
With this method, you're really editing the file and it's twice faster than the sed method. Also, ed is known to be "large file safe"!
Done.
There's no easy way, you're going to have to rewrite the file. Probably the safest way is to
( echo "id1,id2,id3,id4" ; cat file ) > newFile && rm file
IHTH
echo "id1,id2,id3,id4" >> data.csv