I need to replace the whole line with sed if it matches a pattern.
For example if the line is 'one two six three four' and if 'six' is there, then the whole line should be replaced with 'fault'.
You can do it with either of these:
sed 's/.*six.*/fault/' file # check all lines
sed '/six/s/.*/fault/' file # matched lines -> then remove
It gets the full line containing six and replaces it with fault.
Example:
$ cat file
six
asdf
one two six
one isix
boo
$ sed 's/.*six.*/fault/' file
fault
asdf
fault
fault
boo
It is based on this solution to Replace whole line containing a string using Sed
More generally, you can use an expression sed '/match/s/.*/replacement/' file. This will perform the sed 's/match/replacement/' expression in those lines containing match. In your case this would be:
sed '/six/s/.*/fault/' file
What if we have 'one two six eight eleven three four' and we want to
include 'eight' and 'eleven' as our "bad" words?
In this case we can use the -e for multiple conditions:
sed -e 's/.*six.*/fault/' -e 's/.*eight.*/fault/' file
and so on.
Or also:
sed '/eight/s/.*/XXXXX/; /eleven/s/.*/XXXX/' file
Above answers worked fine for me, just mentioning an alternate way
Match single pattern and replace with a new one:
sed -i '/six/c fault' file
Match multiple pattern and replace with a new one(concatenating commands):
sed -i -e '/one/c fault' -e '/six/c fault' file
To replace whole line containing a specified string with the content of that line
Text file:
Row: 0 last_time_contacted=0, display_name=Mozart, _id=100, phonebook_bucket_alt=2
Row: 1 last_time_contacted=0, display_name=Bach, _id=101, phonebook_bucket_alt=2
Single string:
$ sed 's/.* display_name=\([[:alpha:]]\+\).*/\1/'
output:
100
101
Multiple strings delimited by white-space:
$ sed 's/.* display_name=\([[:alpha:]]\+\).* _id=\([[:digit:]]\+\).*/\1 \2/'
output:
Mozart 100
Bach 101
Adjust regex to meet your needs
[:alpha] and [:digit:]
are Character Classes and Bracket Expressions
This might work for you (GNU sed):
sed -e '/six/{c\fault' -e ';d}' file
or:
sed '/six/{c\fault'$'\n'';d}' file
Related
how to remove comment lines (as # bal bla ) and empty lines (lines without charecters) from file with one sed command?
THX
lidia
If you're worried about starting two sed processes in a pipeline for performance reasons, you probably shouldn't be, it's still very efficient. But based on your comment that you want to do in-place editing, you can still do that with distinct commands (sed commands rather than invocations of sed itself).
You can either use multiple -e arguments or separate commands with a semicolon, something like (just one of these, not both):
sed -i 's/#.*$//' -e '/^$/d' fileName
sed -i 's/#.*$//;/^$/d' fileName
The following transcript shows this in action:
pax> printf 'Line # with a comment\n\n# Line with only a comment\n' >file
pax> cat file
Line # with a comment
# Line with only a comment
pax> cp file filex ; sed -i 's/#.*$//;/^$/d' filex ; cat filex
Line
pax> cp file filex ; sed -i -e 's/#.*$//' -e '/^$/d' filex ; cat filex
Line
Note how the file is modified in-place even with two -e options. You can see that both commands are executed on each line. The line with a comment first has the comment removed then all is removed because it's empty.
In addition, the original empty line is also removed.
#paxdiablo has a good answer but it can be improved.
(1) The '/^$/d' clause only matches 100% blank lines.
If you want to also match lines that are entirely whitespace (spaces, tabs etc.) use this instead:
'/^\s*$/d'
(2) The 's/#.*$//' clause only matches lines that start with the # character in column 0.
If you want to also match lines that have only whitespace before the first # use this instead:
'/^\s*#.*$/d'
The above criteria may not be universal (e.g. within a HEREDOC block, or in a Python multi-line string the different approaches could be significant), but in many cases the conventional definition of "blank" lines include whitespace-only, and "comment" lines include whitespace-then-#.
(3) Lastly, on OSX at least, the #paxdiablo solution in which the first clause turns comment lines into blank lines, and the second clause strips blank lines (including what were originally comments) doesn't work. It seems to be more portable to make both clauses /d delete actions as I've done.
The revised command incorporating the above is:
sed -e '/^\s*#.*$/d' -e '/^\s*$/d' inputFile
This tiny jewel removes all # comments, no matter where they begin in a line (see caution below):
sed -e 's/\s*#.*$//'
Example:
text="
this is a # test
#this is a test
#this is a #test
this is # another #test
"
$echo "$text" | sed -e 's/\s*#.*$//'
this is a
this is
Next this removes any resulting blank lines:
$echo "$text" | sed -e 's/\s*#.*$//' | sed -e '/^\s*$/d'
Caution: Depending on the syntax and/or interpretation of the lines your processing, this might not be an appropriate solution, as it just stupidly removes end of lines, even if the '#' is part of your data or code. However, for use cases where you'll never use a hash except for as an end of line comment then it works fine. So just as with all coding, context must be taken into consideration.
Alternative variant, using grep:
cat file.txt | grep -Ev '(#.*$)|(^$)'
you can use awk
awk 'NF{gsub(/^[ \t]*#/,"");print}' file
First example(paxdiablo) is very good except its not change file, just output result. If you want to change it inline:
sudo sed -i 's/#.*$//;/^$/d' inputFile
On (one of) my linux boxes, sed understands extended regular expressions with the -r option, so:
sed -r '/(^\s*#)|(^\s*$)/d' squid.conf.installed
is very useful for showing all non-blank, non comment lines.
The regex matches either start of line followed by zero or more spaces or tabs followed by either a hash or end of line, and deletes those matching lines from the input.
How can I use a cat and sed to read data from a file and insert it into another file under known line?
For example I have a file named script1.txt that contains a few hundred lines, one of the line has the value "COMMANDS="commands"
If I wanted use sed to insert a line under it, simply I can use sed as the command bellow.
sed -i '/^COMMANDS=.*/a NEW LINE HERE' script1.txt
But if I want to insert a multi lines and these lines inside a file, and these line changes every a few hours.. how can i do that ?
I tried:
DATA=$(cat data.txt)
sed -i '/^COMMANDS=.*/a '$DATA'' script1.txt
I got the error bellow.
sed: -e expression #1, char 1: unknown command: `"'
Is there a way other than sed to insert the data from file under known line with no issues?
This might work for you (GNU sed):
sed -i '/^COMMANDS=/r dataFile' file
This will append the contents of the file dataFile after the line beginning COMMANDS= and update file
If the data you want to append is multi-line, you might want to replace newlines with \n.
#!/bin/sh
DATA="$(awk '{gsub(/[]\/$*.^&[]/, "\\\\&");printf (FNR>1)?"\\n%s":"%s",$0}END{print ""}' data.txt)"
sed -i -e '/^COMMANDS=.*/a\' -e "$DATA" script1.txt
Here the awk command escapes sed special characters (for basic regular expressions), then prints "%s" for the first line, and "\\n%s" for the others. A newline is printed at the end, but it's somewhat pointless as $() strips it anyway.
The sed command is almost the same but multiple expressions are used which is equivalent to a multi-line sed script (The a text sed alternative syntax can act weirdly with leading spaces/backslashes).
I can't figure how to tell sed dot match new line:
echo -e "one\ntwo\nthree" | sed 's/one.*two/one/m'
I expect to get:
one
three
instead I get original:
one
two
three
sed is line-based tool. I don't think these is an option.
You can use h/H(hold), g/G(get).
$ echo -e 'one\ntwo\nthree' | sed -n '1h;1!H;${g;s/one.*two/one/p}'
one
three
Maybe you should try vim
:%s/one\_.*two/one/g
If you use a GNU sed, you may match any character, including line break chars, with a mere ., see :
.
Matches any character, including newline.
All you need to use is a -z option:
echo -e "one\ntwo\nthree" | sed -z 's/one.*two/one/'
# => one
# three
See the online sed demo.
However, one.*two might not be what you need since * is always greedy in POSIX regex patterns. So, one.*two will match the leftmost one, then any 0 or more chars as many as possible, and then the rightmost two. If you need to remove one, then any 0+ chars as few as possible, and then the leftmost two, you will have to use perl:
perl -i -0 -pe 's/one.*?two//sg' file # Non-Unicode version
perl -i -CSD -Mutf8 -0 -pe 's/one.*?two//sg' file # S&R in a UTF8 file
The -0 option enables the slurp mode so that the file could be read as a whole and not line-by-line, -i will enable inline file modification, s will make . match any char including line break chars, and .*? will match any 0 or more chars as few as possible due to a non-greedy *?. The -CSD -Mutf8 part make sure your input is decoded and output re-encoded back correctly.
You can use python this way:
$ echo -e "one\ntwo\nthree" | python -c 'import re, sys; s=sys.stdin.read(); s=re.sub("(?s)one.*two", "one", s); print s,'
one
three
$
This reads the entire python's standard input (sys.stdin.read()), then substitutes "one" for "one.*two" with dot matches all setting enabled (using (?s) at the start of the regular expression) and then prints the modified string (the trailing comma in print is used to prevent print from adding an extra newline).
This might work for you:
<<<$'one\ntwo\nthree' sed '/two/d'
or
<<<$'one\ntwo\nthree' sed '2d'
or
<<<$'one\ntwo\nthree' sed 'n;d'
or
<<<$'one\ntwo\nthree' sed 'N;N;s/two.//'
Sed does match all characters (including the \n) using a dot . but usually it has already stripped the \n off, as part of the cycle, so it no longer present in the pattern space to be matched.
Only certain commands (N,H and G) preserve newlines in the pattern/hold space.
N appends a newline to the pattern space and then appends the next line.
H does exactly the same except it acts on the hold space.
G appends a newline to the pattern space and then appends whatever is in the hold space too.
The hold space is empty until you place something in it so:
sed G file
will insert an empty line after each line.
sed 'G;G' file
will insert 2 empty lines etc etc.
How about two sed calls:
(get rid of the 'two' first, then get rid of the blank line)
$ echo -e 'one\ntwo\nthree' | sed 's/two//' | sed '/^$/d'
one
three
Actually, I prefer Perl for one-liners over Python:
$ echo -e 'one\ntwo\nthree' | perl -pe 's/two\n//'
one
three
Below discussion is based on Gnu sed.
sed operates on a line by line manner. So it's not possible to tell it dot match newline. However, there are some tricks that can implement this. You can use a loop structure (kind of) to put all the text in the pattern space, and then do the operation.
To put everything in the pattern space, use:
:a;N;$!ba;
To make "dot match newline" indirectly, you use:
(\n|.)
So the result is:
root#u1804:~# echo -e "one\ntwo\nthree" | sed -r ':a;N;$!ba;s/one(\n|.)*two/one/'
one
three
root#u1804:~#
Note that in this case, (\n|.) matches newline and all characters. See below example:
root#u1804:~# echo -e "oneXXXXXX\nXXXXXXtwo\nthree" | sed -r ':a;N;$!ba;s/one(\n|.)*two/one/'
one
three
root#u1804:~#
Trying to replace a line with a multi-line file. I can easily do this with a single-line file or multi-line string (see below).
#!/bin/bash
NEW_STRING="apple\nbanana\ncarrot"
sed -i "3s/.*/$(echo "${NEW_STRING}")/" tmp.txt
# Outputs...
# line 1
# line 2
# apple
# banana
# carrot
# line 4
# line 5
# ...
# etc
However, when I change the code to use a multi-file, such as replace.txt, I receive the following error:
sed: -e expression #1, char 10: unterminated `s' command
broken_script.bash
#!/bin/bash
FILE=`cat replace.txt`
sed -i "3s/.*/$(echo "${FILE}")/" tmp.txt
replace.txt
++++
++++++++++++++++++++++++++++++++++++++++++++++
apple size=8, align=2, ..., etc.
banana size=64, align=8, ..., etc.
...
carrot size=92, align=4, ..., etc.
Note broken_script.bash works if I delete replace.txt to be a single-line (i.e. just ++++).
Does anyone see what I'm doing wrong? Why doesn't this work with a multi-line file as the replacement text (i.e. like the single-line file or multi-line string)?
To replace lines with the contents of a file, you can use the r command:
sed -e 3rreplace.txt -e 3d tmp.txt
As asked in a comment, sed -e 3d -e 3rreplace.txt does not work because the d command immediately returns to the top of the program after reading the next line and never executes the r command.
First of all: Drop the echo part. You can use the variable directly.
Back to the actual problem:
The difference is how you encode the newline. In the first command you wrote \n do denote a newline. That \n is not interpreted by 'bash', but directly sent to sed. In the second command, the file content is sent to sed, including literal newline characters. For a file with the two lines line1 and line2 the command sed sees is
3s/.*/line1
line2/
sed cannot handle such multi-lined commands.
Non-sed solution:
As it seems, you just want to replace the third line of one file with the content of another file. This can be done with
cat <(head -n 2 file1) file2 <(tail -n +4 file1)
head -n2 file1 prints the first two lines of file1.
tail -n +4 file 1 prints all lines of file1 starting at line 4.
<(command) is called process substitution and emulates a file containing the output of command.
cat concatenates the three "files".
sed can do this with some quote juggling
$ seq 5 | sed -e '/3/{r replace.txt' -e 'd}'
1
2
++++
++++++++++++++++++++++++++++++++++++++++++++++
apple size=8, align=2, ..., etc.
banana size=64, align=8, ..., etc.
...
carrot size=92, align=4, ..., etc.
4
5
for in place replacement you need to provide file
$ sed -i -e '/3/{r replace.txt' -e 'd}' file
of course test first or take backup.
as title said, Im trying to change only the first occurrence of word.By using
sed 's/this/that/' file.txt
though i'm not using g option it replace entire file. How to fix this.?
UPDATE:
$ cat file.txt
first line
this
this
this
this
$ sed -e '1s/this/that/;t' file.txt
first line
this // ------> I want to change only this "this" to "that" :)
this
this
this
http://www.faqs.org/faqs/editor-faq/sed/
4.3. How do I change only the first occurrence of a pattern?
sed -e '1s/LHS/RHS/;t' -e '1,/LHS/s//RHS/'
Where LHS=this and RHS=that for your example.
If you know the pattern won't occur on the first line, omit the first -e and the statement following it.
sed by itself applies the edit thru out the file and combined with "g" flag the edit is applied to all the occurrences on the same line.
e.g.
$ cat file.txt
first line
this this
this
this
this
$ sed 's/this/that/' file.txt
first line
that this
that
that
that
$ sed 's/this/that/g' file.txt
first line
that that <-- Both occurrences of "this" have changed
that
that
that