I am attempting to write a bash script that will use sed to replace an entire line in a text file beginning with a given string, and I only want it to perform this replacement for the first match.
For example, in my text file I may have:
hair=brown
age=25
eyes=blue
age=35
weight=177
And I may want to simply replace the first occurrence of a line beginning with "age" with a different number without affecting the 2nd instance of age:
hair=brown
age=55
eyes=blue
age=35
weight=177
So far, I've come up with
sed -i "0,/^PATTERN/s/^PATTERN/PATTERN=XY/" test.txt
but this will only replace the string "age" itself rather than the entire line. I've been trying to throw a "\c" in there somewhere to change the entire line but nothing is working so far. Does anyone have any ideas as to how this can be resolved? Thanks.
Like #ruakh suggests, you can use
sed -i "0,/^PATTERN/ s/^PATTERN=.*$/PATTERN=XY/" test.txt
A shorter and less repetitive way of doing the same would be
sed -i '0,/^\(PATTERN=\).*/s//\1XY/' test.txt
which takes advantage of backreferences and the fact that not specifying a pattern in an s-expression will use the previously matched pattern.
0,...-ranges only work in GNU sed. An alternative might be to use shell redirect with sed:
{ sed '/^\(PATTERN\).*/!n; s//\1VAL;q'; cat ;} < file
or use awk:
awk '$1=="LABEL" && !n++ {$2="VALUE"}1' FS=\\= OFS=\\= file
Related
I am using sed to find and replace words in a file that have matching words with
sed -i "s|word|words|g" file
I would like it to change the first, skip the second and change the third match
file
var1=word
var2=/tmp/word
var3=word.txt
file
var1=words
var2=/tmp/word
var3=words.txt
Is it possible to keep var2 as is?
sed -i "s|word|words|;s||words|2" file
Change the 1st than the second (that is the 3th originaly due to first replacement)
but this is by line (default for sed)
so it is (for a file)
sed -i 'H;${x;s/word/&s/;s//&s/2;s/.//p;}' file
that first load the whole file in buffer before making the change
Try with awk:
awk <file '/word/{num=num+1}{if (num!=2) gsub("word","words"); print}'
sed 'H;$!d;x;:a;s/word/MATCH/1;s/word/IGNORE/1;s/word/MATCH/1;t a;s/MATCH/words/g;s/IGNORE/word/g' file
Okay, this may seem complicated at first but its simple - this copies all the text into the Hold space, after that it uses a 3 pattern first occurence match to modify the looked for text (word) with a substitute constant (to avoid reprocessing) then loops (t a) back to the label (:a) until the whole file is processed afterward it replaces the substitute constants.
You can use whatever substitute constants (MATCH,IGNORE) you want if those are problem for your data.
I need to insert the text at each line only if the given pattern matches with that line.
For example,
sed -n '/pattern/p' /etc/inittab/
so, if the pattern matches with any of the lines in inittab file, then i need to insert '#' at the beginning of those lines in the same file itself.
Kindly suggest me, how to make this.
Using sed:
sed '/pattern/s/^/#/' file
This will look for lines matching the pattern and once it finds it, it will place # in front of it. This will not modify the file. In order to do so, you need to use -i option to make in-place changes. You can put an extension like -i.bak to make an optional back if you'd like.
Using awk:
awk '/pattern/{$0="#"$0}1' file
awk is made up by pattern action statements. For the matching pattern, the action we do is modify the line by placing # in front of it. The 1 at the end will print the lines for us. GNU awk v4.1 or later has in-place editing just like sed. If you are using an older version you can redirect the output to another file and mv it back to original by saying:
awk '/pattern/{$0="#"$0}1' file > tmp && mv tmp file
The in-place changes is nothing special. It does the same job as redirecting to a temp file and then moving it back. It just does all the dirty work for you behind the scenes.
This is achieved with the following sed invocation
% sed -i.orig -e '/pattern/s/^/#/' inittab
The -i.orig option tells sed to operate in place on the file, previously saving the original as inittab.orig. The editing pattern
/pattern/ selects lines matching pattern
s/^/#/ and substitute the empty word at the beginning of line with #
I know sed could be used to delete specific line from file:
sed -i "/pattern/d" file
While the pattern of my case includes slash, like /var/log,
So I know I need escape: sed -i "/\/tmp\/dir/d" file
However, for my case, the pattern is dynamic, should be a variable
in a shell file, so I have to convert the variable value to replace
"/" with "\\/", then got this:
sed -i "/^${pattern_variable//\\//\\\\\\/}$/d" file
My question is, is there any better implementation which is more readable or simpler? Not only sed, other utility is also acceptable. Is it possible to handle not only slash but also other various symbols, like backslash or # ()?
you can use char other than /:
sed "\#$varHasSlash#d"
example:
kent$ foo="b/c"
kent$ echo "a
ab/cd
e"|sed "\#$foo#d"
a
e
For some reason I can't seem to find a straightforward answer to this and I'm on a bit of a time crunch at the moment. How would I go about inserting a choice line of text after the first line matching a specific string using the sed command. I have ...
CLIENTSCRIPT="foo"
CLIENTFILE="bar"
And I want insert a line after the CLIENTSCRIPT= line resulting in ...
CLIENTSCRIPT="foo"
CLIENTSCRIPT2="hello"
CLIENTFILE="bar"
Try doing this using GNU sed:
sed '/CLIENTSCRIPT="foo"/a CLIENTSCRIPT2="hello"' file
if you want to substitute in-place, use
sed -i '/CLIENTSCRIPT="foo"/a CLIENTSCRIPT2="hello"' file
Output
CLIENTSCRIPT="foo"
CLIENTSCRIPT2="hello"
CLIENTFILE="bar"
Doc
see sed doc and search \a (append)
Note the standard sed syntax (as in POSIX, so supported by all conforming sed implementations around (GNU, OS/X, BSD, Solaris...)):
sed '/CLIENTSCRIPT=/a\
CLIENTSCRIPT2="hello"' file
Or on one line:
sed -e '/CLIENTSCRIPT=/a\' -e 'CLIENTSCRIPT2="hello"' file
(-expressions (and the contents of -files) are joined with newlines to make up the sed script sed interprets).
The -i option for in-place editing is also a GNU extension, some other implementations (like FreeBSD's) support -i '' for that.
Alternatively, for portability, you can use perl instead:
perl -pi -e '$_ .= qq(CLIENTSCRIPT2="hello"\n) if /CLIENTSCRIPT=/' file
Or you could use ed or ex:
printf '%s\n' /CLIENTSCRIPT=/a 'CLIENTSCRIPT2="hello"' . w q | ex -s file
Sed command that works on MacOS (at least, OS 10) and Unix alike (ie. doesn't require gnu sed like Gilles' (currently accepted) one does):
sed -e '/CLIENTSCRIPT="foo"/a\'$'\n''CLIENTSCRIPT2="hello"' file
This works in bash and maybe other shells too that know the $'\n' evaluation quote style. Everything can be on one line and work in
older/POSIX sed commands. If there might be multiple lines matching the CLIENTSCRIPT="foo" (or your equivalent) and you wish to only add the extra line the first time, you can rework it as follows:
sed -e '/^ *CLIENTSCRIPT="foo"/b ins' -e b -e ':ins' -e 'a\'$'\n''CLIENTSCRIPT2="hello"' -e ': done' -e 'n;b done' file
(this creates a loop after the line insertion code that just cycles through the rest of the file, never getting back to the first sed command again).
You might notice I added a '^ *' to the matching pattern in case that line shows up in a comment, say, or is indented. Its not 100% perfect but covers some other situations likely to be common. Adjust as required...
These two solutions also get round the problem (for the generic solution to adding a line) that if your new inserted line contains unescaped backslashes or ampersands they will be interpreted by sed and likely not come out the same, just like the \n is - eg. \0 would be the first line matched. Especially handy if you're adding a line that comes from a variable where you'd otherwise have to escape everything first using ${var//} before, or another sed statement etc.
This solution is a little less messy in scripts (that quoting and \n is not easy to read though), when you don't want to put the replacement text for the a command at the start of a line if say, in a function with indented lines. I've taken advantage that $'\n' is evaluated to a newline by the shell, its not in regular '\n' single-quoted values.
Its getting long enough though that I think perl/even awk might win due to being more readable.
A POSIX compliant one using the s command:
sed '/CLIENTSCRIPT="foo"/s/.*/&\
CLIENTSCRIPT2="hello"/' file
Maybe a bit late to post an answer for this, but I found some of the above solutions a bit cumbersome.
I tried simple string replacement in sed and it worked:
sed 's/CLIENTSCRIPT="foo"/&\nCLIENTSCRIPT2="hello"/' file
& sign reflects the matched string, and then you add \n and the new line.
As mentioned, if you want to do it in-place:
sed -i 's/CLIENTSCRIPT="foo"/&\nCLIENTSCRIPT2="hello"/' file
Another thing. You can match using an expression:
sed -i 's/CLIENTSCRIPT=.*/&\nCLIENTSCRIPT2="hello"/' file
Hope this helps someone
The awk variant :
awk '1;/CLIENTSCRIPT=/{print "CLIENTSCRIPT2=\"hello\""}' file
I had a similar task, and was not able to get the above perl solution to work.
Here is my solution:
perl -i -pe "BEGIN{undef $/;} s/^\[mysqld\]$/[mysqld]\n\ncollation-server = utf8_unicode_ci\n/sgm" /etc/mysql/my.cnf
Explanation:
Uses a regular expression to search for a line in my /etc/mysql/my.cnf file that contained only [mysqld] and replaced it with
[mysqld]
collation-server = utf8_unicode_ci
effectively adding the collation-server = utf8_unicode_ci line after the line containing [mysqld].
I had to do this recently as well for both Mac and Linux OS's and after browsing through many posts and trying many things out, in my particular opinion I never got to where I wanted to which is: a simple enough to understand solution using well known and standard commands with simple patterns, one liner, portable, expandable to add in more constraints. Then I tried to looked at it with a different perspective, that's when I realized i could do without the "one liner" option if a "2-liner" met the rest of my criteria. At the end I came up with this solution I like that works in both Ubuntu and Mac which i wanted to share with everyone:
insertLine=$(( $(grep -n "foo" sample.txt | cut -f1 -d: | head -1) + 1 ))
sed -i -e "$insertLine"' i\'$'\n''bar'$'\n' sample.txt
In first command, grep looks for line numbers containing "foo", cut/head selects 1st occurrence, and the arithmetic op increments that first occurrence line number by 1 since I want to insert after the occurrence.
In second command, it's an in-place file edit, "i" for inserting: an ansi-c quoting new line, "bar", then another new line. The result is adding a new line containing "bar" after the "foo" line. Each of these 2 commands can be expanded to more complex operations and matching.
I have a flat file that contains something like this:
11|30646|654387|020751520
11|23861|876521|018277154
11|30645|765418|016658304
Using shell script, I would like to append a string to certain lines in this file, if those lines contain a specific string.
For example, in the above file, for lines containing 23861, I would like to append a string "Processed" at the end, so that the file becomes:
11|30646|654387|020751520
11|23861|876521|018277154|Processed
11|30645|765418|016658304
I could use sed to append the string to all lines in the file, but how do I do it for specific lines ?
I'd do it this way
sed '/\|23861\|/{s/$/|Something/;}' file
This is similar to Marcelo's answer but doesn't require extended expressions and is, I think, a little cleaner.
First, match lines having 23861 between pipes
/\|23861\|/
Then, on those lines, replace the end-of-line with the string |Something
{s/$/|Something/;}
If you want to do more than one of these you could simply list them
sed '/\|23861\|/{s/$/|Something/;};/\|30645\|/{s/$/|SomethingElse/;}' file
Use the following awk-script:
$ awk '/23861/ { $0=$0 "|Processed" } {print}' input
11|30646|654387|020751520
11|23861|876521|018277154|Processed
11|30645|765418|016658304
or, using sed:
$ sed 's/\(.*23861.*$\)/\1|Processed/' input
11|30646|654387|020751520
11|23861|876521|018277154|Processed
11|30645|765418|016658304
Use the substitution command:
sed -i~ -E 's/(\|23861\|.*)/\1|Processed/' flat.file
(Note: the -i~ performs the substitution in-place. Just leave it out if you don't want to modify the original file.)
You can use the shell
while read -r line
do
case "$line" in
*23681*) line="$line|Processed";;
esac
echo "$line"
done < file > tempo && mv tempo file
sed is just a stream version of ed, which has a similar command set but was designed to edit files in place (allegedly interactively, but you wouldn't want to use it that way unless all you had was one of these). Something like
field_2_value=23861
appended_text='|processed'
line_match_regex="^[^|]*|$field_2_value|"
ed "$file" <<EOF
g/$line_match_regex/s/$/$appended_text/
wq
EOF
should get you there.
Note that the $ in .../s/$/... is not expanded by the shell, as are $line_match_regex and $appended_text, because there's no such thing as $/ - instead it's passed through as-is to ed, which interprets it as text to substitute ($ being regex-speak for "end of line").
The syntax to do the same job in sed, should you ever want to do this to a stream rather than a file in place, is very similar except that you don't need the leading g before the regex address:
sed -e "/$line_match_regex/s/$/$appended_text/" "$input_file" >"$output_file"
You need to be sure that the values you put in field_2_value and appended_text never contain slashes, because ed's g and s commands use those for delimiters.
If they might do, and you're using bash or some other shell that allows ${name//search/replace} parameter expansion syntax, you could fix them up on the fly by substituting \/ for every / during expansion of those variables. Because bash also uses / as a substitution delimiter and also uses \ as a character escape, this ends up looking horrible:
appended_text='|n/a'
ed "$file" <<EOF
g/${line_match_regex//\//\\/}/s/$/${appended_text//\//\\/}/
wq
EOF
but it does work. Nnote that both ed and sed require a trailing / after the replacement text in s/search/replace/ while bash's ${name//search/replace} syntax doesn't.