I know sed could be used to delete specific line from file:
sed -i "/pattern/d" file
While the pattern of my case includes slash, like /var/log,
So I know I need escape: sed -i "/\/tmp\/dir/d" file
However, for my case, the pattern is dynamic, should be a variable
in a shell file, so I have to convert the variable value to replace
"/" with "\\/", then got this:
sed -i "/^${pattern_variable//\\//\\\\\\/}$/d" file
My question is, is there any better implementation which is more readable or simpler? Not only sed, other utility is also acceptable. Is it possible to handle not only slash but also other various symbols, like backslash or # ()?
you can use char other than /:
sed "\#$varHasSlash#d"
example:
kent$ foo="b/c"
kent$ echo "a
ab/cd
e"|sed "\#$foo#d"
a
e
Related
The normal expression to change server1 to server1-bck is
sed -i 's/server1/server1-bck/g' file.out
so all server1 will be changed to server1-bck. What I need is to change the second occurence of the expression in every line.
For example,
before text:
rename files tsm_node1 //server1/document/users/ //server1/document/users/
desired after text:
rename files tsm_node1 //server1/document/users/ //server1-bck/document/users/
How can I do that?
echo "rename files tsm_node1 //server1/document/users/ //server1/document/users/" |\
sed 's/server1/server1-bck/2g'
sed's famous substitution works like this:
sed 's/regex/replacement/flags'
Flags could be a number, in your case 2 for advising sed to execute this command on 2nd occurrence and if you need more, therefore the g flag is. If you are sure, there are no more items to be substituted, you can leave and forget the g flag.
If you don't pipe and have a file, do this:
sed -i 's/server1/server1-bck/2g' file.out.
Additionally you can replace parts of your regex pattern with sed's & replacement if you want to substitute with that what you have found and will have:
sed -i 's/server1/&-bck/2g' file.out.
How to replace a line / part of a line in a file using SED command?
search_text_1_server=value.env_1.path_to_file
search_text_2_server=value.env_1.path_to_file
search_text_3_server=value.env_1.path_to_file
some_other_key=value.env_1.another_path
Now I want a sed command to find the lines which match the regular expression search_text_{any}_server and then replace env_1 with env_2
Found the regular expression to find the required lines.
^search_text_[a-z_]\*_server.*$
Now how to add the SED syntax to replace
PS : I am not an expert in shell
Your regex is close. You can use:
sed -E 's/^(search_text_[a-z_]*_server=.*)env_1\./\1env_2\./' file
search_text_1_server=value.env_2.path_to_file
search_text_2_server=value.env_2.path_to_file
search_text_3_server=value.env_2.path_to_file
some_other_key=value.env_1.another_path
Assuming country code to be two alphabets, you could do
sed -Ei 's/(search_text_[a-z]{2}_server=value\.)env_1/\1env_2/' file
should do it.
What's happening here
[a-z]{2} checks for two alphabets which make a country code
sed s command is for substitution -> s/pattern/replacement
() selects the matched regex pattern for reuse, Note \1 for reuse
-i is the inplace edit option of sed which makes changes permanent in the file
I have a line in a autoexec.py file that I want another script to be able to modify:
kodi.executebuiltin("PlayMedia(/path/to/file)")
I was thinking of using sed to override the value within PlayMedia() depending on certain conditions.
Can sed be used to only touch and overwrite the parts between the PlayMedia() brackets and nothing else? Or is further processing/regex needed?
You could use sed. The below code will replace the chars present inside the brackets following PlayMedia string with foo.
sed 's/\b\(PlayMedia\)([^)]*)/\1(foo)/g' file
If the string you want to replace is a path, you must use a different sed delimiter because filepath may contain forward slashes.
sed 's~\b\(PlayMedia\)([^)]*)~\1(foo)~g' file
Example:
$ echo 'kodi.executebuiltin("PlayMedia(/path/to/file)")' | sed 's/\b\(PlayMedia\)([^)]*)/\1(foo)/g'
kodi.executebuiltin("PlayMedia(foo)")
I have file which contain following context like
abc...
include /home/user/file.txt'
some text
I need to remove include and also complete path after include.
I have used following command which remove include but did not remove path.
sed -i -r 's#include##g' 'filename'
I am also trying to understand above command but did not understand following thing ( copy paste from somewhere)
i - modify file change
r - read file
s- Need input
g - Need input
Try this,
$ sed '/^include /s/.*//g' file.txt
abc...
some text
It remove all the texts in a line which starts with include. s means substitute. so s/.*//g means replace all the texts with null.g means global. The substitution will be applied globally.
OR
$ sed '/^include /d' file.txt
abc...
some text
d means delete.
It deletes the line which starts with include. To save the changes made(inline edit), your commands should be
sed -i '/^include /s/.*//g' file.txt
sed -i '/^include /d' file.txt
I your case if you just want to delete the second line, you can use:
sed -i '2d' file
If you want to explore something about linux commands then man pages are there for you.
Just go to terminal and type:
man sed
as per your question, The above command without -i will show the file content on terminal by deleting the second line from the input file. However, the input file remains unchanged. To update the original file or to make the changes permanently in the source file, use the -i option.
-i[SUFFIX], --in-place[=SUFFIX] :
edit files in place (makes backup if extension supplied)
-r or --regexp-extended :
option is to use extended regular expressions in the script.
s/regexp/replacement/ :
Attempt to match regexp against the pattern space. If successâ
ful, replace that portion matched with replacement. The
replacement may contain the special character & to refer to that
portion of the pattern space which matched, and the special
escapes \1 through \9 to refer to the corresponding matching
sub-expressions in the regexp.
g G : Copy/append hold space to pattern space.
grep -v
This is not about learning sed, but as an alternative (and short) solution, there is:
grep -v '^include' filename_in
Or with output redirection:
grep -v '^include' filename_in > filename_out
-v option for grep inverts matching (hence printing non-matching lines).
For simple deletion that's what I'd use; if you have to modify your path after the include, stick with sed instead.
You can use awk to just delete the line:
awk '/^include/ {next}1' file
sed -i -r 's#include##g' 'filename'
-i: you directly modify the treated file, by default, sed read a file, modify the content via stdout (the original file stay the same).
-r: use of extended regular expression (and not reduce to POSIX limited one).This is not necessary in this case due to simple POSIX compliant action in action list (the s### string).
s#pattern#NewValue#: substitute in current line the pattern (Regular Expression) with "Newvalue" (that also use internal buffer or specific value). The traditionnal form is s/// but in this case, using / in path (pattern or new value) an alternate form is used to avoid to escape all / in pattern or new value
g: is an option of s### that specify change EVERY occurence and not the first (by default)
so here it replace ANY occurence of include by nothing (remove) directly into your file
As per the Avinash Raj solution you got what you want but you want some explaination about some parameter used in sed command
First one is
command: s for substitution
With the sed command the substitute command s changes all occurrences of the regular expression into a new value. A simple example is changing "my" in the "file1" to "yours" in the "file2" file:
sed s/my/yours/ file1 >file2
The character after the s is the delimiter. It is conventionally a slash, because this is what ed, more, and vi use. It can be anything you want, however. If you want to change a pathname that contains a slash - say /usr/local/bin to /common/bin - you could use the backslash to quote the slash:
sed 's/\/usr\/local\/bin/\/common\/bin/' <old >new
/g - Global replacement
Replace all matches, not just the first match.
If you tell it to change a word, it will only change the first occurrence of the word on a line. You may want to make the change on every word on the line instead of the first then add a g after the last delimiter and use the work-around:
Delete with d
Delete the pattern space; immediately start next cycle.
You can delete line by specifying the line number. like
sed '$d' filename.txt
It will remove last line of file
sed '2 d' file.txt
It will delete second line of file.
-i option
This option specifies that files are to be edited in-place. GNU sed does this by creating a temporary file and sending output to this file rather than to the standard output.
To modify file actully you can use -i option without it sed command repressent changes on stdout not actual file. You can take backup of original file before modification by using -i.bak option.
-r option
--regexp-extended
Use extended regular expressions rather than basic regular expressions. Extended regexps are those that egrep accepts; they can be clearer because they usually have less backslashes, but are a GNU extension and hence scripts that use them are not portable.
I have a flat file that contains something like this:
11|30646|654387|020751520
11|23861|876521|018277154
11|30645|765418|016658304
Using shell script, I would like to append a string to certain lines in this file, if those lines contain a specific string.
For example, in the above file, for lines containing 23861, I would like to append a string "Processed" at the end, so that the file becomes:
11|30646|654387|020751520
11|23861|876521|018277154|Processed
11|30645|765418|016658304
I could use sed to append the string to all lines in the file, but how do I do it for specific lines ?
I'd do it this way
sed '/\|23861\|/{s/$/|Something/;}' file
This is similar to Marcelo's answer but doesn't require extended expressions and is, I think, a little cleaner.
First, match lines having 23861 between pipes
/\|23861\|/
Then, on those lines, replace the end-of-line with the string |Something
{s/$/|Something/;}
If you want to do more than one of these you could simply list them
sed '/\|23861\|/{s/$/|Something/;};/\|30645\|/{s/$/|SomethingElse/;}' file
Use the following awk-script:
$ awk '/23861/ { $0=$0 "|Processed" } {print}' input
11|30646|654387|020751520
11|23861|876521|018277154|Processed
11|30645|765418|016658304
or, using sed:
$ sed 's/\(.*23861.*$\)/\1|Processed/' input
11|30646|654387|020751520
11|23861|876521|018277154|Processed
11|30645|765418|016658304
Use the substitution command:
sed -i~ -E 's/(\|23861\|.*)/\1|Processed/' flat.file
(Note: the -i~ performs the substitution in-place. Just leave it out if you don't want to modify the original file.)
You can use the shell
while read -r line
do
case "$line" in
*23681*) line="$line|Processed";;
esac
echo "$line"
done < file > tempo && mv tempo file
sed is just a stream version of ed, which has a similar command set but was designed to edit files in place (allegedly interactively, but you wouldn't want to use it that way unless all you had was one of these). Something like
field_2_value=23861
appended_text='|processed'
line_match_regex="^[^|]*|$field_2_value|"
ed "$file" <<EOF
g/$line_match_regex/s/$/$appended_text/
wq
EOF
should get you there.
Note that the $ in .../s/$/... is not expanded by the shell, as are $line_match_regex and $appended_text, because there's no such thing as $/ - instead it's passed through as-is to ed, which interprets it as text to substitute ($ being regex-speak for "end of line").
The syntax to do the same job in sed, should you ever want to do this to a stream rather than a file in place, is very similar except that you don't need the leading g before the regex address:
sed -e "/$line_match_regex/s/$/$appended_text/" "$input_file" >"$output_file"
You need to be sure that the values you put in field_2_value and appended_text never contain slashes, because ed's g and s commands use those for delimiters.
If they might do, and you're using bash or some other shell that allows ${name//search/replace} parameter expansion syntax, you could fix them up on the fly by substituting \/ for every / during expansion of those variables. Because bash also uses / as a substitution delimiter and also uses \ as a character escape, this ends up looking horrible:
appended_text='|n/a'
ed "$file" <<EOF
g/${line_match_regex//\//\\/}/s/$/${appended_text//\//\\/}/
wq
EOF
but it does work. Nnote that both ed and sed require a trailing / after the replacement text in s/search/replace/ while bash's ${name//search/replace} syntax doesn't.