So I have a bunch of files that I want to edit at once using sed but the problem is that I need to edit a line and change the text to a file name stored in variable filename. Every time I tried, it changes the text to the literal "filename" and I don't know how to fix it.
The command I've used is:
sed -i 's/$x/'$filename'/g' *.html
From your description and the command used I assume that you try to replace a variable text stored in
X="text2replace"
with a filename stored in
FILENAME="filename"
According this, a command like
sed -i "s/${X}/${FILENAME}/g" *.html
should do the job. It will replace all occurrences of text2replace in all HTML files found with the string filename.
You may have also a look into
When do we need curly braces around shell variables
When to wrap quotes around a shell variable
This might work for you (GNU parallel and sed):
parallel --header : sed -i 's#{x}#{filename}#' {file} ::: file *.html ::: x pattern ::: filename name
Related
I have a file r. I want to replace the words File and MINvac.pdb in it with nothing. The commands I used are
sed -i 's/File//g' /home/kanika/standard_minimizer_prosee/r
and
sed -i 's/MINvac.pdb//g' /home/kanika/standard_minimizer_prosee/r
I want to combine both sed commands into one, but I don't know the way. Can anyone help?
The file looks like this:
-6174.27 File10MINvac.pdb
-514.451 File11MINvac.pdb
4065.68 File12MINvac.pdb
-4708.64 File13MINvac.pdb
6674.54 File14MINvac.pdb
8563.58 File15MINvac.pdb
sed is a scripting language. You separate commands with semicolon or newline. Many sed dialects also allow you to pass each command as a separate -e option argument.
sed -i 's/File//g;s/MINvac\.pdb//g' /home/kanika/standard_minimizer_prosee/r
I also added a backslash to properly quote the literal dot before pdb, but in this limited context that is probably unimportant.
For completeness, here is the newline variant. Many newcomers are baffled that the shell allows literal newlines in quoted strings, but it can be convenient.
sed -i 's/File//g
s/MINvac\.pdb//g' /home/kanika/standard_minimizer_prosee/r
Of course, in this limited case, you could also combine everything into one regex:
sed -i 's/\(File\|MINvac\.pdb\)//g' /home/kanika/standard_minimizer_prosee/r
(Some sed dialects will want this without backslashes, and/or offer an option to use extended regular expressions, where they should be omitted. BSD sed, and thus also MacOS sed, demands a mandatory argument to sed -i which can however be empty, like sed -i ''.)
Use the -e flag:
sed -i -e 's/File//g' -e 's/MINvac.pdb//g' /home/kanika/standard_minimizer_prosee/r
Once you get more commands than are convenient to define with -es, it is better to store the commands in a separate file and include it with the -f flag.
In this case, you'd make a file containing:
s/File//g
s/MINvac.pdb//g
Let's call that file 'sedcommands'. You'd then use it with sed like this:
sed -i -f sedcommands /home/kanika/standard_minimizer_prosee/r
With only two commands, it's probably not worthwhile using a separate file of commands, but it is quite convenient if you have a lot of transformations to make.
file.txt
...
<LOCAL_PATH_TO_REPO>/src/java/example.java
...
^A longer file but this pretty much explains what I am trying to do.
script.sh
dir=$(pwd)
# replace <LOCAL_PATH_TO_REPO> with dir
I tried using the sed command but it did not work for some reason. Any ideas on how to do this?
Your error means you have backslashes in the variable text.
The simplest solution is to change the delimiter to the one that does not occur in the variable text.
If there are no commas use a comma:
sed -i "s,LOCAL_PATH_TO_REPO,$PWD," file.yml
The -i flag introduces changes into the input file (works for GNU sed).
I have a line in a autoexec.py file that I want another script to be able to modify:
kodi.executebuiltin("PlayMedia(/path/to/file)")
I was thinking of using sed to override the value within PlayMedia() depending on certain conditions.
Can sed be used to only touch and overwrite the parts between the PlayMedia() brackets and nothing else? Or is further processing/regex needed?
You could use sed. The below code will replace the chars present inside the brackets following PlayMedia string with foo.
sed 's/\b\(PlayMedia\)([^)]*)/\1(foo)/g' file
If the string you want to replace is a path, you must use a different sed delimiter because filepath may contain forward slashes.
sed 's~\b\(PlayMedia\)([^)]*)~\1(foo)~g' file
Example:
$ echo 'kodi.executebuiltin("PlayMedia(/path/to/file)")' | sed 's/\b\(PlayMedia\)([^)]*)/\1(foo)/g'
kodi.executebuiltin("PlayMedia(foo)")
I have file which contain following context like
abc...
include /home/user/file.txt'
some text
I need to remove include and also complete path after include.
I have used following command which remove include but did not remove path.
sed -i -r 's#include##g' 'filename'
I am also trying to understand above command but did not understand following thing ( copy paste from somewhere)
i - modify file change
r - read file
s- Need input
g - Need input
Try this,
$ sed '/^include /s/.*//g' file.txt
abc...
some text
It remove all the texts in a line which starts with include. s means substitute. so s/.*//g means replace all the texts with null.g means global. The substitution will be applied globally.
OR
$ sed '/^include /d' file.txt
abc...
some text
d means delete.
It deletes the line which starts with include. To save the changes made(inline edit), your commands should be
sed -i '/^include /s/.*//g' file.txt
sed -i '/^include /d' file.txt
I your case if you just want to delete the second line, you can use:
sed -i '2d' file
If you want to explore something about linux commands then man pages are there for you.
Just go to terminal and type:
man sed
as per your question, The above command without -i will show the file content on terminal by deleting the second line from the input file. However, the input file remains unchanged. To update the original file or to make the changes permanently in the source file, use the -i option.
-i[SUFFIX], --in-place[=SUFFIX] :
edit files in place (makes backup if extension supplied)
-r or --regexp-extended :
option is to use extended regular expressions in the script.
s/regexp/replacement/ :
Attempt to match regexp against the pattern space. If success‐
ful, replace that portion matched with replacement. The
replacement may contain the special character & to refer to that
portion of the pattern space which matched, and the special
escapes \1 through \9 to refer to the corresponding matching
sub-expressions in the regexp.
g G : Copy/append hold space to pattern space.
grep -v
This is not about learning sed, but as an alternative (and short) solution, there is:
grep -v '^include' filename_in
Or with output redirection:
grep -v '^include' filename_in > filename_out
-v option for grep inverts matching (hence printing non-matching lines).
For simple deletion that's what I'd use; if you have to modify your path after the include, stick with sed instead.
You can use awk to just delete the line:
awk '/^include/ {next}1' file
sed -i -r 's#include##g' 'filename'
-i: you directly modify the treated file, by default, sed read a file, modify the content via stdout (the original file stay the same).
-r: use of extended regular expression (and not reduce to POSIX limited one).This is not necessary in this case due to simple POSIX compliant action in action list (the s### string).
s#pattern#NewValue#: substitute in current line the pattern (Regular Expression) with "Newvalue" (that also use internal buffer or specific value). The traditionnal form is s/// but in this case, using / in path (pattern or new value) an alternate form is used to avoid to escape all / in pattern or new value
g: is an option of s### that specify change EVERY occurence and not the first (by default)
so here it replace ANY occurence of include by nothing (remove) directly into your file
As per the Avinash Raj solution you got what you want but you want some explaination about some parameter used in sed command
First one is
command: s for substitution
With the sed command the substitute command s changes all occurrences of the regular expression into a new value. A simple example is changing "my" in the "file1" to "yours" in the "file2" file:
sed s/my/yours/ file1 >file2
The character after the s is the delimiter. It is conventionally a slash, because this is what ed, more, and vi use. It can be anything you want, however. If you want to change a pathname that contains a slash - say /usr/local/bin to /common/bin - you could use the backslash to quote the slash:
sed 's/\/usr\/local\/bin/\/common\/bin/' <old >new
/g - Global replacement
Replace all matches, not just the first match.
If you tell it to change a word, it will only change the first occurrence of the word on a line. You may want to make the change on every word on the line instead of the first then add a g after the last delimiter and use the work-around:
Delete with d
Delete the pattern space; immediately start next cycle.
You can delete line by specifying the line number. like
sed '$d' filename.txt
It will remove last line of file
sed '2 d' file.txt
It will delete second line of file.
-i option
This option specifies that files are to be edited in-place. GNU sed does this by creating a temporary file and sending output to this file rather than to the standard output.
To modify file actully you can use -i option without it sed command repressent changes on stdout not actual file. You can take backup of original file before modification by using -i.bak option.
-r option
--regexp-extended
Use extended regular expressions rather than basic regular expressions. Extended regexps are those that egrep accepts; they can be clearer because they usually have less backslashes, but are a GNU extension and hence scripts that use them are not portable.
I have a very basic understanding of shell scripting, but what I need to do requires more complex commands.
For one task, I need to find and replace html code within the index.html files on my server. These files are in multiple directories with a consistent naming convention. ([letter][3-digit number]) See the example below.
files: index.html
path: /www/mysite/board/today/[rsh][0-9]/
string to find: (div id="id")[code](/div)<--#include="(path)"-->(div id="id")[more code](/div)
string to replace with: (div id="id")<--include="(path)"-->(/div)
I hope you don't mind the pseudo-regex. The folders containing my target index.html files look similar to r099, s017, h123. And suffice the say, the html code I'm trying to replace is relatively long, but its still just a string.
The second task is similar to the first, only the filename changes as well.
files: [rsh][0-9].html
path: www/mysite/person/[0-9]/[0-9]/[0-9]/card/2011/
string: (div id="id")[code](/div)<--include="(path)"-->(div id="id")[more code](/div)
string to replace with: (div id="id")<--include="(path)"-->(/div)
I've seen other examples on SO and elsewhere on the net that simply show scripts modifying files under a single directory to find & replace a string without any special characters, but I haven't seen an example similar to what I'm trying to do just yet.
Any assistance would be greatly appreciated.
Thank You.
You have three separate sub-problems:
replacing text in a file
coping with special characters
selecting files to apply the transformation to
1. The canonical text replacement tool is sed:
sed -e 's/PATTERN/REPLACEMENT/g' <INPUT_FILE >OUTPUT_FILE
If you have GNU sed (e.g. on Linux or Cygwin), pass -i to transform the file in place. You can act on more than one file in the same command line.
sed -i -e 's/PATTERN/REPLACEMENT/g' FILE OTHER_FILE…
If your sed doesn't have the -i option, you need to write to a different file and move that into place afterwards. (This is what GNU sed does behind the scenes.)
sed -e 's/PATTERN/REPLACEMENT/g' <FILE >FILE.tmp
mv FILE.tmp FILE
2. If you want to replace a literal string by a literal string, you need to prefix all special characters by a backslash. For sed patterns, the special characters are .\[^$* plus the separator for the s command (usually /). For sed replacement text, the special characters are \& and newlines. You can use sed to turn a string into a suitable pattern or replacement text.
pattern=$(printf %s "$string_to_replace" | sed -e 's![.\[^$*/]!\\&!g')
replacement=$(printf %s "$replacement_string" | sed -e 's![\&]!\\&!g')
3. To act on multiple files directly in one or more directories, use shell wildcards. Your requirements don't seem completely consistent; I think these are the patterns you're looking for, but be sure to review them.
/www/mysite/board/today/[rsh][0-9][0-9][0-9]/index.html
/www/mysite/person/[0-9]/[0-9]/[0-9]/card/2011/[rsh][0-9].html
This will match files like /www/mysite/board/today/r012/index.html and /www/mysite/person/4/5/6/card/2011/h7.html, but not /www/mysite/board/today/subdir/s012/index.html or /www/mysite/board/today/r1234/index.html.
If you need to act on files in subdirectories recursively, use find. It doesn't seem to be in your requirements and this answer is long enough already, so I'll stop here.
4. Putting it all together:
string_to_replace='(div id="id")[code](/div)<--#include="(path)"-->(div id="id")[more code](/div)'
replacement_string='(div id="id")<--include="(path)"-->(/div)'
pattern=$(printf %s "$string_to_replace" | sed -e 's![.\[^$*/]!\\&!g')
replacement=$(printf %s "$replacement_string" | sed -e 's![\&]!\\&!g')
sed -i -e "s/$pattern/$replacement/g" \
/www/mysite/board/today/[rsh][0-9][0-9][0-9]/index.html \
/www/mysite/person/[0-9]/[0-9]/[0-9]/card/2011/[rsh][0-9].html
Final note: you seem to be working on HTML with regular expressions. That's often not a good idea.
Finding the files can easily be done using find -regex:
find www/mysite/board/today -regex ".*[rsh][0-9][0-9][0-9]/index.html"
find www/mysite/person -regex ".*[0-9]/[0-9]/[0-9]/card/2011/[rsh][0-9][0-9][0-9].html"
Due to nature of HTML, replacing the content might not be very easy with sed, so I would suggest using an HTML or XML parsing library in a perl script. Can you provide a short sample of an actual html file and the result of the replacements?