I have file which contain following context like
abc...
include /home/user/file.txt'
some text
I need to remove include and also complete path after include.
I have used following command which remove include but did not remove path.
sed -i -r 's#include##g' 'filename'
I am also trying to understand above command but did not understand following thing ( copy paste from somewhere)
i - modify file change
r - read file
s- Need input
g - Need input
Try this,
$ sed '/^include /s/.*//g' file.txt
abc...
some text
It remove all the texts in a line which starts with include. s means substitute. so s/.*//g means replace all the texts with null.g means global. The substitution will be applied globally.
OR
$ sed '/^include /d' file.txt
abc...
some text
d means delete.
It deletes the line which starts with include. To save the changes made(inline edit), your commands should be
sed -i '/^include /s/.*//g' file.txt
sed -i '/^include /d' file.txt
I your case if you just want to delete the second line, you can use:
sed -i '2d' file
If you want to explore something about linux commands then man pages are there for you.
Just go to terminal and type:
man sed
as per your question, The above command without -i will show the file content on terminal by deleting the second line from the input file. However, the input file remains unchanged. To update the original file or to make the changes permanently in the source file, use the -i option.
-i[SUFFIX], --in-place[=SUFFIX] :
edit files in place (makes backup if extension supplied)
-r or --regexp-extended :
option is to use extended regular expressions in the script.
s/regexp/replacement/ :
Attempt to match regexp against the pattern space. If successâ
ful, replace that portion matched with replacement. The
replacement may contain the special character & to refer to that
portion of the pattern space which matched, and the special
escapes \1 through \9 to refer to the corresponding matching
sub-expressions in the regexp.
g G : Copy/append hold space to pattern space.
grep -v
This is not about learning sed, but as an alternative (and short) solution, there is:
grep -v '^include' filename_in
Or with output redirection:
grep -v '^include' filename_in > filename_out
-v option for grep inverts matching (hence printing non-matching lines).
For simple deletion that's what I'd use; if you have to modify your path after the include, stick with sed instead.
You can use awk to just delete the line:
awk '/^include/ {next}1' file
sed -i -r 's#include##g' 'filename'
-i: you directly modify the treated file, by default, sed read a file, modify the content via stdout (the original file stay the same).
-r: use of extended regular expression (and not reduce to POSIX limited one).This is not necessary in this case due to simple POSIX compliant action in action list (the s### string).
s#pattern#NewValue#: substitute in current line the pattern (Regular Expression) with "Newvalue" (that also use internal buffer or specific value). The traditionnal form is s/// but in this case, using / in path (pattern or new value) an alternate form is used to avoid to escape all / in pattern or new value
g: is an option of s### that specify change EVERY occurence and not the first (by default)
so here it replace ANY occurence of include by nothing (remove) directly into your file
As per the Avinash Raj solution you got what you want but you want some explaination about some parameter used in sed command
First one is
command: s for substitution
With the sed command the substitute command s changes all occurrences of the regular expression into a new value. A simple example is changing "my" in the "file1" to "yours" in the "file2" file:
sed s/my/yours/ file1 >file2
The character after the s is the delimiter. It is conventionally a slash, because this is what ed, more, and vi use. It can be anything you want, however. If you want to change a pathname that contains a slash - say /usr/local/bin to /common/bin - you could use the backslash to quote the slash:
sed 's/\/usr\/local\/bin/\/common\/bin/' <old >new
/g - Global replacement
Replace all matches, not just the first match.
If you tell it to change a word, it will only change the first occurrence of the word on a line. You may want to make the change on every word on the line instead of the first then add a g after the last delimiter and use the work-around:
Delete with d
Delete the pattern space; immediately start next cycle.
You can delete line by specifying the line number. like
sed '$d' filename.txt
It will remove last line of file
sed '2 d' file.txt
It will delete second line of file.
-i option
This option specifies that files are to be edited in-place. GNU sed does this by creating a temporary file and sending output to this file rather than to the standard output.
To modify file actully you can use -i option without it sed command repressent changes on stdout not actual file. You can take backup of original file before modification by using -i.bak option.
-r option
--regexp-extended
Use extended regular expressions rather than basic regular expressions. Extended regexps are those that egrep accepts; they can be clearer because they usually have less backslashes, but are a GNU extension and hence scripts that use them are not portable.
Related
I want to remove the first two characters of a column in a text file.
I am using the below but this is also truncating the headers.
sed -i 's/^..//' file1.txt
Below is my file:
FileName,Age
./Acct_Bal_Tgt.txt,7229
./IDQ_HB1.txt,5367
./IDQ_HB_LOGC.txt,5367
./IDQ_HB.txt,5367
./IGC_IDQ.txt,5448
./JobSchedule.txt,3851
I want the ./ to be removed from each line in the file name.
Transferring comments to an answer, as requested.
Modify your script to:
sed -e '2,$s/^..//' file1.txt
The 2,$ prefix limits the change to lines 2 to the end of the file, leaving line 1 unchanged.
An alternative is to remove . and / as the first two characters on a line:
sed -e 's%^[.]/%%' file1.txt
I tend to use -e to specify that the script option follows; it isn't necessary unless you split the script over several arguments (so it isn't necessary here where there's just one argument for the script). You could use \. instead of [.]; I'm allergic to backslashes (as you would be if you ever spent time working out whether you needed 8 or 16 consecutive backslashes to get the right result in a troff document).
Advice: Don't use the -i option until you've got your script working correctly. It overwrites your file with the incorrect output just as happily as it will with the correct output. Consequently, if you're asking about how to write a sed script on SO, it isn't safe to be using the -i option. Also note that the -i option is non-standard and behaves differently with different versions of sed (when it is supported at all). Specifically, on macOS, the BSD sed requires a suffix specified; if you don't want a backup, you have to use two arguments: -i ''.
Use this Perl one-liner:
perl -pe 's{^[.]/}{}' file1.txt > output.txt
The Perl one-liner uses these command line flags:
-e : Tells Perl to look for code in-line, instead of in a file.
-p : Loop over the input one line at a time, assigning it to $_ by default. Add print $_ after each loop iteration.
s{^[.]/}{} : Replace a literal dot ([.]) followed by a slash ('/'), found at the beginning of the line (^), with nothing (delete them). This does not modify the header since it does not match the regex.
If you prefer to modify the file in-place, you can use this:
perl -i.bak -pe 's{^[.]/}{}' file1.txt
This creates the backup file file1.txt.bak.
SEE ALSO:
perldoc perlrun: how to execute the Perl interpreter: command line switches
perldoc perlrequick: Perl regular expressions quick start
I would like some advice on this script.
I'm trying to use sed (I didn't manage it with rename) to change a file that contains lines of the format (my test file name is sedtest):
COPY W:\Interfaces\Payments\Tameia\Unprocessed\X151008\E*.*
(that's not the only content of the file).
My goal is to replace the 151008 date part with a different date, I've tried to come up with a solution in sed using this:
sed -i -e "s/Unprocessed\X.*/Unprocessed\X'BLABLA'/" sedtest
but it doesnt seem to work, the line remains unchanged, it's like it doesn't recognize the pattern because of the \. I've tried some alternative delimiters like #, but to no avail.
Thanks in advance for any advice.
There's a couple of issues with your sed command. I would suggest changing it to this:
sed -r 's/(Unprocessed\\X)[0-9]+/\1BLABLA/' file
Since your version of sed supports -i without requiring that you add a suffix to create a backup file, I assume you're using the GNU version, which also supports extended regular expressions with the -r switch. The command captures the part within the () and uses it in the replacement \1. Don't forget that backslashes must be escaped.
If you're going to use -i, I would recommend doing so like -i.bak, so a backup of your file is made to file.bak before it is overwritten.
You haven't shown the exact output you were looking for but I assumed that you wanted the line to become:
COPY W:\Interfaces\Payments\Tameia\Unprocessed\XBLABLA\E*.*
Remember that * is greedy, so .* would match everything up to the end of the line. That's why I changed it to [0-9]+, so that only the digits were replaced, leaving the rest of the line intact.
As you've mentioned using a variable in the replacement, you should use something like this:
sed -r -i.bak "s/(Unprocessed\\X)[0-9]+/\1$var/" file
This assumes that $var is safe to use, i.e. doesn't contain characters that will be interpreted by sed, like \, / or &. See this question for details on handling such cases reliably.
I need to insert the text at each line only if the given pattern matches with that line.
For example,
sed -n '/pattern/p' /etc/inittab/
so, if the pattern matches with any of the lines in inittab file, then i need to insert '#' at the beginning of those lines in the same file itself.
Kindly suggest me, how to make this.
Using sed:
sed '/pattern/s/^/#/' file
This will look for lines matching the pattern and once it finds it, it will place # in front of it. This will not modify the file. In order to do so, you need to use -i option to make in-place changes. You can put an extension like -i.bak to make an optional back if you'd like.
Using awk:
awk '/pattern/{$0="#"$0}1' file
awk is made up by pattern action statements. For the matching pattern, the action we do is modify the line by placing # in front of it. The 1 at the end will print the lines for us. GNU awk v4.1 or later has in-place editing just like sed. If you are using an older version you can redirect the output to another file and mv it back to original by saying:
awk '/pattern/{$0="#"$0}1' file > tmp && mv tmp file
The in-place changes is nothing special. It does the same job as redirecting to a temp file and then moving it back. It just does all the dirty work for you behind the scenes.
This is achieved with the following sed invocation
% sed -i.orig -e '/pattern/s/^/#/' inittab
The -i.orig option tells sed to operate in place on the file, previously saving the original as inittab.orig. The editing pattern
/pattern/ selects lines matching pattern
s/^/#/ and substitute the empty word at the beginning of line with #
I know sed could be used to delete specific line from file:
sed -i "/pattern/d" file
While the pattern of my case includes slash, like /var/log,
So I know I need escape: sed -i "/\/tmp\/dir/d" file
However, for my case, the pattern is dynamic, should be a variable
in a shell file, so I have to convert the variable value to replace
"/" with "\\/", then got this:
sed -i "/^${pattern_variable//\\//\\\\\\/}$/d" file
My question is, is there any better implementation which is more readable or simpler? Not only sed, other utility is also acceptable. Is it possible to handle not only slash but also other various symbols, like backslash or # ()?
you can use char other than /:
sed "\#$varHasSlash#d"
example:
kent$ foo="b/c"
kent$ echo "a
ab/cd
e"|sed "\#$foo#d"
a
e
I have a series of text files that I want to convert to markdown. I want to remove any leading spaces and add a hash sign to the first line in every file. If I run this:
sed -i.bak '1s/ *\(.*\)/\#\1/g' *.md
It alters the first line of the first file and processes them all, leaving the rest of the files unchanged.
What am I missing that will search and replace something on the n-th line of multiple files?
Using bash on OSX 10.7
The problem is that sed by default treats any number of files as a single stream, and thus line-number offsets are relative to the start of the first file.
For GNU sed, you can use the -s (--separate) flag to modify this behavior:
sed -s -i.bak '1s/^ */#/' *.md
...or, with non-GNU sed (including the one on Mac OS X), you can loop over the files and invoke once per each:
for f in *.md; do sed -i.bak '1s/^ */#/' "$f"; done
Note that the regex is a bit simplified here -- no need to match parts of the line that you aren't going to change.
XARgs will do the trick for you:
http://en.wikipedia.org/wiki/Xargs
Remove the *.md from the end of your sed command, then use XArgs to gather your files one at a time and send them to your sed command as a single entity, sorry I don't have time to work it out for you but the wikiPedia article should show you what you need to know.
sed -rsi.bak '1s/^/#/;s/^[ \t]+//' *.md
You don't need g(lobally) at the end of the command(s), because you wan't to replace something at the begin of line, and not multiple times.
You use two commands, one to modify line 1 (1s...), seperated from the second command for the leading blanks (and tabs? :=\t) with a semicolon. To remove blanks in the first line, switch the order:
sed -rsi.bak 's/^[ \t]+//;1s/^/#/' *.md
Remove the \t if you don't need it. Then you don't need a group either:
sed -rsi.bak 's/^ +//;1s/^/#/' *.md
-r is a flag to signal special treatment of regular expressions. You don't need to mask the plus in that case.