I want to use command to edit the specific line of a file instead of using vi. This is the thing. If there is a # starting with the line, then replace the # to make it uncomment. Otherwise, add the # to make it comment. I'd like to use sed or awk. But it won't work as expected.
This is the file.
what are you doing now?
what are you gonna do? stab me?
this is interesting.
This is a test.
go big
don't be rude.
For example, I just want to add the # at the beginning of the the line 4 This is a test if it doesn't start with #. And if it starts with #, then remove the #.
I've already tried via sed & gawk (awk)
gawk -i inplace '$1!="#" {print "#",$0;next};{print substr($0,3,length-1)}' file
sed -i /test/s/^#// file # make it uncomment
sed -i /test/s/^/#/ file # make it comment
I don't know how to use if else to make sed work. I could only make it with a single command, then use another regex to make the opposite.
Using gawk, it works as the main line. But it will mess the rest of the code up.
This might work for you (GNU sed):
sed '4{s/^/#/;s/^##//}' file
On line 4 prepend a # to the line and if there 2 #'s remove them.
Could also be written:
sed '4s/^/#/;4s/^##//' file
This will remove # from the start of line 4 or add it if it wasn't already there:
sed -i '4s/^#/\n/; 4s/^[^\n]/#&/; 4s/^\n//' File
The above assume GNU sed. If you have BSD/MacOS sed, some minor changes will be required.
When sed reads a new line, the one thing that we know for sure about the new line is that it does not contain \n. (If it did, it would be two lines, not one.) Using this knowledge, the script works by:
s/^#/\n/
If the fourth line starts with #, replace # with \n. (The \n serves as a notice that the line had originally been commented out.)
4s/^[^\n]/#&/
If the fourth line now starts with anything other than \n (meaning that it was not originally commented), put a # in front.
4s/^\n//
If the fourth line now starts with \n, remove it.
Alternative: Modifying lines that contain test
To comment/uncomment lines that contain test:
sed '/test/{s/^#/\n/; s/^[^\n]/#&/; s/^\n//}' File
Alternative: using awk
The exact same logic can be applied using awk. If we want to comment/uncomment line 4:
awk 'NR==4 {sub(/^#/, "\n"); sub(/^[^\n]/, "#&"); sub(/^\n/, "")} 1' File
If we want to comment/uncomment any line containing test:
awk '/test/ {sub(/^#/, "\n"); sub(/^[^\n]/, "#&"); sub(/^\n/, "")} 1' File
Alternative: using sed but without newlines
To comment/uncomment any line containing test:
sed '/test/{s/^#//; t; s/^/#/; }' File
How it works:
s/^#//; t
If the line begins with #, then remove it.
t tells sed that, if the substitution succeeded, then it should skip the rest of the commands.
s/^/#/
If we get to this command, that means that the substitution did not succeed (meaning the line was not originally commented out), so we insert #.
If you end up on a system with a sed that doesn't support in-place editing, you can fall back to its uncle ed:
ed -s file 2>/dev/null <<EOF
4 s/^/#/
s/^##//
w
q
EOF
(Standard error is redirected to /dev/null because in ed, unlike sed, it's an error if s doesn't replace anything and a question mark is thus printed to standard error.)
$ awk 'NR==4{$0=(sub(/^#/,"") ? "" : "#") $0} 1' file
what are you doing now?
what are you gonna do? stab me?
this is interesting.
#This is a test.
go big
don't be rude.
$ awk 'NR==4{$0=(sub(/^#/,"") ? "" : "#") $0} 1' file |
awk 'NR==4{$0=(sub(/^#/,"") ? "" : "#") $0} 1'
what are you doing now?
what are you gonna do? stab me?
this is interesting.
This is a test.
go big
don't be rude.
Related
I am in a situation where I have so many fastq files that I want to convert to fasta.
Since they belong to the same sample, I would like to merge the fasta files to get a single file.
I tried running these two commands:
sed -n '1~4s/^#/>/p;2~4p' INFILE.fastq > OUTFILE.fasta
cat infile.fq | awk '{if(NR%4==1) {printf(">%s\n",substr($0,2));} else if(NR%4==2) print;}' > file.fa
And the output files is correctly a fasta file.
However I get a problem in the next step. When I merge files with this command:
cat $1 >> final.fasta
The final file apparently looks correct. But when I run makeblastdb it gives me the following error:
FASTA-Reader: Ignoring invalid residues at position(s): On line 512: 1040-1043, 1046-1048, 1050-1051, 1053, 1055-1058, 1060-1061, 1063, 1066-1069, 1071-1076
Looking at what's on that line I found that a file header was put at the end of the previous file sequence. And it turns out like this:
GGCTTAAACAGCATT>e45dcf63-78cf-4769-96b7-bf645c130323
So how can I add a blank line to the end of the file within the scripts that convert fastq to fasta?
So that when I merge they are placed on top of each other correctly and not at the end of the sequence of the previous file.
So how can I add a blank line to the end of the file within the
scripts that convert fastq to fasta?
I would use GNU sed following replace
cat $1 >> final.fasta
using
sed '$a\\n' $1 >> final.fasta
Explanation: meaning of expression for sed is at last line ($) append newline (\n) - this action is undertaken before default one of printing. If you prefer GNU AWK then you might same behavior following way
awk '{print}END{print ""}' $1 >> final.fasta
Note: I was unable to test any of solution as you doesnot provide enough information to this. I assume above line is somewhere inside loop and $1 is always name of file existing in current working directory.
if the only thing you need is extra blank line, and the input files are within 1.5 GB in size, then just directly do :
awk NF=NF RS='^$' FS='\n' OFS='\n'
Should work for mawk 1/2, gawk, and nawk, maybe others as well. This works despite appearing not to do anything special is that the extra \n comes from ORS.
I have a file with structure
"name1";"surname1";23;44
"name2";"surname2
www.so.org/443";56;33
"name3";"surname3";223;4554
"name4";"surname5
surname#so.net";77;889
I need an output:
"name1";"surname1";23;44
"name2";"surname2 www.so.org/443";56;33
"name3";"surname3";223;4554
"name4";"surname5 surname#so.net";77;889
The pattern here is alphanum at the start of the line and not \". I would like to paste the line with this pattern to the line above.
Edit:
I am using Debian stable.
I have used sed but I realized that it is a stream editor and I thought that it cannot paste a line to a previous one (which is false).
sed -e 's/^[a-z:A-Z]/ /g' which only help me to find the correct line.
My second trial was with a text editor. I opened the file with emacs and used M-x replace-regexp and find the corresponding lines with ^J[a-zA-Z] and replace with nothing. It did the job but it also deletes the first character and I need it after a single empty space.
This awk one-liner should give you a hand:
awk '{printf "%s%s",(/^"/&&NR>1?RS:""),$0}END{print ""}' file
The key to the problem is to decide when should we output/print the line break.
This one-liner works for even this format:
cat f
"name1";"surname1";23;44
"name2";"surname2
w
ww.
so.
org/
44
3";5
6;33
"name3";"surname3";223;4554
"name4";"surname5
surname#so.net";77;889
awk '{printf "%s%s",(/^"/&&NR>1?RS:""),$0}END{print ""}' f
"name1";"surname1";23;44
"name2";"surname2 www.so.org/443";56;33
"name3";"surname3";223;4554
"name4";"surname5 surname#so.net";77;889
This might work for you (GNU sed):
sed ':a;s/;/&/3;t;N;s/\n//;ta' file
If the current line does not contain 3 or more ;'s, append the next line, remove the introduced newline and repeat.
I'm looking for a line in bash that would work on both linux as well as OS X to remove the second line containing the desired string:
Header
1
2
...
Header
10
11
...
Should become
Header
1
2
...
10
11
...
My first attempt was using the deletion option of sed:
sed -i '/^Header.*/d' file.txt
But well, that removes the first occurence as well.
How to delete the matching pattern from given occurrence suggests to use something like this:
sed -i '/^Header.*/{2,$d} file.txt
But on OS X that gives the error
sed: 1: "/^Header.*/{2,$d}": extra characters at the end of d command
Next, i tried substitution, where I know how to use 2,$, and subsequent empty line deletion:
sed -i '2,$s/^Header.*//' file.txt
sed -i '/^\s*$/d' file.txt
This works on Linux, but on OS X, as mentioned here sed command with -i option failing on Mac, but works on Linux , you'd have to use
sed -i '' '2,$s/^Header.*//' file.txt
sed -i '' '/^\s*$/d' file.txt
And this one in return doesn't work on Linux.
My question then, isn't there a simple way to make this work in any Bash? Doesn't have to be sed, but should be as shell independent as possible and i need to modify the file itself.
Since this is file-dependent and not line-dependent, awk can be a better tool.
Just keep a counter on how many times this happened:
awk -v patt="Header" '$0 == patt && ++f==2 {next} 1' file
This skips the line that matches exactly the given pattern and does it for the second time. On the rest of lines, it prints normally.
I would recommend using awk for this:
awk '!/^Header/ || !f++' file
This prints all lines that don't start with "Header". Short-circuit evaluation means that if the left hand side of the || is true, the right hand side isn't evaluated. If the line does start with Header, the second part !f++ is only true once.
$ cat file
baseball
Header and some other stuff
aardvark
Header for the second time and some other stuff
orange
$ awk '!/^Header/ || !f++' file
baseball
Header and some other stuff
aardvark
orange
This might work for you (GNU sed):
sed -i '1b;/^Header/d' file
Ignore the first line and then remove any occurrence of a line beginning with Header.
To remove subsequent occurrences of the first line regardless of the string, use:
sed -ri '1h;1b;G;/^(.*)\n\1$/!P;d' file
I need to insert the text at each line only if the given pattern matches with that line.
For example,
sed -n '/pattern/p' /etc/inittab/
so, if the pattern matches with any of the lines in inittab file, then i need to insert '#' at the beginning of those lines in the same file itself.
Kindly suggest me, how to make this.
Using sed:
sed '/pattern/s/^/#/' file
This will look for lines matching the pattern and once it finds it, it will place # in front of it. This will not modify the file. In order to do so, you need to use -i option to make in-place changes. You can put an extension like -i.bak to make an optional back if you'd like.
Using awk:
awk '/pattern/{$0="#"$0}1' file
awk is made up by pattern action statements. For the matching pattern, the action we do is modify the line by placing # in front of it. The 1 at the end will print the lines for us. GNU awk v4.1 or later has in-place editing just like sed. If you are using an older version you can redirect the output to another file and mv it back to original by saying:
awk '/pattern/{$0="#"$0}1' file > tmp && mv tmp file
The in-place changes is nothing special. It does the same job as redirecting to a temp file and then moving it back. It just does all the dirty work for you behind the scenes.
This is achieved with the following sed invocation
% sed -i.orig -e '/pattern/s/^/#/' inittab
The -i.orig option tells sed to operate in place on the file, previously saving the original as inittab.orig. The editing pattern
/pattern/ selects lines matching pattern
s/^/#/ and substitute the empty word at the beginning of line with #
I'm working in bash trying to use sed substitution on a file and show both the line number where the substitution occurred and the final version of the line. For a file with lines that contain foo, trying with
sed -n 's/foo/bar/gp' filename
will show me the lines where substitution occurred, but I can't figure out how to include the line number. If I try to use = as a flag to print the current line number like
sed -n 's/foo/bar/gp=' filename
I get
sed: -e expression #1, char 14: unknown option to `s'
I can accomplish the goal with awk like
awk '{if (sub("foo","bar",$0)){print NR $0}}' filename
but I'm curious if there's a way to do this with one line of sed. If possible I'd love to use a single sed statement without a pipe.
I can't think of a way to do it without listing the search pattern twice and using command grouping.
sed -n "/foo/{s/foo/bar/g;=;p;}" filename
EDIT: mklement0 helped me out there by mentioning that if the pattern space is empty, the default pattern space is the last one used, as mentioned in the manual. So you could get away with it like this:
sed -n "/foo/{s//bar/g;=;p;}" filename
Before that, I figured out a way not to repeat the pattern space, but it uses branches and labels. "In most cases," the docs specify, "use of these commands indicates that you are probably better off programming in something like awk or Perl. But occasionally one is committed to sticking with sed, and these commands can enable one to write quite convoluted scripts." [source]
sed -n "s/foo/bar/g;tp;b;:p;=;p" filename
This does the following:
s/foo/bar/g does your substitution.
tp will jump to :p iff a substitution happened.
b (branch with no label) will process the next line.
:p defines label p, which is the target for the tp command above.
= and p will print the line number and then the line.
End of script, so go back and process the next line.
See? Much less readable...and maybe a distant cousin of :(){ :|:& };:. :)
It cannot be done in any reasonable way with sed, here's how to really do it clearly and simply in awk:
awk 'sub(/foo/,"bar"){print NR, $0}' filename
sed is an excellent tool for simple substitutions on a single line, for anything else use awk.