How do you replace a blank line in a file with a certain character using sed?
I have used the following command but it still returns the original input:
sed 's/^$/>/' filename
Original input:
ACTCTATCATC
CTACTATCTATCC
CCATCATCTACTC
...
Desired output:
ACTCTATCATC
>
CTACTATCTATCC
>
CCATCATCTACTC
>
...
Thanks for any help
Here is a way with awk. This wouldn't care if you have spaces or blank lines:
awk '!NF{$0=">"}1' file
NF stands for number of fields. Since blank lines or lines with just spaces have no fields, we use that to insert your text. 1 triggers the condition to be true and prints the line:
Test:
$ cat -vet file
ACTCTATCATC$
$
CTACTATCTATCC$
$
CCATCATCTACTC$
$
$ are end of line markers
$ awk '!NF{$0=">"}1' file
ACTCTATCATC
>
CTACTATCTATCC
>
CCATCATCTACTC
>
You may have tabs or white spaces in your filename' empty lines, try the following:
sed 's/^\s*$/>/' filename
You may have whitespace in your input. First thing to try is:
sed 's/^[[:blank:]]*$/>/' filename
The following code should work:
sed -i 's/^[[:space:]]*$/string/' foo
What's missing here is the escape character. This will work for you.
sed 's/^$/\>/g' filename
And if you need to delete the empty lines and print others, Try
sed '/^$/d' filename
Related
I have a text file and I would like to only delete the first character of the text file, is there a way to do this in shell script?
I'm new to writing scripts so I really don't know where to start. I understand that the main command most people use is "sed" but I can only find how to use that as a find and replace tool.
All help is appreciated.
You can use the tail command, telling it to start from character 2:
tail -c +2 infile > outfile
You can use sed
sed '1s/^.//' startfile > endfile
1s means match line 1, in substitution mode (s)
^. means at the beginning of the line (^), match any character (.)
There's nothing between the last slashes, which means substitute with nothing (remove)
I used to use cut command to do this.
For example:
cat file|cut -c2-80
Will show characters from column 2 to 80 only.
In your case you can use:
cat file|cut -c2-10000 > newfile
I hope this help you.
[]s
You can also use the 0,addr2 address-range to limit replacements to the first substitution, e.g.
sed '0,/./s/^.//' file
That will remove the 1st character of the file and the sed expression will be at the end of its range -- effectively replacing only the 1st occurrence.
To edit the file in place, use the -i option, e.g.
sed -i '0,/./s/^.//' file
or simply redirect the output to a new file:
sed '0,/./s/^.//' file > newfile
A few other ideas:
awk '{print (NR == 1 ? substr($0,2) : $0)}' file
perl -0777 -pe 's/.//' file
perl -pe 's/.// unless $done; $done = 1' file
ed file <<END
1s/.//
w
q
END
dd allows you to specify an offset at which to start reading:
dd ibs=1 seek=1 if="$input" of="$output"
(where the variables are set to point to your input and output files, respectively)
I have a file, which has multiple lines.
For example:
a
ab#
ad.
a12fs
b
c
...
I want to use sed or awk delete the line, if the line include symbols or numbers. (For example, I want to delete: ab#, ad., a12fs.... lines)
or in another words, I just want to keep the line which include [a-z][A-Z] .
I know how to delete number line,
sed '/[0-9]/d' file.txt
but I do not know how to delete symbols lines.
Or there has any easy way to do that?
To keep blank lines:
grep '^[[:alpha:]]*$' file
sed '/[^[:alpha:]]/d' file
awk '/^[[:alpha:]]*$/' file
To remove blank lines:
grep '^[[:alpha:]]+$' file
sed -E -n '/^[[:alpha:]]+$/p' file
awk '/^[[:alpha:]]+$/' file
grep works well too and is even simpler: just do the reverse: keep the lines that interest you, which are way easier to define
grep -i '^[a-z]*$' file.txt
(match lines containing only letters and empty lines, and -i option makes grep case-insensitive)
to remove empty lines as well:
grep -i '^[a-z]+$' file.txt
caution when using Windows text files, as there's a carriage return at the end of the line, so nothing would match depending on grep versions (tested on windows here and it works)
but just in case:
grep -iP '^[a-z]*\r?$'
(note the P option to enable perl expressions or \r is not recognized)
You can use this sed:
sed '/^[A-Za-z0-9]\+$/!d' file
(OR)
sed '/[^A-Za-z0-9]/d' file
$ awk '!/[^[:alpha:]]/' file.txt
a
b
c
I have a tsv.-file and there are some lines which do not end with an '"'. So now I would like to remove every line break which is not directly after an '"'.
How could I accomplish that with sed? Or any other bash shell program...
Kind regards,
Snafu
This sed command should do it:
sed '/"$/!{N;s/\n//}' file
It says: on every line not matching "$ do:
read next line, append it to pattern space;
remove linebreak between the two lines.
Example:
$ cat file.txt
"test"
"qwe
rty"
foo
$ sed '/"$/!{N;s/\n//}' file.txt
"test"
"qwerty"
foo
To elaborate on #Lev's answer, the BSD (OSX) version of sed is less forgiving about the command syntax within the curly braces -- the semicolon command separator is required for both commands:
sed '/"$/!{N;s/\n//;}' file.txt
per the documentation here -- an excerpt:
Following an address or address range, sed accepts curly braces '{...}' so several commands may be applied to that line or to the lines matched by the address range. On the command line, semicolons ';' separate each instruction and must precede the closing brace.
give this awk one-liner a try:
awk '{printf "%s%s",$0,(/"$/?"\n":"")}' file
test
kent$ cat f
"foo"
"bar"
"a long
text with
many many
lines"
"lalala"
kent$ awk '{printf "%s%s",$0,(/"$/?"\n":"")}' f
"foo"
"bar"
"a longtext withmany manylines"
"lalala"
This might work for you (GNU sed):
sed ':a;/"$/!{N;s/\n//;ta}' file
This checks if the last character of the pattern space is a " and if not appends another line, removes a newline and repeats until the condition is met or the end-of-file is encountered.
An alternative is:
sed -r ':a;N;s/([^"])\n/\1/;ta;P;D' file
The mechanism is left for the reader to ponder.
All of the lines with comments in a file begin with #. How can I delete all of the lines (and only those lines) which begin with #? Other lines containing #, but not at the beginning of the line should be ignored.
This can be done with a sed one-liner:
sed '/^#/d'
This says, "find all lines that start with # and delete them, leaving everything else."
I'm a little surprised nobody has suggested the most obvious solution:
grep -v '^#' filename
This solves the problem as stated.
But note that a common convention is for everything from a # to the end of a line to be treated as a comment:
sed 's/#.*$//' filename
though that treats, for example, a # character within a string literal as the beginning of a comment (which may or may not be relevant for your case) (and it leaves empty lines).
A line starting with arbitrary whitespace followed by # might also be treated as a comment:
grep -v '^ *#' filename
if whitespace is only spaces, or
grep -v '^[ ]#' filename
where the two spaces are actually a space followed by a literal tab character (type "control-v tab").
For all these commands, omit the filename argument to read from standard input (e.g., as part of a pipe).
The opposite of Raymond's solution:
sed -n '/^#/!p'
"don't print anything, except for lines that DON'T start with #"
you can directly edit your file with
sed -i '/^#/ d'
If you want also delete comment lines that start with some whitespace use
sed -i '/^\s*#/ d'
Usually, you want to keep the first line of your script, if it is a sha-bang, so sed should not delete lines starting with #!. also it should delete lines, that just contain only a hash but no text. put it all together:
sed -i '/^\s*\(#[^!].*\|#$\)/d'
To be conform with all sed variants you need to add a backup extension to the -i option:
sed -i.bak '/^\s*#/ d' $file
rm -Rf $file.bak
You can use the following for an awk solution -
awk '/^#/ {sub(/#.*/,"");getline;}1' inputfile
This answer builds upon the earlier answer by Keith.
egrep -v "^[[:blank:]]*#" should filter out comment lines.
egrep -v "^[[:blank:]]*(#|$)" should filter out both comments and empty lines, as is frequently useful.
For information about [:blank:] and other character classes, refer to https://en.wikipedia.org/wiki/Regular_expression#Character_classes.
If you want to delete from the file starting with a specific word, then do this:
grep -v '^pattern' currentFileName > newFileName && mv newFileName currentFileName
So we have removed all the lines starting with a pattern, writing the content into a new file, and then copy the content back into the source/current file.
You also might want to remove empty lines as well
sed -E '/(^$|^#)/d' inputfile
Delete all empty lines and also all lines starting with a # after any spaces:
sed -E '/^$|^\s*#/d' inputfile
For example, see the following 3 deleted lines (including just line numbers!):
1. # first comment
2.
3. # second comment
After testing the command above, you can use option -i to edit the input file in place.
Just this!
Here is it with a loop for all files with some extension:
ll -ltr *.filename_extension > list.lst
for i in $(cat list.lst | awk '{ print $8 }') # validate if it is the 8 column on ls
do
echo $i
sed -i '/^#/d' $i
done
as title said, Im trying to change only the first occurrence of word.By using
sed 's/this/that/' file.txt
though i'm not using g option it replace entire file. How to fix this.?
UPDATE:
$ cat file.txt
first line
this
this
this
this
$ sed -e '1s/this/that/;t' file.txt
first line
this // ------> I want to change only this "this" to "that" :)
this
this
this
http://www.faqs.org/faqs/editor-faq/sed/
4.3. How do I change only the first occurrence of a pattern?
sed -e '1s/LHS/RHS/;t' -e '1,/LHS/s//RHS/'
Where LHS=this and RHS=that for your example.
If you know the pattern won't occur on the first line, omit the first -e and the statement following it.
sed by itself applies the edit thru out the file and combined with "g" flag the edit is applied to all the occurrences on the same line.
e.g.
$ cat file.txt
first line
this this
this
this
this
$ sed 's/this/that/' file.txt
first line
that this
that
that
that
$ sed 's/this/that/g' file.txt
first line
that that <-- Both occurrences of "this" have changed
that
that
that