I would like to add two lines after the first string search. I am using:
$ cat file1
HAI
BYE
HAI
ONE
TWO
$ VAR=`cat -n file1 |grep -w HAI |head -1 |awk '{print $1}'`
$ sed "$VAR a\
LINE ONE \
LINE TWO
" file1
It is giving the following output.
HAI
LINE ONE LINE TWO
BYE
HAI
ONE
TWO
But I want the output to be:
HAI
LINE ONE
LINE TWO
BYE
HAI
ONE
TWO
How can I achieve this? I tried to keep the \n but it is giving errors.
Replace your sed command with this:
sed $VAR' a\
LINE ONE\
LINE TWO
' file1
btw your earlier grep, awk can also be reduced to this:
VAR=$(awk '$1 == "HAI" && NR==1{print NR}' file1)
Much Better is to get full answer in single awk command like this:
awk '{if ($1=="HAI" && done!=1) {done=1; printf("%s\nLINE ONE\nLINE TWO\n", $0);} \
else print $0}' file1
OUTPUT:
HAI
LINE ONE
LINE TWO
BYE
HAI
ONE
TWO
$ sed '/HAI/{s/.*/&\nLINE ONE\nLINE TWO/;:a;n;ba}' file1
HAI
LINE ONE
LINE TWO
BYE
HAI
ONE
TWO
Explanation:
/HAI/ search this pattern
s/.*/&\nLINE ONE\nLINE TWO/ append two lines
:a define a label(a)
n read next line
ba jump to label(a)
You can use the a command instead of s:
$ sed '
> /HAI/{
> a\
> LINE ONE\
> LINE TWO
> :a
> n
> ba
> }' file1
This might work for you (GUN sed):
sed '0,/HAI/a\LINE ONE\nLINE TWO' file
or this (any sed):
sed 'x;/./{x;b};x;/HAI/!b;h;a\LINE ONE\nLINE TWO' file
Related
I have shell script variable var="7,8,9"
These are the line number use to delete to file using sed.
Here I tried:
sed -i "$var"'d' test_file.txt
But i got error `sed: -e expression #1, char 4: unknown command: ,'
Is there any other way to remove the line?
sed command doesn't accept comma delimited line numbers.
You can use this awk command that uses a bit if BASH string manipulation to form a regex with the given comma separated line numbers:
awk -v var="^(${var//,/|})$" 'NR !~ var' test_file.txt
This will set awk variable var as this regex:
^(7|8|9)$
And then condition NR !~ var ensures that we print only those lines that don't match above regex.
For inline editing, if you gnu-awk with version > 4.0 then use:
awk -i inplace -v var="^(${var//,/|})$" 'NR !~ var' test_file.txt
Or for older awk use:
awk -v var="^(${var//,/|})$" 'NR !~ var' test_file.txt > $$.tmp && mv $$.tmp test_file.txt
I like sed, you were close to it. You just need to split each line number into a separate command. How about this:
sed -e "$(echo 1,3,4 | tr ',' '\n' | while read N; do printf '%dd;' $N; done)"
do like this:
sed -i "`echo $var|sed 's/,/d;/g'`d;" file
Another option to consider would be ed, with printf '%s\n' to put commands onto separate lines:
lines=( 9 8 7 )
printf '%s\n' "${lines[#]/%/d}" w | ed -s file
The array lines contains the line numbers to be deleted; it's important to put these in descending order! The expansion ${lines[#]/%/d} adds a d (delete) command to each line number and w writes to the file at the end. You can change this to ,p instead, to check the output before overwriting your file.
As an aside, for this example, you could also just use 7,9 as a single entry in the array.
File1:
count line_num
xy 55
ab 67
File2:
a|b|c
d|e|f
I want to print 55, 67 line numbers of file2
am trying:
#!/usr/bin/ksh
while read file_name; do
line_num=`echo $file_name | awk '{print $2}'`
awk 'NR==$line_num{print;exit}' file2 >> file3.txt
done < file1
but it's not working!
Using awk you can do:
awk 'NR==FNR{line[$2]; next} FNR in line' file1 file2
We iterate the first file and store second column in a map called line (we could ignore the first line which is the header by doing NR>1 but since it doesn't contain numbers we don't need to). Once the first file is loaded in map, we iterate the second file and print out lines that are in our map. NR and FNR are awk variables that remembers the line numbers.
You can use awk to read the line numbers in a loop and sed to print out the specific lines:
while read a; do sed -n ${a}p f2.txt; done < <(awk 'NR>1{print$2}' f1.txt)
If you have a bigger file, performance can be an issue as Ed pointed out, in that case you can use awk alone:
awk 'NR==FNR{if(NR>1)l[$2]=1;next}{if(l[FNR])print $0}' f1.txt f2.txt
Another way, is to use xargs:
awk 'NR>1{print $2}' f1.txt | xargs -n1 -I {} sed -n {}p f2.txt
Use sed to construct a sed one-liner (in the case of file1 it'd output and run sed -n "55p;67p;" file2):
sed -n "$(sed -n '2~1{s/.* //;s/.*/&p/p}' file1)" file2
A good advertisement for awk, alas!
I have two files: file1 and file2.
Any match in file2 should append "-W" to the word in file1.
File1:
Verb=Applaud,Beg,Deliver
Adjective=Bitter,Salty,Minty
Adverb=Quickly,Truthfully,Firmly
file2:
Gate
Salty
Explain
Quickly
Hook
Deliver
Earn
Jones
Applaud
Take
Output:
Verb=Applaud-W,Beg,Deliver-W
Adjective=Bitter,Salty-W,Minty
Adverb=Quickly-W,Truthfully,Firmly
Tried but not working and may take too long:
for i in `cat file2` ; do
nawk -v DEE="$i" '{gsub(DEE, DEE"-W")}1' file1 > newfile
mv newfile file1
done
This should work:
sed 's=^=s/\\b=;s=$=\\b/\&-W/g=' file2 | sed -f- file1
Output:
Verb=Applaud-W,Beg,Deliver-W
Adjective=Bitter,Salty-W,Minty
Adverb=Quickly-W,Truthfully,Firmly
To make changes in place:
sed 's=^=s/\\b=;s=$=\\b/\&-W/g=' file2 | sed --in-place -f- file1
Your approach was not that bad but I would prefer sed here, since it has an in place option.
while read i
do
sed -i "s/$i/$i-W/g" file1
done < file2
Here is one using pure bash:
#!/bin/bash
while read line
do
while read word
do
if [[ $line =~ $word ]]; then
line="${line//$word/$word-W}"
fi
done < file2
echo $line
done < file1
An awk:
awk 'BEGIN{FS=OFS=",";RS="=|\n"}
NR==FNR{a[$1]++;next}
{
for (i=1;i<=NF;i++){
$i=($i in a) ? $i"-W":$i
}
printf("%s%s",$0,FNR%2?"=":"\n")
}' file2 file1
Results
Verb=Applaud-W,Beg,Deliver-W
Adjective=Bitter,Salty-W,Minty
Adverb=Quickly-W,Truthfully,Firmly
I have the following file:
>A6NGG8_201_I_F
line2
>B1AK53_719_S_R
line4
>B1AK53_744_D_N
line5
>B7U540_205_R_H
line6
>B7U540_354_T_M
line7
where I want to print out all odd lines. I can do this by:
$ sed -n 1~2p file
>A6NGG8_201_I_F
>B1AK53_719_S_R
>B1AK53_744_D_N
>B7U540_205_R_H
>B7U540_354_T_M
and so I want to store the number in each line as a variable in bash, however I run into a problem - storing the result of sed puts the output all on one line:
#!/bin/bash
line1=$(sed -n 1~2p)
echo ${line1}
in which the output is:
>A6NGG8_201_I_F >B1AK53_719_S_R >B1AK53_744_D_N >B7U540_205_R_H >B7U540_354_T_M
so that when I do something like:
#!/bin/bash
line1=$(sed -n 1~2p)
pos=$(echo ${line1} | awk -F"[__]" 'NF>2{print $2}')
echo ${pos}
I get
201
where I of course want:
201
719
744
205
354
How do I store the result of sed into separate lines so that they are processed properly when piped into my awk statement? I see you can use the /anotation, however when I tried sed -n '/1~2p/a' filethis does not work in my bash script. Thanks
As said in comments, you need to quote the variable to make this happen:
echo "${line1}"
instead of
echo ${line1}
However, you can directly say:
awk -F_ 'NR%2 && NF>2 {print $2}' file
This will process even lines and, in them, print the 2nd field on _ separated, just if it there are more than 2 fields.
From tripleee's answer I observe that a FASTA file can contain a different format. If so, I guess you will still want to get the ID in the lines starting with ">". This can be translated as:
awk -F_ '/^>/ && NF>2 {print $2}' file
See an example of how quoting preserves the format:
The file:
$ cat a
hello
bye
Read it into a variable:
$ var=$(< a)
echo without quoting:
$ echo $var
hello bye
Let's quote!
$ echo "$var"
hello
bye
If you are trying to get the header lines out of a FASTA file, your problem statement is wrong -- the data between the headers could be more than one line. You could simply do
sed -n '/^>/!d;s/^[^_]*//;s/_.*//p' file.fasta
to get just the second underscore-delimited field out of each header line; or equivalently, in Awk,
awk -F _ '/^>/ { print $2 }' file.fasta
I have a file1:
hello
world
And file2:
hello A
hello X
world B
byebye C
And I want to delete all lines that match the string in file1 from file2, to get this output:
byebye C
I am a beginner programming so I could only come up with this:
for i in {1..2}
do p=`sed -n ${i}p file1`
sed '/$p\t\w/d' file2 > file2.tmp && mv file.tmp file2
done
Thanks for the help!
Using awk:
awk 'FNR==NR{a[$0];next} !($1 in a)' f1 f2
byebye C
Using grep -vf
grep -vFf f1 f2
byebye C
You can use this grep command
grep -vf file1 file2 > file2
Your coding:
for i in {1..2}
do
p=`sed -n ${i}p file1`
sed -i "/^$p/d" file2
done
Why didnt you code work
sed does not understand \w
solution use -r flag for extended regex
'/$p\t\w/d' the variables are not expanded in single quotes
solution use double quotes instead
Corrected
$for i in {1..2} ;
do
p=`sed -n ${i}p file1`
sed -r "/$p\t\w/d" file2 > file2.tmp && mv file2.tmp file2
done
$ cat file2
byebye C
This will provide output as
byebye C
Note
Offcourse this provides the expected output, But there is always some other easy ways of accomplishing the task as in other answers