I have a file1:
hello
world
And file2:
hello A
hello X
world B
byebye C
And I want to delete all lines that match the string in file1 from file2, to get this output:
byebye C
I am a beginner programming so I could only come up with this:
for i in {1..2}
do p=`sed -n ${i}p file1`
sed '/$p\t\w/d' file2 > file2.tmp && mv file.tmp file2
done
Thanks for the help!
Using awk:
awk 'FNR==NR{a[$0];next} !($1 in a)' f1 f2
byebye C
Using grep -vf
grep -vFf f1 f2
byebye C
You can use this grep command
grep -vf file1 file2 > file2
Your coding:
for i in {1..2}
do
p=`sed -n ${i}p file1`
sed -i "/^$p/d" file2
done
Why didnt you code work
sed does not understand \w
solution use -r flag for extended regex
'/$p\t\w/d' the variables are not expanded in single quotes
solution use double quotes instead
Corrected
$for i in {1..2} ;
do
p=`sed -n ${i}p file1`
sed -r "/$p\t\w/d" file2 > file2.tmp && mv file2.tmp file2
done
$ cat file2
byebye C
This will provide output as
byebye C
Note
Offcourse this provides the expected output, But there is always some other easy ways of accomplishing the task as in other answers
Related
I have multiple files like this:
BOB_1.brother_bob12.txt
BOB_2.brother_bob12.txt
..
BOB_35.brother_bob12.txt
How to join these files in order from {1..36} and append filename at the end of each row? I have tried:
for i in *.txt; do sed 's/$/ '"$i"'/' $i; done > outfile #joins but not in order
cat $(for((i=1;i<38;i++)); do echo -n "BOB_${i}.brother_bob12.txt "; done) # joins in order but no filename at the end
file sample:
1 345 378 1 3 4 5 C T
1 456 789 -1 2 3 4 A T
Do not do cat $(....). You may just:
for ((i=1;i<38;i++)); do
f="BOB_${i}.brother_bob12.txt"
sed "s/$/ $f/" "$f"
done
You may also do:
printf "%s\n" bob.txt BOB_{1..38}.brother_bob12.txt |
xargs -d'\n' -i sed 's/$/ {}/' '{}'
You may use:
for i in {1..36}; do
fn="BOB_${i}.brother_bob12.txt"
[[ -f $fn ]] && awk -v OFS='\t' '{print $0, FILENAME}' "$fn"
done > output
Note that it will insert FILENAME as the last field in every record. If this is not what you want then show your expected output in question.
This might work for you (GNU sed);
sed -n 'p;F' BOB_{1..36}.brother_bob12.txt | sed 'N;s/\n/ /' >newFile
Used 2 invocations of sed, the first to append the file name after each line of each file. The second to replace the newline between each 2 lines by a space.
I have many files for which I have to estimate the number of lines in each file and add that value as first line. To estimate that, I used something like this:
wc -l 000600.txt | awk '{ print $1 }'
However, no success on how to do it for all files and then to add the value corresponding to each file as first line.
An example:
a.txt b.txt c.txt
>>print a
15
>> print b
22
>>print c
56
Then 15, 22 and 56 should be added respectively to: a.txt b.txt and c.txt
I appreciate the help.
You can add a pattern for example (LINENUM) in first line of file and then use the following script.
wc -l a.txt | awk 'BEGIN {FS =" ";} { print $1;}' | xargs -I {} sed -i 's/LINENUM/LINENUM:{}/' a.txt
or just use from this script:
wc -l a.txt | awk 'BEGIN {FS =" ";} { print $1;}' | xargs -I {} sed -i '1s/^/LINENUM:{}\n/' a.txt
This way you can add the line number as the first line for all *.txt files in current directory. Also using that group command here would be faster than inplace editing commands, in case of large files. Do not modify spaces or semicolons into the grouping.
for f in *.txt; do
{ wc -l < "$f"; cat "$f"; } > "${f}.tmp" && mv "${f}.tmp" "$f"
done
For iterate over the all file you can add use from this script.
for f in `ls *` ; do if [ -f $f ]; then wc -l $f | awk 'BEGIN {FS =" ";} { print $1;}' | xargs -I {} sed -i '1s/^/LINENUM:{}\n/' $f ; fi; done
This might work for you (GNU sed):
sed -i '1h;1!H;$!d;=;x' file1 file2 file3 etc ...
Store each file in memory and insert the last lines line number as the file size.
Alternative:
sed -i ':a;$!{N;ba};=' file?
File1:
count line_num
xy 55
ab 67
File2:
a|b|c
d|e|f
I want to print 55, 67 line numbers of file2
am trying:
#!/usr/bin/ksh
while read file_name; do
line_num=`echo $file_name | awk '{print $2}'`
awk 'NR==$line_num{print;exit}' file2 >> file3.txt
done < file1
but it's not working!
Using awk you can do:
awk 'NR==FNR{line[$2]; next} FNR in line' file1 file2
We iterate the first file and store second column in a map called line (we could ignore the first line which is the header by doing NR>1 but since it doesn't contain numbers we don't need to). Once the first file is loaded in map, we iterate the second file and print out lines that are in our map. NR and FNR are awk variables that remembers the line numbers.
You can use awk to read the line numbers in a loop and sed to print out the specific lines:
while read a; do sed -n ${a}p f2.txt; done < <(awk 'NR>1{print$2}' f1.txt)
If you have a bigger file, performance can be an issue as Ed pointed out, in that case you can use awk alone:
awk 'NR==FNR{if(NR>1)l[$2]=1;next}{if(l[FNR])print $0}' f1.txt f2.txt
Another way, is to use xargs:
awk 'NR>1{print $2}' f1.txt | xargs -n1 -I {} sed -n {}p f2.txt
Use sed to construct a sed one-liner (in the case of file1 it'd output and run sed -n "55p;67p;" file2):
sed -n "$(sed -n '2~1{s/.* //;s/.*/&p/p}' file1)" file2
A good advertisement for awk, alas!
I have two files: file1 and file2.
Any match in file2 should append "-W" to the word in file1.
File1:
Verb=Applaud,Beg,Deliver
Adjective=Bitter,Salty,Minty
Adverb=Quickly,Truthfully,Firmly
file2:
Gate
Salty
Explain
Quickly
Hook
Deliver
Earn
Jones
Applaud
Take
Output:
Verb=Applaud-W,Beg,Deliver-W
Adjective=Bitter,Salty-W,Minty
Adverb=Quickly-W,Truthfully,Firmly
Tried but not working and may take too long:
for i in `cat file2` ; do
nawk -v DEE="$i" '{gsub(DEE, DEE"-W")}1' file1 > newfile
mv newfile file1
done
This should work:
sed 's=^=s/\\b=;s=$=\\b/\&-W/g=' file2 | sed -f- file1
Output:
Verb=Applaud-W,Beg,Deliver-W
Adjective=Bitter,Salty-W,Minty
Adverb=Quickly-W,Truthfully,Firmly
To make changes in place:
sed 's=^=s/\\b=;s=$=\\b/\&-W/g=' file2 | sed --in-place -f- file1
Your approach was not that bad but I would prefer sed here, since it has an in place option.
while read i
do
sed -i "s/$i/$i-W/g" file1
done < file2
Here is one using pure bash:
#!/bin/bash
while read line
do
while read word
do
if [[ $line =~ $word ]]; then
line="${line//$word/$word-W}"
fi
done < file2
echo $line
done < file1
An awk:
awk 'BEGIN{FS=OFS=",";RS="=|\n"}
NR==FNR{a[$1]++;next}
{
for (i=1;i<=NF;i++){
$i=($i in a) ? $i"-W":$i
}
printf("%s%s",$0,FNR%2?"=":"\n")
}' file2 file1
Results
Verb=Applaud-W,Beg,Deliver-W
Adjective=Bitter,Salty-W,Minty
Adverb=Quickly-W,Truthfully,Firmly
I would like to add two lines after the first string search. I am using:
$ cat file1
HAI
BYE
HAI
ONE
TWO
$ VAR=`cat -n file1 |grep -w HAI |head -1 |awk '{print $1}'`
$ sed "$VAR a\
LINE ONE \
LINE TWO
" file1
It is giving the following output.
HAI
LINE ONE LINE TWO
BYE
HAI
ONE
TWO
But I want the output to be:
HAI
LINE ONE
LINE TWO
BYE
HAI
ONE
TWO
How can I achieve this? I tried to keep the \n but it is giving errors.
Replace your sed command with this:
sed $VAR' a\
LINE ONE\
LINE TWO
' file1
btw your earlier grep, awk can also be reduced to this:
VAR=$(awk '$1 == "HAI" && NR==1{print NR}' file1)
Much Better is to get full answer in single awk command like this:
awk '{if ($1=="HAI" && done!=1) {done=1; printf("%s\nLINE ONE\nLINE TWO\n", $0);} \
else print $0}' file1
OUTPUT:
HAI
LINE ONE
LINE TWO
BYE
HAI
ONE
TWO
$ sed '/HAI/{s/.*/&\nLINE ONE\nLINE TWO/;:a;n;ba}' file1
HAI
LINE ONE
LINE TWO
BYE
HAI
ONE
TWO
Explanation:
/HAI/ search this pattern
s/.*/&\nLINE ONE\nLINE TWO/ append two lines
:a define a label(a)
n read next line
ba jump to label(a)
You can use the a command instead of s:
$ sed '
> /HAI/{
> a\
> LINE ONE\
> LINE TWO
> :a
> n
> ba
> }' file1
This might work for you (GUN sed):
sed '0,/HAI/a\LINE ONE\nLINE TWO' file
or this (any sed):
sed 'x;/./{x;b};x;/HAI/!b;h;a\LINE ONE\nLINE TWO' file