Pattern matches by row name with multiple columns in multiple files - bash

Question

I have two files, one with a full list of gene names and three others with partial lists of gene names. I want to match these files all into one. All the partial files are different number of rows but have 3000 columns, all representing different cells. I have been trying to join these files completely but when I use awk only one column is kept.
mergedAll.txt
GENE
SOX2
BRCA1
BRCA2
RHO
ultimatecontrolMed.txt
GENE CELL1 CELL2 CELL3
SOX2 30 152 2000
BRCA2 400 234 73
RHO 12 2 0
My Desired output would be
GENE CELL1 CELL2 CELL3
SOX2 30 152 2000
BRCA1 0 0 0
BRCA2 400 234 73
RHO 12 2 0
I run:
awk 'NR==FNR{k[$1];next}{b[$1]=$0;k[$1]}
END{for(x in k)
if ( x== "GENE" )
printf"%s %s\n",x,b[x]
else
printf"%s %d\n",x,b[x]
}' mergedAll.txt ultimatecontrolMed.txt > test.txt
And I get:
GENE CELL1 CELL 2 CELL3
SOX2 2000
BRCA1 0
BRCA2 73
RHO 0
For some reason it will keep the last column of counts but not any of the other lines, and keep all the cell names. I don't have any experience with awk so this has been a major challenge for me overall and would love it if someone could offer a better solution.

Answer 1

awk to the rescue!
$ awk 'NR==FNR {a[$1]=$0; next}
{print (a[$1]?a[$1]:($1 FS 0 FS 0 FS 0))}' file2 file1 |
column -t
GENE CELL1 CELL2 CELL3
SOX2 30 152 2000
BRCA1 0 0 0
BRCA2 400 234 73
RHO 12 2 0
final pipe to column is for pretty printing. Note the order of the files.
Not to hard code the number of columns you can try this alternative
$ awk 'NR==1 {for(i=2;i<=NF;i++) missing=missing FS 0}
NR==FNR {a[$1]=$0; next}
{print (a[$1]?a[$1]:($1 missing))}' file2 file1

Answer 2

Could you please try following awk and let me know if this helps you.
awk 'FNR==NR{a[$0];next} ($1 in a){print;delete a[$1];next} END{for(i in a){print i,"0 0 0"}}' mergedAll.txt ultimatecontrolMed.txt

Answer 3

The problem is that you're printing b[x] with %d format. That's for printing a single integer, so it will ignore all the other integers in b[x]. Change
printf"%s %d\n",x,b[x]
to:
if (b[x]) {
printf "%s\t%s\n", x, b[x]
} else {
printf "%s" x;
for (i = 0; i < 3000; i++) printf "\t0"
print ""
}
so that it will print the entire value. If there's no corresponding value, it will print zeroes.
Replace 3000 with the appropriate number of cells. If you don't want to hard-code it, you can get it from NF-1 when FNR == 1 && FNR != NR (the first line of the second file).

Answer 4

join -a 1 -a 2 -e 0 -o 0 2.{2..4} mergedAll.txt ultimatecontrolMed.txt
2.{2..4} prints a list of output fields and can easily adapted to any number of fields.
As you mention three input files, it would be possible to pipe the result of a first join into a second one
join .... file1 file2 | join ... file3
join needs sorted input. That may be a killer argument for this solution.