Join two CSV files in Ruby without using tables - ruby

Question

I have 2 CSV files with columns like A, B, C.. & D, E, F. I want to join these two CSV files into a new file with rows where File1.B = File2.E and the row would have columns A, B/E, C, D, F. How can I achieve this JOIN without using tables?

Answer 1

Givens
We are given the following.
The paths for the two input files:
fname1 = 't1.csv'
fname2 = 't2.csv'
The path for the output file:
fname3 = 't3.csv'
The names of the headers to match in each of the two input files:
target1 = 'B'
target2 = 'E'
I do assume that (as is the case with the example) the two files necessarily contain the same number of lines.
Create test files
Let's first create the two files:
str = [%w|A B C|, %w|1 1 1|, %w|2 2 2|, %w|3 4 5|, %w|6 9 9|].
map { |a| a.join(",") }.join("\n")
#=> "A,B,C\n1,1,1\n2,2,2\n3,4,5\n6,9,9"
File.write(fname1, str)
#=> 29
str = [%w|D E F|, %w|21 1 41|, %w|22 5 42|, %w|23 8 45|, %w|26 9 239|].
map { |a| a.join(",") }.join("\n")
#=> "D,E,F\n21,1,41\n22,5,42\n23,8,45\n26,9,239"
File.write(fname2, str)
#=> 38
Read the input files into CSV::Table objects
When reading fname1 I will use the :header_converters option to convert the header "B" to "B/E". Note that this does not require knowledge of the location of the column with header "B" (or whatever it may be).
require 'csv'
new_target1 = target1 + "/" + target2
#=> "B/E"
csv1 = CSV.read(fname1, headers: true,
header_converters: lambda { |header| header==target1 ? new_target1 : header})
csv2 = CSV.read(fname2, headers: true)
Construct arrays of headers to be written from each input file
headers1 = csv1.headers
#=> ["A", "B/E", "C"]
headers2 = csv2.headers - [target2]
#=> ["D", "F"]
Create the output file
We will first write the new headers headers1 + headers2 to the output file.
Next, for each row index i (i = 0 corresponding to the first row after the header row in each file), for which a condition is satisfied, we write as a single row the elements of csv1[i] and csv2[i] that are in the columns having headers in headers1 and headers2. The condition to be satisfied to write the rows at index i is that i satisfies:
csv1[i][new_target1] == csv2[i][target2] #=> true
Now open fname3 for writing, write the headers and then the body.
CSV.open(fname3, 'w') do |csv|
csv << headers1 + headers2
[csv1.size, csv2.size].min.times do |i|
csv << (headers1.map { |h| csv1[i][h] } +
headers2.map { |h| csv2[i][h] }) if
csv1[i][new_target1] == csv2[i][target2]
end
end
#=> 4
Let's confirm that what was written is correct.
puts File.read(fname3)
A,B/E,C,D,F
1,1,1,21,41
6,9,9,26,239

Answer 2

If you have CSV files like these:
first.csv:
A | B | C
1 | 1 | 1
2 | 2 | 2
3 | 4 | 5
6 | 9 | 9
second.csv:
D | E | F
21 | 1 | 41
22 | 5 | 42
23 | 8 | 45
26 | 9 | 239
You can do something like this:
require 'csv'
first = CSV.read('first.csv')
second = CSV.read('second.csv')
CSV.open("result.csv", "w") do |csv|
csv << %w[A B.E C D F]
first.each do |rowF|
second.each do |rowS|
csv << [rowF[0],rowF[1],rowF[2],rowS[0],rowS[2]] if rowF[1] == rowS[1]
end
end
end
To get this:
result.csv:
A | B.E | C | D | F
1 | 1 | 1 | 21 | 41
6 | 9 | 9 | 26 | 239

Answer 3

The answer is to use group by to create a hash table and then iterate over the keys of the hash table. Assuming the column you're joining on is unique in each table:
join_column = "whatever"
csv1 = CSV.table("file1.csv").group_by { |r| r[join_column] }
csv2 = CSV.table("file2.csv").group_by { |r| r[join_column] }
joined_data = csv1.keys.sort.map do |join_column_values|
csv1[join_column].first.merge(csv2[join_column].first)
end
If the column is not unique in each table, then you need to decide how you want to handle those cases since there will be more than just the first element in the arrays csv1[join_column] and csv2[join_column]. You could do an O(mxn) join as suggested in one of the other answers (i.e. nested map calls), or you could filter or combine them based on some criteria. The choice really depends on your usecase.