Split multiple CSV cell values from multiple columns using Ruby - ruby

Question

I'm trying to split multiple values in a CSV cell. I can do it right if the multiple values in a cell is found in a single column only, but I'm having difficulty doing it if the multiple values are found in multiple columns. Any guidance will be appreciated.
Here's the sample of the data I'm trying to split:
| Column A | Column B |
|Value1, Value2, Value3 | Value3, Value4, Value5 |
|Value6 | Value7, Value8 |
I'm aiming to have a result like this:
| Column A | Column B |
| Value1 | Value3 |
| Value2 | Value4 |
| Value3 | Value5 |
| Value6 | Value7 |
| Value6 | Value8 |
Here's my code:
require 'csv'
split_a = []
split_b = []
def split_values(value)
value = value.to_s
value = value.gsub('/', ',').gsub('|', ',').gsub(' ', ',')
return value.split(',').map(&:strip)
end
source_csv = kendo_shipment = CSV.read('source_file.csv', headers: true, header_converters: :symbol, liberal_parsing: true).map(&:to_h)
source_csv.each do |source_csv|
column_a = source_csv[:column_a]
column_b = source_csv[:column_b]
column_a = split_values(column_a)
column_a.each do |column_a|
next if column_a.nil? || column_a.empty?
split_a << [
column_a: column_a,
column_b: column_b
]
end
end
split_a.each do |key, split_a|
column_a = key[:column_a]
column_b = key[:column_b]
column_b = split_values(column_b)
column_b.each do |column_b|
next if column_b.nil? || column_b.empty?
split_b << [
column_a,
column_b
]
end
end

Answer 1

There is a special option to define a column separator col_sep: '|' it simplifies the code.
require 'csv'
source_csv = CSV.read('source_file.csv', col_sep: '|', headers: true, header_converters: :symbol, liberal_parsing: true)
split_a = []
split_b = []
# I just assign values to separate arrays, because I am not sure what kind of data structure you want to get at the end.
source_csv.each do |row|
split_a += row[:column_a].split(',').map(&:strip)
split_b += row[:column_b].split(',').map(&:strip)
end
# The result
split_a
# => ["Value1", "Value2", "Value3", "Value6"]
split_b
# => ["Value3", "Value4", "Value5", "Value7", "Value8"]

Answer 2

Here is the code:
require 'csv'
source_csv = CSV.read('source_file.csv',
col_sep: '|',
headers: true,
header_converters: lambda {|h| h.strip},
liberal_parsing: true
)
COLUMN_NAMES = ['Column A', 'Column B']
# Column values
columns = COLUMN_NAMES.map do |col_name|
source_csv&.map do |row|
row[col_name]&.split(',')&.map(&:strip)
end&.flatten(1) || []
end
# repeat the last value in case the number of values in the columns differs:
vals_last_id = columns.map {|col| col.count}.max - 1
columns.each do |col|
# replace `col.last` with `nil` on the next line if you want to leave the value blank
col.fill(col.last, col.length..vals_last_id) if col.length <= vals_last_id
end
values = columns[0].zip(*columns[1..-1])
# result:
pp values; 1
# [["Value1", "Value3"],
# ["Value2", "Value4"],
# ["Value3", "Value5"],
# ["Value6", "Value7"],
# ["Value6", "Value8"]]
Generate CSV text, with | (pipe) delimiter instead comma:
csv = CSV.new( '',
col_sep: '|',
headers: COLUMN_NAMES,
write_headers: true
);
values.each {|row| csv << row};
puts csv.string
# Column A|Column B
# Value1|Value3
# Value2|Value4
# Value3|Value5
# Value6|Value7
# Value6|Value8
Formatted output:
col_val = [COLUMN_NAMES] + values
col_widths = (0..(COLUMN_NAMES.count - 1)).map do |col_id|
col_val.map {|row| row[col_id]&.length || 0}.max
end
fmt = "|" + col_widths.map {|w| " %-#{w}s "}.join('|') + "|\n"
col_val.each {|row| printf fmt % row}; 1
# | Column A | Column B |
# | Value1 | Value3 |
# | Value2 | Value4 |
# | Value3 | Value5 |
# | Value6 | Value7 |
# | Value6 | Value8 |

Answer 3

As you want the output to be a CSV file I would suggest that it look like:
Column A|Column B
Value1|Value3
Value2|Value4
Value3|Value5
Value6|Value7
Value6|Value8
rather than
| Column A | Column B |
| Value1 | Value3 |
| Value2 | Value4 |
| Value3 | Value5 |
| Value6 | Value7 |
| Value6 | Value8 |
Enclosing each line with column separators and adding unnecessary spaces makes it unnecessarily difficult to extract the text of interest from the file.
Let's begin by creating the file you are given, though I have modified your example to make it easier to follow what is happening.
str=<<~_
| Column A | Column B |
| A1, A2, A3 | B1, B2, B3 |
| A4 | B4, B5 |
_
IN_NAME = 'in.csv'
OUT_NAME = 'out.csv'
File.write(IN_NAME, str)
#=> 84
See IO::write1.
As the structure of this file resembles a CSV file only vaguely, I think it's easiest to read it using ordinary file I/O methods.
header, *body = IO.foreach(IN_NAME, chomp: true).with_object([]) do |line,arr|
arr << line.gsub(/^\| *| *\|$/, '')
.split(/ *\| */)
.flat_map { |s| s.split(/, +/) }
end
(I provide an explanation of this calculation later.) This results in the following:
header
#=> ["Column A", "Column B"]
body
#=> [["A1", "A2", "A3", "B1", "B2", "B3"], ["A4", "B4", "B5"]]
See IO::foreach, Enumerator#with_object and Enumerable#flat_map. Note that foreach without a block returns an enumerator that I have chained to with_object.
At this point it is convenient to compute the number of rows to be written to the output file after the header row.
mx = body.map(&:size).max
#=> 6
Next we need to modify body to make it suitable for writing the output CSV file.
mod_body = Array.new(body.size) do |i|
Array.new(mx) { |j| body[i][j] || body[i].last }
end.transpose
#=> [["A1", "A4"], ["A2", "B4"], ["A3", "B5"], ["B1", "B5"],
# ["B2", "B5"], ["B3", "B5"]]
See Array::new.
It is now a simple matter to write the output CSV file.
require 'csv'
CSV.open(OUT_NAME, "wb", col_sep: '|', headers: header, write_headers: true) do |csv|
mod_body.each { |a| csv << a }
end
See CSV::open.
Lastly, let's look at the file that was written.
puts File.read(OUT_NAME)
diplays
Column A|Column B
A1|A4
A2|B4
A3|B5
B1|B5
B2|B5
B3|B5
See IO::read1.
To explain the calculations made in
header, *body = IO.foreach(IN_NAME, chomp: true).with_object([]) do |line,arr|
arr << line.gsub(/^\| *| *\|$/, '')
.split(/ *\| */)
.flat_map { |s| s.split(/, +/) }
end
it is easiest to run it with some puts statements inserted.
header, *body = IO.foreach(IN_NAME, chomp: true).with_object([]) do |line,arr|
puts "line = #{line}"
puts "arr = #{arr}"
arr << line.gsub(/^\| *| *\|$/, '')
.tap { |l| puts " line after gsub = #{l}" }
.split(/ *\| */)
.tap { |a| puts " array after split = #{a}" }
.flat_map { |s| s.split(/, +/) }
.tap { |a| puts " array after flat_map = #{a}" }
end
#=> [["Column A", "Column B"], ["A1", "A2", "A3", "B1", "B2", "B3"],
# ["A4", "B4", "B5"]]
The following is displayed.
line = | Column A | Column B |
arr = []
line after gsub = Column A | Column B
array after split = ["Column A", "Column B"]
array after flat_map = ["Column A", "Column B"]
line = | A1, A2, A3 | B1, B2, B3 |
arr = [["Column A", "Column B"]]
line after gsub = A1, A2, A3 | B1, B2, B3
array after split = ["A1, A2, A3", "B1, B2, B3"]
array after flat_map = ["A1", "A2", "A3", "B1", "B2", "B3"]
line = | A4 | B4, B5 |
arr = [["Column A", "Column B"], ["A1", "A2", "A3", "B1", "B2", "B3"]]
line after gsub = A4 | B4, B5
array after split = ["A4", "B4, B5"]
array after flat_map = ["A4", "B4", "B5"]
1. IO methods are commonly invoked on File. That is permissible since File.superclass #=> IO.