Class: Kiba::Extend::Transforms::Explode::ColumnsRemappedInNewRows

Inherits:
Object
  • Object
show all
Defined in:
lib/kiba/extend/transforms/explode/columns_remapped_in_new_rows.rb

Overview

This one is hard to succintly describe! Use it if you have specific fields grouped together in a row, and you want to abstract them so the fields are broader, and you reduce the number of fields.

In the example, 3 fields in each row represent fruit names and 2 fields in each row represent colors. We want more rows, but each row should have only one fruit column and color column.

Things to notice in example:

  • The fact that some field values are multivalued is completely ignored
  • If all the values for a given remap group are blank, no row is added
  • Values in fields not included in remap_groups are copied to every row created

Examples:

With defaults

# Used in pipeline as:
# transform Explode::ColumnsRemappedInNewRows,
#   remap_groups: [
#     [:f1, :c1],
#     [:f2, :c2],
#     [:f3]
#   ],
#   map_to: [:fruit, :color]
xform = Explode::ColumnsRemappedInNewRows.new(
  remap_groups: [
    [:f1, :c1],
    [:f2, :c2],
    [:f3]
  ],
  map_to: [:fruit, :color]
)
input = [
  {f1: "strawberry", c1: "red", f2: "blueberry", c2: "blue",
    season: "spring", f3: "cherry"},
  {f1: "fig;honeydew", c1: "brown;green", f2: "watermelon",
    c2: "green", season: "summer", f3: nil},
  {f1: nil, c2: nil, f2: nil, c2: nil, season: "winter",
    f3: "grapefruit"},
  {f1: nil, c2: nil, f2: nil, c2: nil, season: "autumn",
    f3: nil}
]
result = Kiba::StreamingRunner.transform_stream(input, xform)
  .map{ |row| row }
expected = [
  {fruit: "strawberry", color: "red", season: "spring"},
  {fruit: "blueberry", color: "blue", season: "spring"},
  {fruit: "cherry", color: nil, season: "spring"},
  {fruit: "fig;honeydew", color: "brown;green", season: "summer"},
  {fruit: "watermelon", color: "green", season: "summer"},
  {fruit: "grapefruit", color: nil, season: "winter"},
  {fruit: nil, color: nil, season: "autumn"}
]
expect(result).to eq(expected)

Instance Method Summary collapse

Constructor Details

#initialize(remap_groups:, map_to:) ⇒ ColumnsRemappedInNewRows

Returns a new instance of ColumnsRemappedInNewRows.

Parameters:

  • remap_groups (Array(Array(Symbol))) —

    The existing field groups that should be mapped into the map_to fields

  • map_to (Array<Symbol>) —

    target fields derived from remap_groups



69
70
71
72
 
# File 'lib/kiba/extend/transforms/explode/columns_remapped_in_new_rows.rb', line 69

def initialize(remap_groups:, map_to:)
  @groups = remap_groups
  @map = map_to
end

Instance Method Details

#process(row) ⇒ Object

Parameters:

  • row (Hash{ Symbol => String, nil }) 


75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
 
# File 'lib/kiba/extend/transforms/explode/columns_remapped_in_new_rows.rb', line 75

def process(row)
  to_new_rows = new_row_groups(row)
  if to_new_rows.empty?
    newrow = @map.map do |field|
      [field, nil]
    end.to_h.merge(other_fields(row))
    yield(newrow)
  else
    to_new_rows.each do |grp_data|
      newrow = @map.zip(grp_data).to_h.merge(other_fields(row))
      yield(newrow)
    end
  end
  nil
end