Class: Kiba::Extend::Transforms::IdGenerator::YearBasedIncrementing

Inherits:
Object
  • Object
show all
Defined in:
lib/kiba/extend/transforms/id_generator/year_based_incrementing.rb

Overview

Generates unique ID values based on the year extracted from a given date field, where available. For each year extracted, the rows are sorted by the full date value, and the id number value is incremented.

The transform supports a multi-segment number pattern. All segments except for the year and incrementing digit(s) are optional. Set them as empty Strings when you set up the transform to omit them. The supported segments are:

  • prefix
  • 4-digit year (for rows with no date, the value given for nodateval will be inserted here)
  • preincrementsegment
  • autoincrementing digit

ASSUMPTIONS OF THIS TRANSFORM

  • You will have already reduced the output to one row per unique ID that needs to be created
  • You will have used DateValue::ForceDayPrecision or some other method to create a field containing only blank values and valid ISO 8601 dates, that will be set as the datefield param

SORTING, or, how the end digit gets incremented within a year

During the first pass over the data, we populate holder hash. Keys are the years extracted from the datefield values in all rows. Values are the rows from whom that year was extracted.

In the final pass, a sorter is applied to the Array of rows for each year key.

The default sorter:

  • parses the datefield value to create a sortable Ruby Date if a value is present; uses 0001-01-01 if not
  • sorts by this Ruby Date
  • if a number of rows have the same date, they’ll be in the same order as they appear in the data before this transform is applied

If your data contains full time stamps or some other sequential value you would like to use to sort, you can pass a custom sorter Lamda. This Lambda should take two positional arguments. The transform passes the Array of rows for the year key as the first argument, and the datefield attr value as the second. See the custom sorter example for how you can indicate the datefield attr won’t be used. It should return the Array of rows, sorted as desired.

Examples:

With default sorter

# Used in pipeline as:
# transform IdGenerator::YearBasedIncrementing,
#   datefield: :d,
#   target: :tada,
#   prefix: "FOO"
#
xform = IdGenerator::YearBasedIncrementing.new(
  datefield: :d,
  target: :tada,
  prefix: "FOO"
)
input = [
  {d: "2007-12-13", i: "a"},
  {d: "2000-05-23", i: "z"},
  {d: "2000-01-01", i: "c"},
  {d: "", i: "q"},
  {d: "2000-05-23", i: "g"},
  {d: "2007-10-31", i: "f"},
  {d: "2000-12-14", i: "h"},
  {d: "2000-05-23", i: "d"},
  {d: nil, i: "y"},
  {d: "2017-02-04", i: "x"}
]
result = Kiba::StreamingRunner.transform_stream(input, xform)
  .map{ |row| row }
expected = [
  {d: "2007-10-31", i: "f", tada: "FOO2007.1"},
  {d: "2007-12-13", i: "a", tada: "FOO2007.2"},
  {d: "2000-01-01", i: "c", tada: "FOO2000.1"},
  {d: "2000-05-23", i: "z", tada: "FOO2000.2"},
  {d: "2000-05-23", i: "g", tada: "FOO2000.3"},
  {d: "2000-05-23", i: "d", tada: "FOO2000.4"},
  {d: "2000-12-14", i: "h", tada: "FOO2000.5"},
  {d: "", i: "q", tada: "FOO0000.1"},
  {d: nil, i: "y", tada: "FOO0000.2"},
  {d: "2017-02-04", i: "x", tada: "FOO2017.1"}
]
expect(result).to eq(expected)

With custom sorter

# Used in pipeline as:
# transform IdGenerator::YearBasedIncrementing,
#   datefield: :d,
#   target: :tada,
#   prefix: "FOO",
#   preincrementsegment: ".1.",
#   sorter: ->(r, _na) do
#     r.sort_by { |row| row[:i] }
#   end
xform = IdGenerator::YearBasedIncrementing.new(
  datefield: :d,
  target: :tada,
  prefix: "FOO",
  preincrementsegment: ".1.",
  sorter: ->(r, _na) do
    r.sort_by { |row| row[:i] }
  end
)
input = [
  {d: "2007-12-13", i: "a"},
  {d: "2000-05-23", i: "z"},
  {d: "2000-01-01", i: "c"},
  {d: "", i: "q"},
  {d: "2000-05-23", i: "g"},
  {d: "2007-10-31", i: "f"},
  {d: "2000-12-14", i: "h"},
  {d: "2000-05-23", i: "d"},
  {d: nil, i: "y"},
  {d: "2017-02-04", i: "x"}
]
result = Kiba::StreamingRunner.transform_stream(input, xform)
  .map{ |row| row }
expected = [
  {d: "2007-12-13", i: "a", tada: "FOO2007.1.1"},
  {d: "2007-10-31", i: "f", tada: "FOO2007.1.2"},
  {d: "2000-01-01", i: "c", tada: "FOO2000.1.1"},
  {d: "2000-05-23", i: "d", tada: "FOO2000.1.2"},
  {d: "2000-05-23", i: "g", tada: "FOO2000.1.3"},
  {d: "2000-12-14", i: "h", tada: "FOO2000.1.4"},
  {d: "2000-05-23", i: "z", tada: "FOO2000.1.5"},
  {d: "", i: "q", tada: "FOO0000.1.1"},
  {d: nil, i: "y", tada: "FOO0000.1.2"},
  {d: "2017-02-04", i: "x", tada: "FOO2017.1.1"}
]
expect(result).to eq(expected)

Constant Summary collapse

DEFAULT_SORTER =

This sorter is used if no custom sorter is passed in when initializing the transform

->(rows, datefield) do
  rows.sort_by do |row|
    dateval = row[datefield]
    dateval.blank? ? Date.new(1, 1, 1) : Date.parse(dateval)
  end
end

Instance Method Summary collapse

Constructor Details

#initialize(datefield:, target:, prefix:, preincrementsegment: ".", nodateval: "0000", sorter: DEFAULT_SORTER) ⇒ YearBasedIncrementing

Returns a new instance of YearBasedIncrementing.

Parameters:

  • datefield (Symbol) —

    field containing valid ISO 8601 dates and blank values only

  • target (Symbol) —

    field in which to write the generated value

  • prefix (String) —

    initial segment of generated value

  • preincrementsegment (String) (defaults to: ".") —

    optional segment of number after the year or nodateval and before the incrementing segment

  • nodateval (String) (defaults to: "0000") —

    value inserted instead of the year, if datefield is blank

  • sorter (Lambda) (defaults to: DEFAULT_SORTER) —

    taking rows and datefield args, and returning sorted rows



160
161
162
163
164
165
166
167
168
169
170
171
172
173
 
# File 'lib/kiba/extend/transforms/id_generator/year_based_incrementing.rb', line 160

def initialize(datefield:,
  target:,
  prefix:,
  preincrementsegment: ".",
  nodateval: "0000",
  sorter: DEFAULT_SORTER)
  @datefield = datefield
  @target = target
  @prefix = prefix
  @preincrementsegment = preincrementsegment
  @nodateval = nodateval
  @sorter = sorter || DEFAULT_SORTER
  @holder = {}
end

Instance Method Details

#close ⇒ Object 



183
184
185
 
# File 'lib/kiba/extend/transforms/id_generator/year_based_incrementing.rb', line 183

def close
  finalized_rows.each { |row| yield row }
end

#finalized_rows ⇒ Array{Hash}

Returns Array of rows to be yielded by close method of using class.

Returns:

  • (Array{Hash}) —

    Array of rows to be yielded by close method of using class



189
190
191
192
 
# File 'lib/kiba/extend/transforms/id_generator/year_based_incrementing.rb', line 189

def finalized_rows
  holder.map { |year, data| generate_for_year(year, data) }
    .flatten
end

#process(row) ⇒ Object 



175
176
177
178
179
180
181
 
# File 'lib/kiba/extend/transforms/id_generator/year_based_incrementing.rb', line 175

def process(row)
  dateval = row[datefield]
  year = dateval.blank? ? nodateval : dateval[0..3]
  populate_holder(row, year)

  nil
end