Class: Kiba::Extend::Utils::MultiSourceNormalizer

Inherits:
Object
  • Object
show all
Defined in:
lib/kiba/extend/utils/multi_source_normalizer.rb

Overview

Note:

This currently only works when using Kiba::Extend::Destinations::CSV destination. It depends on the fields method added to that class to support. This was not added to the Kiba::Extend::Destinations::JsonArray class because it does not require an identical field set in all records

Helper to make it less tedious to ensure the same fields are present in all rows included in a multi-source job. This is annoying to deal with when the source tables have different fields. It’s impossible to deal with without a helper if you are doing any of your jobs/transformations dynamically.

The basic idea is: - an new instance of this class is created somewhere accessible from within jobs. In Kiba::TMS this is a config settig per multi-source job: Kiba::Tms.config.name_compilation.multi_source_normalizer

  • pass this instance in as a helper on the MultiSourcePrepJobs that generate files that will be used as sources in the multisource job:
 Kiba::Extend::Jobs::MultiSourcePrepJob.new(
  files: {
    source: :prep__obj_locations,
    destination: :names__from_obj_locations
  },
  transformer: from_obj_locations_xforms,
  helper: Kiba::Tms.config.name_compilation.multi_source_normalizer
)

Finally, in the multisource job, call the get_fields method of your normalizer as the fields argument of an Append::NilFields transform:

transform Append::NilFields, fields: Tms.config.name_compilation.multi_source_normalizer.get_fields

Since:

  • 2.7.0

Instance Method Summary collapse

Constructor Details

#initialize ⇒ MultiSourceNormalizer

Returns a new instance of MultiSourceNormalizer.

Since:

  • 2.7.0



43
44
45
 
# File 'lib/kiba/extend/utils/multi_source_normalizer.rb', line 43

def initialize
  @fields = []
end

Instance Method Details

#get_fields ⇒ Array<Symbol>

Returns:

  • (Array<Symbol>)

Since:

  • 2.7.0



48
49
50
 
# File 'lib/kiba/extend/utils/multi_source_normalizer.rb', line 48

def get_fields
  fields.flatten.uniq.sort
end

#record_fields(new_fields) ⇒ Object

Parameters:

  • new_fields (Array<Symbol>)

Since:

  • 2.7.0



53
54
55
 
# File 'lib/kiba/extend/utils/multi_source_normalizer.rb', line 53

def record_fields(new_fields)
  fields << new_fields
end