Class: Kiba::Extend::Utils::MultiSourceNormalizer Deprecated

Inherits:
Object
  • Object
show all
Defined in:
lib/kiba/extend/utils/multi_source_normalizer.rb

Overview

Deprecated.

Use Transforms::Clean::EnsureConsistentFields in multi-source jobs instead

Note:

This currently only works when using Kiba::Extend::Destinations::CSV destination. It depends on the fields method added to that class to support. This was not added to the Kiba::Extend::Destinations::JsonArray class because it does not require an identical field set in all records

Helper to make it less tedious to ensure the same fields are present in all rows included in a multi-source job. This is annoying to deal with when the source tables have different fields. It’s impossible to deal with without a helper if you are doing any of your jobs/transformations dynamically.

The basic idea is: - an new instance of this class is created somewhere accessible from within jobs. In Kiba::TMS this is a config settig per multi-source job: Kiba::Tms.config.name_compilation.multi_source_normalizer

  • pass this instance in as a helper on the MultiSourcePrepJobs that generate files that will be used as sources in the multisource job:
 Kiba::Extend::Jobs::MultiSourcePrepJob.new(
  files: {
    source: :prep__obj_locations,
    destination: :names__from_obj_locations
  },
  transformer: from_obj_locations_xforms,
  helper: Kiba::Tms.config.name_compilation.multi_source_normalizer
)

Finally, in the multisource job, call the get_fields method of your normalizer as the fields argument of an Append::NilFields transform:

transform Append::NilFields,
  fields: Tms.config.name_compilation.multi_source_normalizer.get_fields

Since:

  • 2.7.0

Instance Method Summary collapse

Constructor Details

#initialize ⇒ MultiSourceNormalizer

Returns a new instance of MultiSourceNormalizer.

Since:

  • 2.7.0



48
49
50
 
# File 'lib/kiba/extend/utils/multi_source_normalizer.rb', line 48

def initialize
  @fields = []
end

Instance Method Details

#get_fields ⇒ Array<Symbol>

Returns:

  • (Array<Symbol>)

Since:

  • 2.7.0



53
54
55
 
# File 'lib/kiba/extend/utils/multi_source_normalizer.rb', line 53

def get_fields
  fields.flatten.uniq.sort
end

#record_fields(new_fields) ⇒ Object

Parameters:

  • new_fields (Array<Symbol>)

Since:

  • 2.7.0



58
59
60
 
# File 'lib/kiba/extend/utils/multi_source_normalizer.rb', line 58

def record_fields(new_fields)
  fields << new_fields
end