Class: Kiba::Extend::Transforms::Delete::EmptyFields

Inherits:
Object
  • Object
show all
Defined in:
lib/kiba/extend/transforms/delete/empty_fields.rb

Overview

Note:

This transform runs in memory, so for very large sources, it may take a long time or fail.

Removes fields/columns that contain no values. Supports treating Kiba::Extend.nullvalue as an empty value. Also supports specifying field-specific values that should be treated as though they are empty.

Examples

The examples demonstrating usenull assume Kiba::Extend.nullvalue is set to %NULLVALUE%.

Basic usage

Used in pipeline as:

transform Delete::EmptyFields

Input table:

| a   | b   | c   | d   |
|-----+-----+-----+-----|
| a   |     | ccc |     |
|     | nil | c   | nil |
| nil |     | ccc |     |
| a   |     |     |     |

Results in:

| a   | c   |
|-----+-----|
| a   | ccc |
|     | c   |
| nil | ccc |
| a   |     |

Notes

Empty strings and nil values are treated as empty by default.

With usenull true

Used in pipeline as:

transform Delete::EmptyFields, usenull: true

Input table:

| a   | b   | c   | d   | e           |
|-----+-----+-----+-----+-------------|
|     | nil | c   | nil | %NULLVALUE% |
| a   |     | ccc |     |             |
| nil |     | ccc |     | %NULLVALUE% |
| a   |     |     |     |             |

Results in:

| a   | c   |
|-----+-----|
|     | c   |
| a   | ccc |
| nil | ccc |
| a   |     |

With consider_blank config given

Used in pipeline as:

transform Delete::EmptyFields, consider_blank: {b: 'false', c: 'nope', e: "0#{Kiba::Extend.delim}false"}

Input table:

| a   | b     | c           | d   | e     |
|-----+-------+-------------+-----+-------|
|     | nil   |             | nil | 0     |
| a   |       | %NULLVALUE% |     | false |
| nil | false | nope        |     | 0     |
| a   |       | nil         |     |       |

Results in:

| a   | c           |
|-----+-------------|
|     |             |
| a   | %NULLVALUE% |
| nil | nope        |
| a   | nil         |

Notes

Field c is retained because usenull: true is not used. If that argument were given, only Field a would be returned.

Instance Method Summary collapse

Constructor Details

#initialize(usenull: false, consider_blank: nil) ⇒ EmptyFields

Returns a new instance of EmptyFields.

Parameters:

  • usenull (Boolean) (defaults to: false) —

    whether to treat Kiba::Extend.nullvalue as empty/blank value

  • consider_blank (Hash{Symbol=>Array<String>}) (defaults to: nil) —

    specifies field-specific value(s) that should be treated as blank/empty. If multiple values should be considered blank for one field, join them using Kiba::Extend.delimiter



118
119
120
121
122
123
124
125
 
# File 'lib/kiba/extend/transforms/delete/empty_fields.rb', line 118

def initialize(usenull: false, consider_blank: nil)
  @usenull = usenull
  @consider_blank = consider_blank&.transform_values do |val|
    val.split(Kiba::Extend.delim)
  end
  @pop_fields = {}
  @rows = []
end

Instance Method Details

#close ⇒ Object 



133
134
135
136
137
138
139
140
141
 
# File 'lib/kiba/extend/transforms/delete/empty_fields.rb', line 133

def close
  return if rows.empty?

  to_delete = rows.first.keys - pop_fields.keys
  rows.each do |row|
    to_delete.each { |field| row.delete(field) }
    yield row
  end
end

#process(row) ⇒ Object

Parameters:

  • row (Hash{ Symbol => String, nil }) 


128
129
130
131
 
# File 'lib/kiba/extend/transforms/delete/empty_fields.rb', line 128

def process(row)
  populate_tracker(row)
  nil
end