strip special characters out of header names. excel likes to leave odd unicode items, including the unicode bom, laying around. This causes havic. By stopping it right from the start we should prevent saving invisible characters to raw_metadata and other places they get stuck

Author: orangewolf

app/models/bulkrax/csv_entry.rb

@@ -16,11 +16,12 @@ def self.fields_from_data(data)
     class_attribute(:csv_read_data_options, default: {})
 
     # there's a risk that this reads the whole file into memory and could cause a memory leak
+    # we strip any special characters out of the headers. looking at you Excel
     def self.read_data(path)
       raise StandardError, 'CSV path empty' if path.blank?
       options = {
         headers: true,
-        header_converters: ->(h) { h.to_s.strip.to_sym },
+        header_converters: ->(h) { h.to_s.gsub(/[^\w\d -]+/, '').strip.to_sym },
         encoding: 'utf-8'
       }.merge(csv_read_data_options)