Skip to content

Commit

Permalink
Allow multiple metadata spreadsheets
Browse files Browse the repository at this point in the history
  • Loading branch information
lfarrell committed May 26, 2021
1 parent 0c39d6d commit 5206159
Showing 1 changed file with 12 additions and 9 deletions.
21 changes: 12 additions & 9 deletions app/services/tasks/onescience_ingest_service.rb
Original file line number Diff line number Diff line change
Expand Up @@ -211,16 +211,19 @@ def load_data
puts "[#{Time.now}] loaded embargo mappings"

# read from xlsx in projects folder
workbook = Roo::Spreadsheet.open(File.join(@config['metadata_dir'], @config['metadata_file']))
sheets = workbook.sheets
@data = []
sheets.each do |sheet|
if sheet.match('1foldr_UNCCH_01_Part')
data_hash = workbook.sheet(sheet).parse(headers: true)
data_hash.delete_if{|hash| hash['onescience_id'].blank? }
# first hash is of headers
data_hash.delete_at(0)
@data << data_hash
Array.wrap(@config['metadata_file']).each do|metadata_file|
workbook = Roo::Spreadsheet.open(File.join(@config['metadata_dir'], metadata_file))
sheets = workbook.sheets

sheets.each do |sheet|
if sheet.match('1foldr_UNCCH_01_Part')
data_hash = workbook.sheet(sheet).parse(headers: true)
data_hash.delete_if{|hash| hash['onescience_id'].blank? }
# first hash is of headers
data_hash.delete_at(0)
@data << data_hash
end
end
end
@data.flatten!
Expand Down

0 comments on commit 5206159

Please sign in to comment.