Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly extract files (FamilyTree especially) #490

Open
2 tasks
philippbayer opened this issue Mar 27, 2018 · 4 comments
Open
2 tasks

Properly extract files (FamilyTree especially) #490

philippbayer opened this issue Mar 27, 2018 · 4 comments

Comments

@philippbayer
Copy link
Member

Right now the preparsing job uses the zip gem to unzip uploaded files, but uploaded files can come in a variety of flavours. FTDNA just updated their file format to be gz.

To do:

  • In the preparsing job, test more which file format the user has uploaded (zip, tar.gz, gz, flat text, rar?)
  • then use the right gem to unzip instead of going for default zip (probably need to add a few gems as it looks like there's no 'generic' ruby gem for all kinds of compressions)
@raivivek
Copy link
Member

raivivek commented Dec 3, 2018

Parsing some files in the data dump was not a smooth experience. There are file with names such as user7170_file5562_yearofbirth_unknown_sex_unknown.23andme.txt but are zipped in reality. I feel like the code checks for that so I'm not sure why this is happened.

@gedankenstuecke
Copy link
Member

Yep, this is a fix we really(tm) should get to. @philippbayer has already started working on this in #492 I think. @raivivek any chance you wanna pick up that PR and give it more testing etc? 😄

@raivivek
Copy link
Member

raivivek commented Dec 3, 2018

Sure! I'll be happy to help iron it out.

@gedankenstuecke
Copy link
Member

That would be great, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants