dataset: add RaCCooNS by Frank and Aumeistere

[SiQube]

resolves #954

requires #989

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (af69b1c) to head (155b572).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #961   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           87        88    +1     
  Lines         3720      3742   +22     
  Branches       638       638           
=========================================
+ Hits          3720      3742   +22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[SiQube]

there is additional raw gaze data. unfortunately, the files are ascii and we currently only support csv, tsv and feather data, see #962

[dkrako]

Isn't this just documentation related? ToyDatasetEyeLink contains only ascii data and you should be able to load it without issues.

[SiQube]

yes, but we have to have the right parsing criterions. I'd want to do this in a seperate PR and reopen an issue. maybe someone from the original authors can help?

[dkrako]

Ah I see, it's not the EyeLink format then? can you provide some example lines?

[SiQube]

the files are actually eyelink ascii files. however I am not entirely sure how trials are split and I don't want to propagate mistakes. maybe we can setup a meeting with Stefan Frank where we can discuss how the trials were split? CC @izaskr

[izaskr]

Alright, let me check. Will get back to you with this.

[SiQube]

from @izaskr:

There's a variable TRIAL_INDEX that is increased by 1 at the end of each trial.
If you look for this in the text files you find the separations between consecutive trial.
I hope that answers David's question!

I'll implement it and then we can merge the dataset to pymovements.

[saeub]

@SiQube I'll take care of the ASCII parsing (hopefully today)

[saeub]

I see three problems:

1. Participant IDs in TSV files are numbers, but not all participant IDs in ASC filenames are in a number format (e.g. 001_2).
2. Trial IDs in the ASC files are offset by 1 compared to the TSV files, meaning the user won't be able to join the event and gaze frames.
3. Trial variable messages (!V TRIAL_VAR) are at the end of the trial, meaning we can't add info like stimulus ID to the gaze dataframe.

Regarding 2., I guess we can't really fix that right now? @SiQube (One option would be to allow passing functions with patterns in from_asc(), where a function takes the match.groupdict() and returns a dictionary of column values; but this would not be possible in a YAML definition, right?) For the moment, I just used a different column name trial_index0 in the gaze frame to make clear that it's a 0-based index.

For 3., I created an issue #990.

[SiQube]

maybe we can discuss with Frank and @izaskr ?

[dkrako]

Regarding 2., I guess we can't really fix that right now? @SiQube (One option would be to allow passing functions with patterns in from_asc(), where a function takes the match.groupdict() and returns a dictionary of column values; but this would not be possible in a YAML definition, right?) For the moment, I just used a different column name trial_index0 in the gaze frame to make clear that it's a 0-based index.

I have the feeling that we will probably need some custom loading functions for specific datasets that we want to add in the future. This way we can stay flexible for cases like this where we need to postprocess data.

This is a bit of a bummer but we really can't avoid that as datasets vary in their data standards.
We can still keep DatasetDefinition classes for such cases. There we can add a custom_post_processing() method or so.
This way #914 wouldn't be blocked.

Also #352 (adding mat-file support) could benefit from custom pre/post processing functions.

[saeub]

@SiQube Should we wait with merging this until we have a solution for parsing trial variables (#990) and custom postprocessing functions (#961 (comment))? Or do you want to merge this now and improve it later?

(I think the dataset is perfectly usable as it is now, just some of the information about the stimulus is missing, and the gaze and precomputed event frames don't match up.)

[SiQube]

I don't mind merging it asap and fix it later => we can move one of the comments to a new issue. using pymovements for this dataset is still valuable (I think) since downloading and preprocessing works (for most of the data)

[dkrako]

I'm in favor of merging this PR without the additional trial infos and improve on this later on when we have solved the underlying issues.

Also, adding a custom_post_processing() to a DatasetDefinition wouldn't help much when a user simply uses gaze.from_asc() to load a single file. #997 could help with this though.

[dkrako]

We should probably also mention these issues in the docstring of the dataset.

[dkrako]

Alright, lets merge this.

I just have two comments to make the classes clickable in the documentation (following #986).

@SiQube you can resolve the comments without changes in code if you like, as these probably aren't relevant anymore after merging #914

[dkrako]

can you add a yaml definition?

[SiQube]

@izaskr can you check with stefan why the dataset is down?
@izaskr in the paper its written 1920x1018 pixel screen resolution, I assume its a type and it should be 1920x1080

[saeub]

@SiQube I just noticed that I forgot to add trial_index0 to the trial columns here. Although I'm not sure we should add it actually, because it only exists in the gaze dataframe, but not in the event dataframe?

[saeub]

Maybe trial_columns="trial_index0" should only be added to the custom_read_kwargs for gaze?

[saeub]

Also, the parser is getting conflicting information regarding the eye tracker version:

• The ASC files say it's EYELINK II CL v4.594 Jul 6 2012, which is translated to EyeLink 1000 in our parsing logic:

  pymovements/src/pymovements/gaze/_utils/parsing.py

  Lines 508 to 509 in 913c372

  	elif float(version_number) < 5:
  	model = 'EyeLink 1000'

• The experiment metadata (and the paper) says it should be EyeLink 1000 Plus.

Is anyone familiar with these version mappings? (maybe @theDebbister, since she wrote the metadata parsing code?)

[theDebbister]

Regarding the version numbers, they are explained here at the bottom of the answer thread: https://www.sr-research.com/support/thread-8853.html

Which would basically mean that the metadata is wrong? Because the version is 4.x right? I will quickly ask Stefan to be sure.

Update: Stefan will look into it

[SiQube]

@izaskr @theDebbister any update?

[theDebbister]

Yes, Stefan said it is possible that the paper is wrong and they used the EyeLink 1000 instead of 1000 Plus. It seems that our device detection is still correct.