Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PsyR: An R package for better inference in multivariate statistical analysis #12

Closed
2 tasks done
garner-code opened this issue Nov 1, 2024 · 4 comments
Closed
2 tasks done

Comments

@garner-code
Copy link

garner-code commented Nov 1, 2024

Title

PsyR: An R package for better inference in multivariate statistical analysis

Leaders

Kelly Garner (Mattermost: kels)

Collaborators

Christopher Nolan (Mattermost: cnolan)

Brainhack Global 2024 Event

Brainhack Aus

Project Description

We consistently use massive data sets across neuroscience and psychology. The routine gathering of big data requires that we are well equipped with tools that allow us to conduct appropriate multivariate statistics.

Multivariate statistical analysis typically follows a two stage procedure, an omnibus test of the global null hypothesis followed by post-hoc tests of specific effects. It is not well known that under certain circumstances this leads to a drastically inflated rate of type 1 error. It is even less well known that this procedure can also lead to an even lessor known type IV error (incorrect interpretation of a correctly rejected hypothesis)!

It is possible to avoid these dragons by using an alternative procedure where all inferences are derived from simultaneous confidence intervals (SCIs) on contrasts of interest. This approach provides interval inferences on effect sizes and it also ensures that the familywise type 1 error rate associated with directional inferences (the inferences usually derived from tests of null hypotheses) is controlled at alpha. One piece of software (PSY) can produce SCIs appropriate for planned analyses (where contrasts are defined independently of the data) and for more flexible analyses where contrasts are defined on a post hoc basis. However, this software is only available for use on windows and cannot be scripted into reproducible workflows.

We have been building an R package that implements the functions of PSY, and to make this method of statistical inference available to the masses! We are ready for you to play with some of the functions to analyse your own data, and create Quarto documents that document your workflow, to help future users. If you are feeling hacky enough, we could also use help developing further the R package, which is around 80% finished right now.

Link to project repository/sources

https://github.com/kel-github/PSY2R - this has all the code we used to get started. I'll share the package code at the event
https://doi.org/10.1177/0013164402062002001 - this is the paper that defines the functionality of Psy.
https://r-pkgs.org/ - Hadley Wickham's bible for building R packages

Goals for Brainhack Global

Bring your own data and use the functions we have already written. This will help us learn how user friendly they are and what can be done to improve interaction with the function.
Document own data analysis in quarto documents to help future users
Contribute to the R package, particularly with higher level functions that handle data wrangling and standardisation
Build a GitHub website for the package

Good first issues

  1. issue one: read [this paper](https://doi.org/10.1177/0013164402062002001 to get an overview of what Psy does)

  2. issue two: find a dataset that requires an ANOVA analysis, share it with the team and we'll talk you through how to use Psy to analyse it

  3. issue three: If not already, get comfortable with quarto documents - https://quarto.org/

  4. issue four: Go through the exercises that show you how to create an R package in chapter 1 of this book.

Communication channels

https://mattermost.brainhack.org/brainhack/channels/psy2r

Skills

  • R: basic and advanced
  • Stats knowledge: can run an ANOVA and know what a confidence interval is
  • Github: can clone and make pull requests (if needed, we'll show you how)

Onboarding documentation

No response

What will participants learn?

Participants will learn about multivariate data analysis:

  • how to compute confidence intervals for between x within effects, ideally on their own data
  • what goes under the hood of an R package

Data to use

No response

Number of collaborators

4

Credit to collaborators

Project contributors will be listed in the project ReadMe, and as authors on any help documents

Image

Leave this text if you don't have an image yet.

Type

coding_methods, documentation, pipeline_development

Development status

1_basic structure

Topic

statistical_modelling

Tools

other

Programming language

R

Modalities

behavioral, eye_tracking

Git skills

1_commit_push, 2_branches_PRs

Anything else?

No response

Things to do after the project is submitted and ready to review.

  • Add a comment below the main post of your issue saying: Hi @brainhackorg/project-monitors my project is ready!
  • Twitter-sized summary of your project pitch.
@crnolan
Copy link
Collaborator

crnolan commented Nov 1, 2024

Your Mattermost handles are being interpreted as github handles on here...

@garner-code
Copy link
Author

I changed it :)

@crnolan
Copy link
Collaborator

crnolan commented Nov 1, 2024

You ticked the boxes without making the post :) I assume this is ready?

@garner-code
Copy link
Author

garner-code commented Nov 1, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment