Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Il gwascat dag #271

Closed
wants to merge 17 commits into from
Closed

Il gwascat dag #271

wants to merge 17 commits into from

Conversation

ireneisdoomed
Copy link
Contributor

@ireneisdoomed ireneisdoomed commented Nov 24, 2023

This PR defines the tasks that need to happen to process GWASCatalog data in a DAG. The process starts with the ingested harmonized summary stats flat files and finishes with the definition of credible sets.

It is built on top of 'do_clump_step'. It is WIP, as I don't love the current solution of splitting both clumping methods into different steps.
Writing the studies before splitting is terrible as well.
I prototyped a combined clumping step in ce28e04 but I don't like it either. I need to think about it.

@d0choa
Copy link
Collaborator

d0choa commented Nov 24, 2023

There is a bunch of potential clashes with other PRs. Let's agree on the steps we want to have. Treat them as individual PRs and then create the separate DAGs as independent PRs.

Once all the steps have been merged, we can see if the DAGs are still working before merging.

Step-wise, we either need to:

  • split the 2 clumps in 2 steps
  • treat gwas catalog curation and gwas catalog SS as different streams with some but not many repeated calls to similar functions.

I'm more inclined for the second because we don't need to write useless temporary files (e.g. window-based clumped study-locus before LD clumping)

@ireneisdoomed
Copy link
Contributor Author

Superseded by #291 and #288

@ireneisdoomed ireneisdoomed deleted the il-gwascat-dag branch December 1, 2023 16:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants