-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/ebi search dump migration #332
Commits on Jul 12, 2023
-
Initial port of the analysis XML dump generation for EBI Search.
This is a WIP, but the analysis should be nearly done. I've moved the logic for "Runs" of EBI Search Dump: - Code that reads the analysis data from the DB and flat files -> https://github.com/EBI-Metagenomics/MetagenomicsSearchDump/blob/master/src/input/metagenomicsDB.py#L191 - I've replaced the file reading bits with calls to Mongo / MySQL. Which makes the code independent from the filesystem. - Code that generates the xml output file for **one** analysis: https://github.com/EBI-Metagenomics/MetagenomicsSearchDump/blob/master/src/output/run.py We need to port the Samples and Studies; and the analysis aggregation. The aggregation is required to merge the analysis into XML bundles that EBI Search ingest. AnalysesAggregation script -> https://github.com/EBI-Metagenomics/MetagenomicsSearchDump/blob/master/src/RunEntryAggregator.py Another task: Is to add a indexed_at field on Analyses, Samples and Studies. This is going to be useful to have incremental dump generation, instead of having to re-generate all the files. Moving to this will also required the modification of the AnalysesAggregation script OR for us to move to EBI Search incremental dumps (https://www.ebi.ac.uk/seqdb/confluence/display/EXT/Preparation+for+incremental+indexing) Last one. EBI Search also supports JSONs (https://www.ebi.ac.uk/seqdb/confluence/display/EXT/JSON+data+format)
Configuration menu - View commit details
-
Copy full SHA for 651a8bb - Browse repository at this point
Copy the full SHA 651a8bbView commit details
Commits on Sep 25, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 6f6c070 - Browse repository at this point
Copy the full SHA 6f6c070View commit details
Commits on Oct 20, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 2b02f66 - Browse repository at this point
Copy the full SHA 2b02f66View commit details -
Configuration menu - View commit details
-
Copy full SHA for 974371f - Browse repository at this point
Copy the full SHA 974371fView commit details
Commits on Oct 23, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 14f0842 - Browse repository at this point
Copy the full SHA 14f0842View commit details
Commits on Oct 24, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 1a3acfd - Browse repository at this point
Copy the full SHA 1a3acfdView commit details
Commits on Nov 10, 2023
-
Configuration menu - View commit details
-
Copy full SHA for afa4cb5 - Browse repository at this point
Copy the full SHA afa4cb5View commit details -
Configuration menu - View commit details
-
Copy full SHA for d914ec1 - Browse repository at this point
Copy the full SHA d914ec1View commit details -
Configuration menu - View commit details
-
Copy full SHA for ca360e0 - Browse repository at this point
Copy the full SHA ca360e0View commit details
Commits on Nov 13, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 2827c2b - Browse repository at this point
Copy the full SHA 2827c2bView commit details
Commits on Nov 15, 2023
-
Configuration menu - View commit details
-
Copy full SHA for d18573c - Browse repository at this point
Copy the full SHA d18573cView commit details
Commits on Nov 16, 2023
-
Configuration menu - View commit details
-
Copy full SHA for 28908ca - Browse repository at this point
Copy the full SHA 28908caView commit details
Commits on Jan 3, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 5852201 - Browse repository at this point
Copy the full SHA 5852201View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6b40098 - Browse repository at this point
Copy the full SHA 6b40098View commit details -
Merge branch 'develop' into feature/ebi-search-dump-migration
# Conflicts: # emgcli/__init__.py # pyproject.toml
Configuration menu - View commit details
-
Copy full SHA for 480d951 - Browse repository at this point
Copy the full SHA 480d951View commit details -
Configuration menu - View commit details
-
Copy full SHA for d185d4e - Browse repository at this point
Copy the full SHA d185d4eView commit details
Commits on Jan 5, 2024
-
Configuration menu - View commit details
-
Copy full SHA for f36021e - Browse repository at this point
Copy the full SHA f36021eView commit details
Commits on Jan 16, 2024
-
Merge branch 'develop' into feature/ebi-search-dump-migration
# Conflicts: # emgapi/models.py # emgcli/__init__.py # pyproject.toml
Configuration menu - View commit details
-
Copy full SHA for 5eb0fc7 - Browse repository at this point
Copy the full SHA 5eb0fc7View commit details -
Merge branch 'develop' into feature/ebi-search-dump-migration
# Conflicts: # emgcli/__init__.py # pyproject.toml
Configuration menu - View commit details
-
Copy full SHA for 57a391b - Browse repository at this point
Copy the full SHA 57a391bView commit details
Commits on Jan 17, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 67edbcd - Browse repository at this point
Copy the full SHA 67edbcdView commit details -
Configuration menu - View commit details
-
Copy full SHA for 0e21458 - Browse repository at this point
Copy the full SHA 0e21458View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0a8f872 - Browse repository at this point
Copy the full SHA 0a8f872View commit details
Commits on Jan 22, 2024
-
materialise (fetch) lazyily referenced mongo embeds when accessing re…
…lated attributes
Configuration menu - View commit details
-
Copy full SHA for e10fd33 - Browse repository at this point
Copy the full SHA e10fd33View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0c8c61c - Browse repository at this point
Copy the full SHA 0c8c61cView commit details -
Configuration menu - View commit details
-
Copy full SHA for a036402 - Browse repository at this point
Copy the full SHA a036402View commit details -
Configuration menu - View commit details
-
Copy full SHA for 4fd2f3d - Browse repository at this point
Copy the full SHA 4fd2f3dView commit details -
Configuration menu - View commit details
-
Copy full SHA for de3b415 - Browse repository at this point
Copy the full SHA de3b415View commit details -
Configuration menu - View commit details
-
Copy full SHA for d13789e - Browse repository at this point
Copy the full SHA d13789eView commit details -
Configuration menu - View commit details
-
Copy full SHA for d23c962 - Browse repository at this point
Copy the full SHA d23c962View commit details
Commits on Jan 29, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 8140ef5 - Browse repository at this point
Copy the full SHA 8140ef5View commit details