Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/ebi search dump migration #332

Merged
merged 30 commits into from
Jan 29, 2024

Commits on Jul 12, 2023

  1. Initial port of the analysis XML dump generation for EBI Search.

    This is a WIP, but the analysis should be nearly done.
    
    I've moved the logic for "Runs" of EBI Search Dump:
    - Code that reads the analysis data from the DB and flat files -> https://github.com/EBI-Metagenomics/MetagenomicsSearchDump/blob/master/src/input/metagenomicsDB.py#L191
      - I've replaced the file reading bits with calls to Mongo / MySQL.  Which
        makes the code independent from the filesystem.
    - Code that generates the xml output file for **one** analysis: https://github.com/EBI-Metagenomics/MetagenomicsSearchDump/blob/master/src/output/run.py
    
    We need to port the Samples and Studies; and the analysis aggregation.
    
    The aggregation is required to merge the analysis into XML bundles that EBI Search
    ingest.
    
    AnalysesAggregation script -> https://github.com/EBI-Metagenomics/MetagenomicsSearchDump/blob/master/src/RunEntryAggregator.py
    
    Another task: Is to add a indexed_at field on Analyses, Samples and Studies.
    This is going to be useful to have incremental dump generation, instead of
    having to re-generate all the files. Moving to this will also required
    the modification of the AnalysesAggregation script OR for us to move
    to EBI Search incremental dumps (https://www.ebi.ac.uk/seqdb/confluence/display/EXT/Preparation+for+incremental+indexing)
    
    Last one. EBI Search also supports JSONs (https://www.ebi.ac.uk/seqdb/confluence/display/EXT/JSON+data+format)
    mberacochea committed Jul 12, 2023
    Configuration menu
    Copy the full SHA
    651a8bb View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2023

  1. Configuration menu
    Copy the full SHA
    6f6c070 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. Configuration menu
    Copy the full SHA
    2b02f66 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    974371f View commit details
    Browse the repository at this point in the history

Commits on Oct 23, 2023

  1. Configuration menu
    Copy the full SHA
    14f0842 View commit details
    Browse the repository at this point in the history

Commits on Oct 24, 2023

  1. Configuration menu
    Copy the full SHA
    1a3acfd View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2023

  1. Configuration menu
    Copy the full SHA
    afa4cb5 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    d914ec1 View commit details
    Browse the repository at this point in the history
  3. v2.4.36

    SandyRogers committed Nov 10, 2023
    Configuration menu
    Copy the full SHA
    ca360e0 View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2023

  1. Configuration menu
    Copy the full SHA
    2827c2b View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2023

  1. Configuration menu
    Copy the full SHA
    d18573c View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2023

  1. Configuration menu
    Copy the full SHA
    28908ca View commit details
    Browse the repository at this point in the history

Commits on Jan 3, 2024

  1. Configuration menu
    Copy the full SHA
    5852201 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    6b40098 View commit details
    Browse the repository at this point in the history
  3. Merge branch 'develop' into feature/ebi-search-dump-migration

    # Conflicts:
    #	emgcli/__init__.py
    #	pyproject.toml
    SandyRogers committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    480d951 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d185d4e View commit details
    Browse the repository at this point in the history

Commits on Jan 5, 2024

  1. Configuration menu
    Copy the full SHA
    f36021e View commit details
    Browse the repository at this point in the history

Commits on Jan 16, 2024

  1. Merge branch 'develop' into feature/ebi-search-dump-migration

    # Conflicts:
    #	emgapi/models.py
    #	emgcli/__init__.py
    #	pyproject.toml
    SandyRogers committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    5eb0fc7 View commit details
    Browse the repository at this point in the history
  2. Merge branch 'develop' into feature/ebi-search-dump-migration

    # Conflicts:
    #	emgcli/__init__.py
    #	pyproject.toml
    SandyRogers committed Jan 16, 2024
    Configuration menu
    Copy the full SHA
    57a391b View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2024

  1. Configuration menu
    Copy the full SHA
    67edbcd View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0e21458 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    0a8f872 View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2024

  1. Configuration menu
    Copy the full SHA
    e10fd33 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    0c8c61c View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    a036402 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    4fd2f3d View commit details
    Browse the repository at this point in the history
  5. typo

    SandyRogers committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    de3b415 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    d13789e View commit details
    Browse the repository at this point in the history
  7. 2.4.44

    SandyRogers committed Jan 22, 2024
    Configuration menu
    Copy the full SHA
    d23c962 View commit details
    Browse the repository at this point in the history

Commits on Jan 29, 2024

  1. Configuration menu
    Copy the full SHA
    8140ef5 View commit details
    Browse the repository at this point in the history