Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature/metagenomics exchange #325

Merged
merged 36 commits into from
Feb 8, 2024
Merged

Feature/metagenomics exchange #325

merged 36 commits into from
Feb 8, 2024

Commits on Jan 29, 2024

  1. adds incremental ebi search dumping for analysisjobs

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    0a8a761 View commit details
    Browse the repository at this point in the history

  2. integrates indexable sample data into analyses ebi search dump

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    4d51fe3 View commit details
    Browse the repository at this point in the history

  3. adds ebi search dump for studies/projects (plus some tweaks to analyses)

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    240d8f4 View commit details
    Browse the repository at this point in the history

  4. resolve migration merge conflicts

    @SandyRogers @mberacochea
    SandyRogers authored and mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    ed97ccb View commit details
    Browse the repository at this point in the history

  5. adds support for chunking of analysis-runs ebi search dump

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    ab829f4 View commit details
    Browse the repository at this point in the history

  6. bugfixes on analysis-run dumper

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    fcb9300 View commit details
    Browse the repository at this point in the history

  7. Add metagenomics exchange support

    @KateSakharova @mberacochea
    KateSakharova authored and mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    6438c3b View commit details
    Browse the repository at this point in the history

  8. Added 2 tests for ME, population script WIP

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    64f4900 View commit details
    Browse the repository at this point in the history

  9. Move metagenomic exchange commands to separate Class. Tests fail in P… 

    …OST request
    @KateSakharova @mberacochea
    KateSakharova authored and mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    7484116 View commit details
    Browse the repository at this point in the history

  10. My revised version: 

    - Refactored the cli wrapper
- Refactored the settings to be in line with the rest
- Modified the unit tests
    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    fee19bd View commit details
    Browse the repository at this point in the history

  11. Modified ME dj function to add new ME records, fixed tests for ME and… 

    … api
    @KateSakharova @mberacochea
    KateSakharova authored and mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    e7d9b17 View commit details
    Browse the repository at this point in the history

  12. Move test for mock API to test_mgx_api.py

    @KateSakharova @mberacochea
    KateSakharova authored and mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    2e0b019 View commit details
    Browse the repository at this point in the history

  13. Add settings

    @KateSakharova @mberacochea
    KateSakharova authored and mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    c92c381 View commit details
    Browse the repository at this point in the history

  14. change models and add patch and delete to ME

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    bdac39c View commit details
    Browse the repository at this point in the history

  15. Add delete and patch commands with tests to ME API

    @KateSakharova @mberacochea
    KateSakharova authored and mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    65e64e0 View commit details
    Browse the repository at this point in the history

  16. add population scripts and tests for dry-run

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    3c664e6 View commit details
    Browse the repository at this point in the history

  17. add population scripts and tests for dry-run

    @KateSakharova @mberacochea
    KateSakharova authored and mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    f082415 View commit details
    Browse the repository at this point in the history

  18. change api adress

    @KateSakharova @mberacochea
    KateSakharova authored and mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    bb1ce81 View commit details
    Browse the repository at this point in the history

  19. change api adress

    @KateSakharova @mberacochea
    KateSakharova authored and mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    0f309c4 View commit details
    Browse the repository at this point in the history

  20. WIP

    @KateSakharova @mberacochea
    KateSakharova authored and mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    83b6b3a View commit details
    Browse the repository at this point in the history

  21. Minor refactor of the MGX indexation process WIP

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    97e226e View commit details
    Browse the repository at this point in the history

  22. This is still a WIP - MGX

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    e229f5c View commit details
    Browse the repository at this point in the history

  23. Fixed the migrations for EBI Search and MGX. 

    The test tests/me/test_populate_metagenomics_exchange.py::TestMeAPI::test_removals_dry_mode it's working now.
    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    a6e9eda View commit details
    Browse the repository at this point in the history

  24. Fix some ME tests

    @KateSakharova
    KateSakharova committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    2f83224 View commit details
    Browse the repository at this point in the history

  25. WIP - migration tidy up

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    5961f7d View commit details
    Browse the repository at this point in the history

  26. Merge branch 'feature/metagenomics_exchange' of github.com:EBI-Metage… 

    …nomics/emgapi into feature/metagenomics_exchange
    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    02d3943 View commit details
    Browse the repository at this point in the history

  27. Adjust field names to new convention (last_ebi_search_indexed)

    @mberacochea
    mberacochea committed Jan 29, 2024
    Configuration menu
    Copy the full SHA
    60d2a37 View commit details
    Browse the repository at this point in the history

Commits on Jan 30, 2024

  1. WIP testing

    @KateSakharova
    KateSakharova committed Jan 30, 2024
    Configuration menu
    Copy the full SHA
    ad539e7 View commit details
    Browse the repository at this point in the history

Commits on Jan 31, 2024

  1. Tests for populate ME

    @KateSakharova
    KateSakharova committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    986f222 View commit details
    Browse the repository at this point in the history

  2. rm dev from tests and leave only DEV API

    @KateSakharova
    KateSakharova committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    916ce45 View commit details
    Browse the repository at this point in the history

  3. fix pipeline mock

    @KateSakharova
    KateSakharova committed Jan 31, 2024
    Configuration menu
    Copy the full SHA
    948caa8 View commit details
    Browse the repository at this point in the history

Commits on Feb 1, 2024

  1. fixes

    @KateSakharova
    KateSakharova committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    6a55c40 View commit details
    Browse the repository at this point in the history

  2. add mock to patch test

    @KateSakharova
    KateSakharova committed Feb 1, 2024
    Configuration menu
    Copy the full SHA
    18d4641 View commit details
    Browse the repository at this point in the history

Commits on Feb 6, 2024

  1. some fixes after review

    @KateSakharova
    KateSakharova committed Feb 6, 2024
    Configuration menu
    Copy the full SHA
    bfb1b54 View commit details
    Browse the repository at this point in the history

Commits on Feb 7, 2024

  1. Rename EBI Seach last_indexed fields

    @mberacochea
    mberacochea committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    7dbbb14 View commit details
    Browse the repository at this point in the history

  2. Add an extra migrate step to rename last_indexed -> last_ebi_search_i… 

    …ndexed

This is needed because RenameField doesn't allow you to change
the db_column name.

With this change it seems to work, and the data should be kept:

```sql
--
-- Alter field last_indexed on analysisjob
--
ALTER TABLE "ANALYSIS_JOB" RENAME COLUMN "LAST_INDEXED" TO "LAST_EBI_SEARCH_INDEXED";
--
-- Alter field last_indexed on study
--
ALTER TABLE "STUDY" RENAME COLUMN "LAST_INDEXED" TO "LAST_EBI_SEARCH_INDEXED";
--
-- Rename field last_indexed on analysisjob to last_ebi_search_indexed
--
--
-- Rename field last_indexed on study to last_ebi_search_indexed
--
```
    @mberacochea
    mberacochea committed Feb 7, 2024
    Configuration menu
    Copy the full SHA
    4a8f3b4 View commit details
    Browse the repository at this point in the history