enable parallel testing using pytest-xdist

[CunliangGeng]

This PR adds the pytest-xdist to enable parallel testing to speed up the testing.

To make parallel testing work, some session-scoped pytest fixtures that generate data or files used by multiple test files were updated (see this example).

Results:

• sequential testing: 380s
• parallel testing (8cores): 85s

Using pytest --durations=0 can detect the most time consuming tests. It shows that tests on downloaders are most expensive, and ignoring them can reduce testing time from 85s to 15s!

[CunliangGeng]

Current dependencies on/for this PR:

• dev
  • PR refactor filtering of user specified strains #178
    • PR Enable GCF loader to filter singleton GCFs #181
      • PR return list for get_bgcs methods of BGC loaders #182
        • PR rename strains.py to strain.py #183
          • PR remove deprecated functions of loading genomics data #184
            • PR update the process of loading genomics data #185
              • PR rename genomics.py to utils.py #186
                • PR remove deprecated functions of loading metabolomics data #187
                  • PR refactor class MolecularFamily #188
                  • PR refactor class GCF #189
                  • PR enable parallel testing using pytest-xdist #190 👈
                  • PR Enable filtering of singleton molecular families #191
                  • PR Remove class SingletonFamily and MF attribute id #192
                  • PR refactor Spectrum class #193
                  • PR remove unused mgf.py and its test file #194
                  • PR Update logics of loading metabolomics data #196
                  • PR Update loading of genomics data #197
                  • PR deprecate the method _load_class_info #198
                  • PR remove loading of optional data #199
                  • PR remove filtering of common strains during loading #200
                  • PR update logics of loading mibig data #201
                  • PR refactor initiation of paths to metabolomics files #202
                  • PR Update logics of loading metabolomics data #195

This stack of pull requests is managed by Graphite.

[gcroci2]

Just for me to understand, reading from the docs, pytest-xdist can be used in cases a fixture needs to execute exactly once per test session, for example specifying scope="session" in the fixture and by reading from a FileLock to produce the fixture data only once. I don't see here any .lock file though. So I wonder how the tests can run in parallel with the current edits.

[CunliangGeng]

Just for me to understand, reading from the docs, pytest-xdist can be used in cases a fixture needs to execute exactly once per test session, for example specifying scope="session" in the fixture and by reading from a FileLock to produce the fixture data only once. I don't see here any .lock file though. So I wonder how the tests can run in parallel with the current edits.

We solved the issue in a different way. See the example below:

We used scope="session", it cannot make sure that the data is generated once, but it ensure that the data is generated once in one process (or one cpu core).
We also used tmp_path_factory (it's a session-level fixture provided by pytest, unlike the function-level tmp_path). It will make sure the data generated in different sessions will be stored in different temporary paths.

So for the example below,

1. If the data is generated once for all processes, very good then
2. if the data is generated more than once for all processes, then our setting can make sure the different processes have their own copy of data in a different temporary path. Say, process 1's data is in path1, while process 2's data is in path2, making sure process 1 and process 2 are independent tests.

class TestMibigBGCLoader:
    @pytest.fixture(scope="session") 
    def data_dir(self, tmp_path_factory): 
        # get the temp directory shared by all workers
        download_root = tmp_path_factory.mktemp("download")
        extract_path = tmp_path_factory.mktemp("metadata")
        download_and_extract_mibig_metadata(download_root, extract_path)
        yield str(extract_path)

[CunliangGeng]

Merge activity

• Dec 19, 8:24 AM: @@CunliangGeng started a stack merge that includes this pull request via Graphite.
• Dec 19, 8:25 AM: @@CunliangGeng merged this pull request with Graphite.

[hechth]

@florian-huber we could consider using this also for matchms to speed up test execution