Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multithreading on new MultiLCA in Ubuntu #116

Closed
igt-misc opened this issue Dec 5, 2024 · 1 comment
Closed

Multithreading on new MultiLCA in Ubuntu #116

igt-misc opened this issue Dec 5, 2024 · 1 comment

Comments

@igt-misc
Copy link

igt-misc commented Dec 5, 2024

Hi team,
I've noticed since the change to MultiLCA, it doesn't appear to be using multiprocessing the way it used to, and is primarily running on a single thread.

System in question is Ubuntu 22.04 LTS, on Python 3.12. I have an AMD CPU and have installed Pypardiso, and have tested both running a standalone script as well as in a Jupyter notebook. All BW25 packages up to date via pip. On Windows it appears to be running as expected, though I haven't tested as thoroughly.

I haven't done anything outside of the normal workflow:

  • Make datapackages
  • mlca.lci()
  • mlca.lcia()
  • Process results

Could it be a configuration issue on my side? Or is it just not possible to parallelize the new setup? For what it's worth, the MLCA I'm trying to calculate is between 1 and 7 activities with all 250-ish ReCiPe indicators, so I'm well aware I'm a very, very fringe case.

@cmutel
Copy link
Member

cmutel commented Dec 5, 2024

pypardiso should use multiple cores, at least while solving Ax=b. The other operations are single-threaded.

As both pypardiso and modern scikits_umfpack are multithreaded, and maintaining the multiprocessing code was a pain, this was removed quite a while ago.

In your case I would recommend a different approach which can be much faster than the MultiLCA method - see here: https://gist.github.com/cmutel/159a99be89139a381a243035eb8d4c05. The main advantages of the MultiLCA approach were:

  • Labelled functional unit groups
  • Correlated sampling of normalization and weighting values

Pretty sure this code will evolve over time as what we have right now isn't really solving real user problems.

@cmutel cmutel closed this as completed Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants