Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

multiprocessing #5

Open
quantenschaum opened this issue Apr 2, 2023 · 3 comments
Open

multiprocessing #5

quantenschaum opened this issue Apr 2, 2023 · 3 comments

Comments

@quantenschaum
Copy link

quantenschaum commented Apr 2, 2023

This one is a bit tricky. One would think that if I used 2 processes to compute the alamanc, it would only take half the time. Unfortunately this is not the case, because SkyField internally seems already to use all available cores (the sipce kernel I assume). So, splitting the work does just put more load on the already fully loaded cores, which will not reduce wall clock time. :( There is some reduction of the time due to filling the small gaps when the load drops a bit.

Generating one month on my 4 core machine takes 1m37s singlethreaded and 55s multithreaded, not 24s as expected.

@aendie
Copy link
Owner

aendie commented Apr 2, 2023

I once looked into the possible processing speed enhancements using Skyfield and my report is here:
skyfielders/python-skyfield#612

@quantenschaum
Copy link
Author

quantenschaum commented Apr 3, 2023

Very interesting. What puzzles me is, that SkyField is "pure-Python" but when generating an almanac in a single process, it still uses multiple cores. Any idea why?

SkyField uses numpy and numpy uses native parallel matrix vector operations, that might be the reason.

@quantenschaum
Copy link
Author

IMHO the parallelization could be simplified. Concerning the fact that you do not see a gain in speed on Linux, you may want to read my comment skyfielders/python-skyfield#612 (comment).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants