Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR introduces some optimisations to improve import times (and therefore startup times).
Initially in #7 I requested that the cache is able to be disabled entirely, but with
importlib.reload()
being an edge case andimportlib.invalidate_caches()
existing it seems sufficient that the maturin import hook supports this way of clearing the cache to ensure new modules are detected.Benchmarking
Benchmarking was carried out using the
tests/create_benchmark_data.py
script. this script creates a scenario close to the worst case, with 100 paths insys.path
each very long nested paths with twopyproject.toml
s present on the path back to the root of the filesystem that must be checked.Results:
These results are run on the slowest storage I have (a USB drive). Each value is the average over 3 runs.
The 'cold cache' runs are after running
echo 3 | sudo tee /proc/sys/vm/drop_caches
main
is_maybe_maturin_project
optimisationlru_cache
find_dist_info_path
optimisation(note that for a more reasonable case such as 20 paths in
sys.path
, the time taken is ~0.01s)