Skip to content

Conversation

Doekin
Copy link

@Doekin Doekin commented May 15, 2025

This PR addresses issues such as ModuleNotFoundError that could occur during stub generation when extension modules depend on external shared libraries (e.g., .dll on Windows, .so on Linux) located outside of the standard search paths.

Background

On Windows, when stubgen attempts to import an extension module to generate stubs, importlib.import_module() may fail if the Python interpreter cannot locate the required dependent DLLs—for example, if they are not in the same directory as the .pyd file.

Example error:

 Generating my_ext.pyi
 Traceback (most recent call last):
   File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-snyzxcow\overlay\Lib\site-packages\nanobind\stubgen.py", line 1440, in <module>
     main()
   File "C:\Users\runneradmin\AppData\Local\Temp\pip-build-env-snyzxcow\overlay\Lib\site-packages\nanobind\stubgen.py", line 1367, in main
     mod_imported = importlib.import_module(mod)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   ...
 ModuleNotFoundError: No module named 'my_ext'

In contrast, on Linux, the dynamic linker can use the extension module's embedded RPATH to find dependencies.

Proposed Change

A new LIB_PATH argument has been introduced to nanobind_add_stub. This option allows you to specify a list of directories containing dependent shared libraries. During stub generation:

  • On Windows, these directories are added to the DLL search path using os.add_dll_directory().
  • On Linux, the paths are prepended to the LD_LIBRARY_PATH environment variable (on macOS, to DYLD_LIBRARY_PATH).

Example Usage

# Suppose my_ext.pyd links against some_dynamic_lib.dll,
# and some_dynamic_lib.dll is in a separate output directory.
nanobind_add_module(my_ext my_ext.cpp)
target_link_libraries(my_ext PRIVATE some_dynamic_lib)

nanobind_add_stub(
    my_ext_stub
    MODULE my_ext
    OUTPUT my_ext.pyi
    PYTHON_PATH $<TARGET_FILE_DIR:my_ext>
    # Provide the directory containing some_dynamic_lib.dll or .so
    LIB_PATH $<TARGET_FILE_DIR:some_dynamic_lib>
    DEPENDS my_ext
)

@Doekin Doekin force-pushed the stubgen_win_dll branch from 4b84a7b to 0a3bc72 Compare May 15, 2025 11:58
@wjakob
Copy link
Owner

wjakob commented May 20, 2025

I am generally open to this feature, but what bothers me is the windows-specific nature. The two other supported platforms (Linux, macOS) have analogous ways of adding dynamic library search paths.

@Doekin Doekin force-pushed the stubgen_win_dll branch from 0a3bc72 to 1778524 Compare May 20, 2025 03:01
@Doekin
Copy link
Author

Doekin commented May 20, 2025

Thanks for raising that point! I updated the code to handle dependent shared libraries on Linux and macOS as well. The option has been renamed from DLL_PATH to LIB_PATH, and LIB_PATH now prepends the specified paths to LD_LIBRARY_PATH (on Linux) or DYLD_LIBRARY_PATH (on macOS).

@Doekin Doekin changed the title stubgen: add DLL_PATH option to locate dependent DLLs on Windows stubgen: add LIB_PATH option to locate dependent shared libraries May 20, 2025
@Q-Minh
Copy link

Q-Minh commented Oct 8, 2025

What is the status of this PR? This feature is very much appreciated and valuable.

@hpkfft
Copy link
Contributor

hpkfft commented Oct 9, 2025

Would you be kind enough to add documentation (the source for which is found in the docs subdirectory) to https://nanobind.readthedocs.io/en/latest/api_cmake.html and also to https://nanobind.readthedocs.io/en/latest/typing.html#command-line-interface
The latter would be especially helpful to those not using cmake.

@Doekin
Copy link
Author

Doekin commented Oct 9, 2025

Thanks for the reminder

@Doekin Doekin force-pushed the stubgen_win_dll branch 3 times, most recently from a69b64c to 95db088 Compare October 9, 2025 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants