Import of helper libraries into cuda kernels

[isolin]

I would like to import helper_math.h to be able to easily use vector math in agent functions. I am using Python interface. But when I try to import it, I get strange console output listing some other modules and an error at the end. Is it in conflict with some templating stuff? Any ideas how to properly import a header file please?

myAgentFunction = fr"""
#include <helper_math.h>
FLAMEGPU_AGENT_FUNCTION(myAgentFunction, flamegpu::MessageArray3D, flamegpu::MessageArray3D) {{
    // agent code using float3 here
}}
"""

Error output:

 algorithm
 assert.h
 cassert
 cfloat
 climits
 cstdint
 cstring
 cuda_runtime.h
 curand.h
 curand_discrete.h
 curand_discrete2.h
 curand_globals.h
 curand_kernel.h
 curand_lognormal.h
 curand_mrg32k3a.h
 curand_mtgp32.h
 curand_mtgp32_kernel.h
 curand_normal.h
 curand_normal_static.h
 curand_philox4x32_x.h
 curand_poisson.h
 curand_precalc.h
 curand_uniform.h
 device_launch_parameters.h
 dynamic/curve_rtc_dynamic.h
 flamegpu/defines.h
 flamegpu/exception/FLAMEGPUDeviceException.cuh
 flamegpu/exception/FLAMEGPUDeviceException_device.cuh
 flamegpu/gpu/CUDAScanCompaction.h
 flamegpu/runtime/AgentFunction.cuh
 flamegpu/runtime/AgentFunctionCondition.cuh
 flamegpu/runtime/AgentFunctionCondition_shim.cuh
 flamegpu/runtime/AgentFunction_shim.cuh
 flamegpu/runtime/DeviceAPI.cuh
 flamegpu/runtime/messaging/MessageArray2D.h
 flamegpu/runtime/messaging/MessageArray3D.h
 flamegpu/runtime/messaging/MessageArray3D/MessageArray3DDevice.cuh
 flamegpu/runtime/messaging/MessageBruteForce.h
 flamegpu/runtime/messaging/MessageBruteForce/MessageBruteForceDevice.cuh
 flamegpu/runtime/messaging/MessageNone.h
 flamegpu/runtime/utility/AgentRandom.cuh
 flamegpu/runtime/utility/DeviceEnvironment.cuh
 flamegpu/runtime/utility/DeviceMacroProperty.cuh
 flamegpu/util/detail/StaticAssert.h
 jitify_preinclude.h
 limits
 limits.h
 math.h
 memory.h
 plant_receives_environment_program
 stddef.h
 stdint.h
 stdio.h
 stdlib.h
 string
 string.h
 time.h
 type_traits

Error iteself:
myAgentFunction_impl.cu not in loaded sources! This may be due to a header being loaded by NVRTC without Jitify's knowledge.

[Robadob]

I don't actually recognise that error, it's coming from Jitify which we use for handling RTC compilation. But I know enough about RTC to explain the likely problem.

The RTC compiler does not have access to the CUDA_samples directory or CUDA include directory, I'm not even sure if we specify the working directory as somewhere it can look for include files. Similarly, standard C/C++ system headers are not available to NVRTC, and Jitify provides shims for a subset of them. This generally means including additional headers within RTC kernels can be a bit delicate.

We recognise that basic vector maths are very useful, and have an experimental feature USE_GLM, which enables access to glm vector support (which is far more thorough than that CUDA header which I believe was deprecated long ago).

The reason this feature remains experimental, is that the technique Jitify uses for processing source prior to runtime compilation is inefficient for large include hierarchies. So the simple act of including the core GLM header, prohibitively increases the compile time of RTC kernels (e.g. 60 seconds per agent function). We have an open issue to address this with a workaround, but it's not something we've got around to yet.

If you would like to use GLM, it would require a build of pyflamegpu with USE_GLM enabled at CMake configure time. For the above reason we don't officially provide this ourselves.

Otherwise, I can try and find time to look into this next week, it might be as simple as copying the header to where pyflamegpu finds the flame gpu headers (inside the python work packages dir for pyflamegpu). Or it might require some more advanced tinkering, potentially even another patch to Jitify (I had to submit one to get GLM supported).

Regardless, I don't believe our documentation has any coverage related to including 3rd party headers (especially in the context of RTC), so i'll create an issue for this too.

[isolin]

Thank you for the fast reply! At least it's good to know that it's no error of mine, since C++ and CUDA are not my primary languages. The GLM related increase of compile time to 60s is truly way too much. As a temporary fix I will just copy a few of the vector functions from the header and directly insert them above the agent function as FLAMEGPU_DEVICE_FUNCTIONs. That should fix it (although not in a very nice way).

[Robadob]

We've now merged this PR, which will allow you to include files within RTC code. If you wish to take advantage of this, you would need to build pyflamegpu yourself, as we only produce official builds with releases.

By default it will treat the working directory as the only location to search for includes (in addition to the internal cuda/flamegpu include directories).

If you would prefer to store includes elsewhere, you can setup include paths by setting the FLAMEGPU_RTC_INCLUDE_DIRS environment variable (note this is an operating system environment variable, not a FLAME GPU environment property). Where you can specify multiple paths separated by ; or : depending on whether you are using Windows or Linux.

This is currently only documented in the readme, there's an issue to add it to the user guide, however that's unlikely to occur until the user guide rework has been completed.

---

I also discussed that error message with one of the Jitify developers, and we realised it's reporting the wrong file due to a conflict between FLAME GPU and how Jitify operates. We don't currently plan on fixing it within FLAME GPU, as this would cause RTC compile errors to report the incorrect line numbers. It will possibly be fixed by Jitify in the coming months, as they are currently redesigning the affected algorithm.