Should we document our internal headers

[miscco]

We are currently only publicly exposing the "standard" headers like <cuda/std/functional>

This is fine and safe and we provide the common CTK stability guarantees about them.

However, the C++ standard headers are also heavy. Including all of <cuda/std/type_traits> for is_arithmetic or <cuda/std/utility> just for cuda::std::move is a bit much.

We recently have been hard at work at modularizing our headers so that each feature gets its own properly self contained subheader.

All these headers live in <cuda/std/__$HEADER/$FEATURE.h> which is guarded by the reserved identifier __$HEADER so none of the common stability guarantees apply.

It would be greatly beneficial for users to be able to only include what they need, potentially dramatically improving compile times. I am personally strongly against making these headers stable because we need the ability to change them and are doing so currently. That said, 99% of those headers are never going to go away like <cuda/std/__type_traits/is_floating_point.h>. Their contents might change but not the header itself.

Is there a way to document those headers to the public but retain our ability to remove / rename them when needed?

@gonzalobg

[gonzalobg]

I think that:

• Internal headers should, per definition, be internal, unstable, don't have to be documented, etc.
• Other projects have shown that there is value in providing two kinds of public headers:
  • wide public headers providing a wide range or all of the functionality for productive development out of the box (e.g.range/v3/all.hpp, <algorithm>, etc.).
  • narrow public headers providing an as narrow set of functionality as possible to allow programs that want to optimize compile-times to only include the features the program uses, and to avoid including features the program does not use.

[bernhardmgruber]

What if we give you a "narrow public header", but suddently that header requires a tweak for, say, FP8 types, and we have to include <cuda_fp8.h> adding 500ms of parse time. This may be a "breaking" change for your business case, but it may be a fix for somebody else. It's all a trade-off.

[gonzalobg]

We can always turn a narrow public header into a wide public header, by refactoring into including multiple narrow public headers. That allows us to ship the fix, and that allows the customers that are impacted by the higher compile-times, to switch to a smaller set of narrower CCCL headers, to fix that.

[miscco]

That is a nontrivial amount of work for us

[bernhardmgruber]

Should we document our internal headers

No

It would be greatly beneficial for users to be able to only include what they need, potentially dramatically improving compile times.

Users which really need to optimize compile-time this much, can also just directly include an internal header and risk begin broken in the future.

Is there a way to document those headers to the public but retain our ability to remove / rename them when needed?

I don't see one.

[gonzalobg]

Users which really need to optimize compile-time this much, can also just directly include an internal header and risk begin broken in the future.

Since the risk of being broken is not acceptable for any production use case that has to be supported, the main alternative that these users have is to avoid using CCCL, or initially use it and then migrate to their own solutions if compile-times are not acceptable.

[bernhardmgruber]

Or we bless the small set of internal headers on a case by case basis. I assume the user base where the compilation cost of #include <cuda/std/functional> is a blocker is small enough. Those users can always ask us to promise them that we will not break #include <cuda/std/__functional/feature.h>. We can even add a test for that.

[bernhardmgruber]

Btw, you can also:

#if __has_include(<cuda/std/__functional/feature.h>)
#include <cuda/std/__functional/feature.h>
#else
#include <cuda/std/functional>
#endif

Which guards you against a reorganization we may do in the future.

[leofang]

Keep in mind that our team also supports Python users now (through NVRTC and other compilation stack). All of the costs that we discuss becomes run-time (JIT) overheads, so if we can figure out a balanced solution it would be ideal. @gonzalobg's wide/narrow suggestion seems reasonable to me?

[fbusato]

how often we plan to rename <cuda/std> headers? I guess we do this step in a very limited set of cases. We can always do breaking changes between major versions

[jrhemstad]

I think without any concrete data on a specific header being too heavyweight and blocking a potential user, any discussion here is largely just premature optimization.

If and when particular headers are found to be problematic, we can evaluate and handle these on a case by case basis.

Otherwise, I don't think it is worth the effort to try and formalize a library-wide policy.

[gonzalobg]

I see your point about premature optimization, and I also don’t think a general strategy is required.

Some important users measure compile times in milliseconds, and high compile times have been a reason for them to avoid some of our abstractions in the past. I think this is "fine" in general because some of our abstractions pull a lot of weight, and there is no one size fits all.

However, I think we should strive to ensure that these users have no reason to avoid using low-level hardware feature abstractions, because these are the vehicle we use to ship critical bug fixes as we find them (in HW or lower-level SW). These abstractions tend to be small and self-contained, making it easier to provide good compile-times without incurring major costs, which seems like a worthy trade-off (low effort, high return).

[jrhemstad]

I completely support exposing specific features in self-contained headers. We already do this today with things like <cuda/antotated_ptr> or <cuda/pipeline>.

That's what I meant above about handling this on a case-by-case basis. That's very different from what @miscco suggested in the original discussion about categorically exposing every internal header across the library.

[fbusato]

well, if we look at some old results on clang on the host https://artificial-mind.net/projects/compile-health/ they are a bit concerning, especially if we think that nvcc is pretty slow