Create testing corpus of rust constructs

[LegNeato]

The project doesn't currently support all rust constructs. It's not clear what is supported and what is not, so users will suddenly trip over issues.

One idea to generate such a list is to try to compile (not run) all no_std and no alloc crates on crates.io and see which fail...similar to what miri does.

Another way would be to create a rust program generator and "fuzz".

[LegNeato]

Potentially useful:

https://www.ralfj.de/blog/2024/11/25/rustlantis.html

[schell]

We could also come at it from the other direction and maintain a list of constructs known to produce compilation errors, and not worry about it being exhaustive? I have the beginnings of such a list. The benefit being that this would have the lowest barrier to getting something.

[LegNeato]

Looks like there might be a version of rustlantis with no_std support floating around:

https://www.reddit.com/r/rust/comments/1gzhi7x/comment/lyx3l5y/

[LegNeato]

@FractalFir Hello! Thanks for your offer to help (https://www.reddit.com/r/rust/comments/1gzhi7x/comment/lz85d66/). Let's repurpose this task if that works for you. What can I do to make it so we can leverage your fork? I understand there are likely some additional changes we'll need to do to rustlantis once we get it all hooked up as our reqs are a bit different.

I can invite you to our maintainers zulip if you would prefer chat! It might be useful anyway as we are all working on out-of-tree codegen backends 🍻 .

[FractalFir]

I would gladly join your zulip :).

There are a few things required to get my fork working with rust-gpu, but they should not be too big of an issue.

The first step is creating the right main function, compatible with rust-gpu. I don't think this will be too difficult.

The next issue is getting the printf from the shader to output to standard output. I assume the debug_printf(which seems to be supproted by rust-gpu) works exactly like printf. If this is the case, then we only need to replace calls to printf with debug_printf, and enable the right extensions. I n my testing, I could not get anything to print from a shader, but I assume this is just the result of my lack of experience.

I also assume you support a relatively recent version of Rust(my fork requires support for c-string literals), so that probably will not be an issue.

We will also need some kind of runner program, capable of loading the shader, and executing it on the gpu.

Rustlatins has a built-in script for compiling and comparing code using different backends, but it is not well suited for backends which require a non-standard excution enviorments(eg. need to run in the .NET runtime, or on the GPU). That script also assumes that all backends compile the same code, which may be a bit difficult to pull off.

For my own purposes, I simply wrote a custom rust script for that purpose, since that was easier than getting the solution built-in into rustlatis to work well with my backend.

Also, I think I saw somehwere that you don't support overflow checks for multiplcation / additon, which are required by rustlatnis. This may be already fixed(since I saw that some time ago), but if this is still an issue, maybe I could share my soltuion to it.
Since .NET lacks suitable overflow check intrinsics, all of my checks just use simple arithmetic operations / cast / a little branching, so they could be adapted to work with rust-gpu.

[FractalFir]

I had a bit of free time to work on this, and I am mostly done with changes to rustlatis itself.

When the --rust-gpu option is specified, it will use the debug_printf! macro, and use the spirv attribute to mark the main function.

However, the generated code currently does not compile, mostly due to some... wonkines around the debug_printf! macro. Some of those are inherent to the underlying extension, but some are just a bug in rust-gpu macro.

In C %li or %ld is a valid format specifier, but it is rejected by the debug_printf macro:

error: Unrecognised format specifier: 'l'
   --> examples/shaders/sky-shader/src/lib.rs:103:28
    |
103 |              debug_printf!("%li",*self as c_long);
    |                            ^^^^^

Since those are not supported by the vulcan extension, there is no easy way to print 64 bit signed ints from rust-gpu. There are workarounds, but they are not very efficent.

The simplest solution is printing the sign first, and the rest of the 64 signed intiger as an 64 unsinged int.

As for the bug in rust-gpu implementation of this macro, debug_printf macro also does not escape { to {{ when creating the asm! invocation.

error: invalid asm template string: expected `'}'`, found `'"'`
     --> examples/shaders/sky-shader/src/lib.rs:13016:2
      |
13015 | unsafe{debug_printf!("Variant3{")};
      |        -------------------------- in this macro invocation
13016 |         unsafe{debug_printf!("fld0:")};
      | ----^ expected `'}'` in asm template string
      | |
      | because of this opening brace
      |
      = note: if you intended to print `{`, you can escape it using `{{`
      = note: this error originates in the macro `debug_printf` (in Nightly builds, run with -Z macro-backtrace for more info)

Besides that, there are some issues with the custom_mir feature. I don't know the exact cause of those, but they look like the version of rustc used by rust-gpu is using different custom_mir syntax(or maybe the compiler is just really confused by the invalid asm! generated by the debug_printf macro?)

error[E0061]: this function takes 2 arguments but 3 arguments were supplied
  --> examples/shaders/sky-shader/src/lib.rs:499:1
   |
499 | Call(_12 = fn1(_9.fld1, _8.1, _7, RET, _4, RET, _1, _4), ReturnTo(bb3), UnwindUnreachable())

I don't think this is a rustc version issue, since I am running the exact same files on the newest nightly(they run just fine), and I have not changed them in almost a year.

EDIT: found another bug in the debug_printf macro:

error: 1 % arguments were found, but 2 variables were given
   --> examples/shaders/sky-shader/src/lib.rs:122:28
    |
122 |              debug_printf!("%lx%lx", (*self >> 64) as u64,*self as u64);
    |  

It seems to get confused by two format specifier without a separator.

This line of code:

rust-gpu/crates/spirv-std/macros/src/lib.rs

Line 480 in d3f9af7

ch = match chars.next() {

simply skips all % after the first one, which is not valid in this case.
I am also unsure if this code handles %% correctly.

EDIT2: unsure if this is a bug, but the debug_printf macro does not support 64 bit floats. From what I could see, they are supported in the newer versions of Vulkan(?):

KhronosGroup/Vulkan-ValidationLayers#8341

64 bit signed ints also seem to have recently(October) gained some support, so I am unsure if this is something printf_debug should expose.

KhronosGroup/Vulkan-ValidationLayers#8660

[LegNeato]

Oh wow, that was quick! 🚀 I fully expected there to be some debug_printf bugs as it has only been around for one release.

I'll try to take a look at fixing in the next couple of days. For truly unsupported things we can probably just swallow it for now as we are looking first and foremost for compilation issues before we move onto runtime issues.

@eddyb has changes to update us to a newer rustc that should hopefully land today 🤞

[FractalFir]

I managed to fix one of the issues with debug_printf in this PR: #166. The remaining issues are a bit harder to fix, but they have viable workarounds.

[LegNeato]

Landed. Sorry, I screwed something up w/ git when trying to fix the formatting.

[LegNeato]

These might be useful as well:

https://github.com/rust-lang/rust/tree/43ca9d18e333797f0aa3b525501a7cec8d61a96b/src/tools/miri/tests/pass