Further integration with DreamcastControllerUSBPico

[Tails86]

This post is related to integration with https://github.com/OrangeFox86/DreamcastControllerUsbPico

@flyinghead I hope you don't mind me just dumping my thoughts here. It feels more right to put this here instead of my project since this one is sort of the driver of it. I'd be ok with moving this discussion though if it becomes noise to you or others - just let me know.

@kosekmi, I created discussion to share my thoughts on how I'd like to proceed with the Flycast integration after spending time digging into limitations of USB HID and what we're working with. I am very happy with where things are, but my dream is to support the 4P mode and/or 4 x 1p devices at once in a way that will be plug-and-play.

What I have found:

• The path to the current controller can be retrieved from SDL_JoystickPath(sdl_joystick), and sdl_joystick is given to DreamConnGamepad constructor
• SDL_hid_open_path, SDL_hid_write, and SDL_hid_read work perfectly fine for this controller, and reading data doesn't seem to interrupt normal operation (at least on Windows, but I suspect other OSes work similarly)
• The SDL_hid_enumerate works very well to get all devices, and the interface_number matches with the reverse of the index of the control when it finds 4 devices (0 for P4; 1 for P3, 2 for P2, and 3 for P1)
  • It also properly returns the label in product_string ({"P1", "P2", "P3", "P4"} for host-4p or simply "Dreamcast Controller" for host-1p)
  • The path variable in the enumerated devices will match the path returned from SDL_JoystickPath(sdl_joystick), so this will properly connect the current joystick to an element in the enumeration
• The gamepads use INTERRUPT endpoints at 1 ms, and HID devices seem to be limited to report sizes of 64 bytes (limiting transfer on these endpoints to 64,000 bytes per second for all communication here)
• I can, however, create one or more separate HID devices with BULK endpoints. It would still be limited to 64 byte reports, but BULK reports are streamed rather than limited to any interrupt period, so there isn't a hard upper bandwidth limit. The gamepads can then take precedence and still allow us to transfer everything we need separately.
  • Maximum of 16 IN and 16 OUT endpoints can be made per USB device, and I am currently using 7 IN and 2 OUT

What I will implement on the hardware:

• I think making another 4 separate generic HID devices (one for each controller) would make sense so data can be separately streamed to each, using another 4 IN and 4 OUT endpoints
• Packets will be stitched together and sent once a flag is set in the final packet (command structure is TBD)
• I want the default VMU screens to show A, B, C, or D for host-4p ex:
  • I think I'll add a command to allow this to reflect what is selected on flycast

What I'd like to see implemented on flycast:

• Automatic connection and communication over USB HID protocol
  1. Enumerate all HID devices matching the device's VID/PID
  2. Find the enumerated device with a path that matches current device
  3. Note the interface number and add 4 to get the target communication interface
  4. Find that enumerated device and connect to it
  5. Call SDL_hid_set_nonblocking to have nonblocking writes
  6. Spin up a new thread to read data as it comes in
• The DreamConnGamepad::_name can be updated to include the interface number when host-4p version is used (append A, B, C, D)
• Send a "reset" command on teardown to reset state of everything
• I'd like to find a way to automatically map C, D, and Z buttons which the arcade stick and other obscure devices use. @flyinghead suggested that this could be done through InputMapping in mapping.h. I looked long enough to get the jist, but I get the feeling it will be painful to implement correctly 🤣
• Support saves directly to/from hardware VMU

@kosekmi, is there anything in this list you'd like to tackle or anything else you'd suggest?

[kosekmi]

Thanks for writing this up!

...but my dream is to support the 4P mode and/or 4 x 1p devices at once in a way that will be plug-and-play.

I totally agree! Like we discussed offline, this will most likely mean to supersede the current CDC implementation by either Ethernet (which has its own limitations), or, like you laid out, HID.

• The path to the current controller can be retrieved from SDL_JoystickPath(sdl_joystick), and sdl_joystick is given to DreamConnGamepad constructor
• The gamepads use INTERRUPT endpoints at 1 ms, and HID devices seem to be limited to report sizes of 64 bytes (limiting transfer on these endpoints to 64,000 bytes per second for all communication here)
• I can, however, create one or more separate HID devices with BULK endpoints. It would still be limited to 64 byte reports, but BULK reports are streamed rather than limited to any interrupt period, so there isn't a hard upper bandwidth limit. The gamepads can then take precedence and still allow us to transfer everything we need separately.
• Maximum of 16 IN and 16 OUT endpoints can be made per USB device, and I am currently using 7 IN and 2 OUT

This sounds very promising!

• SDL_hid_open_path, SDL_hid_write, and SDL_hid_read work perfectly fine for this controller, and reading data doesn't seem to interrupt normal operation (at least on Windows, but I suspect other OSes work similarly)
• The SDL_hid_enumerate works very well to get all devices, and the interface_number matches with the reverse of the index of the control when it finds 4 devices (0 for P4; 1 for P3, 2 for P2, and 3 for P1)
• It also properly returns the label in product_string ({"P1", "P2", "P3", "P4"} for host-4p or simply "Dreamcast Controller" for host-1p)
• The path variable in the enumerated devices will match the path returned from SDL_JoystickPath(sdl_joystick), so this will properly connect the current joystick to an element in the enumeration

I can check this on MacOS and Linux and see if we require OS diffs.

• I want the default VMU screens to show A, B, C, or D for host-4p ex: VMU-A
• I think I'll add a command to allow this to reflect what is selected on flycast

Good idea. This can then also be coupled with Send a "reset" command on teardown to reset state of everything when exiting games.

• Support saves directly to/from hardware VMU

The holy grail 🔥 From my (very limited) understanding of how Flycast handles virtual VMUs and your experience on HID SDL_hid_write this should be doable in the long run.

is there anything in this list you'd like to tackle or anything else you'd suggest?

You pretty much covered everything 👍

[kosekmi]

One thing I forgot to mention: when using USB HID, we implicitly get rid of the Windows, MacOS, and Linux limitations - Hello Dreamcast Controllers on Android Emulation Handhelds 🤩

[Tails86]

Good point! I think that may require some more extensive refactoring to make an abstraction layer and proper factory since we're currently very tightly coupled to DreamConn interfaces.

[Tails86]

I had made some assumptions that were incorrect

• An HID device seemingly must be INTERRUPT based 🤦
• SDL_hid_enumerate doesn't seem to actually enumerate all HID devices (at least on Windows)

The options I'm thinking:

1. Live with the 64,000 bytes per second limitation - that's the max bandwidth for each direction
   • I'm not feeling too happy about this, but it does at least seem feasible considering that it's not too much slower than normal DC communication
   • That's about 4 ms to get a screen displayed
   • Sending vibration commands should be within 1 ms
   • Reading or writing the entire VMU would take 2 or 3 seconds
   • Considering the Dreamcast controller is normally polled at 50 or 60 Hz, that isn't too bad
2. Harden the current serial communication - I see a definite path to completing this way, but there are some pitfalls
3. Add libusb to flycast (not sure if that is valid)

@kosekmi, thoughts?

[Tails86]

Also, @flyinghead are there any restrictions/known limitations for bringing libusb into the fold?

[flyinghead]

I've never used it but if it plays well with SDL and other stuff then I have no issue.

[kosekmi]

Thanks for looking into this

Harden the current serial communication - I see a definite path to completing this way, but there are some pitfalls

The serial handler has to be abstracted away from DreamConn, which is manageable. However, we cannot work around the controller assignment issue with this - or did you find a solution? While the Ethernet option has the same limitation, it will simplify things in Flycast, but will probably require some heavy lifting in DreamcastControllerUsb. Did you give this another thought?
From an OS-dependency perspective, both serial and ethernet are equally limited for Windows, MacOS, and Linux. I gave Android a go and found that neither serial nor ethernet will work out of the box:

• Serial: Superuser permissions are required for serial interface access. However, this seems to be dependent on the device type: While superuser permissions are required for our CDC, stuff like USB-to-UART works out of the box (e.g., ESP8266)
• Ethernet: Connecting USB Ethernet, the OS disables the integrated WiFi. While there are a ton of Android retro devices which do not have cellular, and retroachievements and DCNet are a thing, the tradeoffs are something to consider

In general, I don't want to stress Android support, which probably isn't a prevalent use-case for our controller anyway since most Androids are handheld devices. Just for completeness: I did not look into other supported OS like iOS or Switch; I think most use cases will be covered with the former.

Live with the 64,000 bytes per second limitation

So we essentially would trade off proper controller assignment handling and OS-independency for limiting the effective bus bandwidth. Considering a ~10ms VMU LCD update frequency was the fastest I observed until now this should probably be fine, but this is my subjective experience. @flyinghead do you have examples where this might become a problem? At some point, you mentioned that some games are pretty picky on Maple Bus speeds, so this might be a problem we will run into when going down this road.

Another point to consider is libretro, which I did not look into at all. @flyinghead Out of your head, would you expect any mayor roadblocks for the proposed variants (i.e., serial, ethernet, sdl, libusb) for libretro?

[flyinghead]

Regarding the maple bus speed, it shouldn't be an issue since the game is running with its own time, distinct from real time. So the game will be paused until we get the maple response from real hardware.

The situation with libretro is completely different: a libretro core doesn't deal with input controllers at all. This is done by the front-end and the cores only see virtual controllers (a.k.a. RetroPad) exposed by the front-end. There is no way to access physical controllers (and it shouldn't be done).
Any integration would have to be done with RetroArch first. And it won't be a walk in the park to expose anything specific to the core.

[flyinghead]

And feel free to keep this discussion here.