Improve WSL connection

[firejox]

I have used winsocat and socat to bridge the connection between Windows host and WSL. And this has lower latency than using TCP. And now I want to improve it further.

So the current connection between host and WSL is:

[xpra client] <- named pipe -> [winsocat (Windows)] <-  stdio (pipe) -> [socat (WSL)] <- unix socket -> [xpra server]

Both winsocat and socat will allocate buffer to transfer data during the connection.

I come up with some ideas about the improvement.

1. Implement hyper-v socket and unix socket connection on windows client
   As what wslg do, the client side will connect via hyper-v socket and server will listen on vsock.
   This implementation will improve the WSL2 connection. For WSL1, Windows/WSL Interop with AF_UNIX show that xpra client is possible to connect server via unix socket. The connection will be:

WSL2: [xpra client] <- vsock -> [xpra server]
WSL1: [xpra client] <- unix socket -> [xpra server]

2. Improve my pipe connection
   Basically, winsocat will listen named pipe connection and then create process for each client. This can be simplified by letting xpra client able to create wsl process. Then xpra client can communicate with wsl process via pipe. In order to improve the connection, this wsl process may just pass the stdio fds of the process to server with some protocol. The connection will be:

[xpra client] <- stdio (pipe) -+-> [xpra client proxy] -- passing stdio fds -> [xpra server]

It will be great to have these implementation.

[totaam]

Ah!
Funny that, I was just re-working the socket code and moved around some bits.
Implementing hyper-v sockets would be great! How does it relate to vsock?

[firejox]

It is the Windows-defined virtual socket for hyper-v vm, and it will correspond to the vsock of Linux guest vm.

There are some document for hyper-v socket development.
Make your own integration services via hyper-v socket
linux kernerl code about hyper-v socker

[totaam]

AFAICT, AF_HYPERV is not available in the Python 3.10 builds provided by MSYS2.
Looks like this will be available in 3.12: https://github.com/python/cpython/blob/80c08d1cd67afdd1336c65ba23a044b6ac490f33/Doc/library/socket.rst#id87

Found some helpful code here: Using the Hyper-V sockets between Windows host and Linux guest.
Looks like this will need to be developed from a WSL2 environment.

[firejox]

After some testing, I think there is no need to develop under WSL2 environment. The only need is to add Hyper-V Socket functionality. Then Windows client will connect to the WSL2 vm id with the service id [vsock port in hex]-facb-11e6-bd58-64006a7986d3 which port is listened by xpra server.

Here is a demo of spawning a xterm with relay tool via VSOCK and Hyper-V Socket.

[totaam]

@firejox where do you get the [vsock port in hex] from?
What about -facb-11e6-bd58-64006a7986d3?

It might be possible to compile Python 3.12 under MSYS2.

[firejox]

@totaam It comes from the explanation in Linux kernel code

 /****************************************************************************
 * The only valid Service GUIDs, from the perspectives of both the host and *
 * Linux VM, that can be connected by the other end, must conform to this   *
 * format: <port>-facb-11e6-bd58-64006a7986d3.                            *
 ****************************************************************************/

And HyperV Socket document in Microsoft doc also tell that

Note: The Service GUID for a Linux guest uses the VSOCK protocol which addresses via a svm_cid and svm_port rather than a guids. To bridge this inconsistency with Windows the well-known GUID is used as the service template on the host which translates to a port in the guest. To customize your Service GUID simply change the first "00000000" to the port number desired. Ex: "00000ac9" is port 2761.

 // Hyper-V Socket Linux guest VSOCK template GUID
struct __declspec(uuid("00000000-facb-11e6-bd58-64006a7986d3")) VSockTemplate{};

/*
  * GUID example = __uuidof(VSockTemplate);
 * example.Data1 = 2761; // 0x00000AC9
 */

[totaam]

Kernel 6.4 adds:

*
* Microsoft Hyper-V guest support
*
Microsoft Hyper-V client drivers (HYPERV) [N/y/?] (NEW) ?

CONFIG_HYPERV:

Select this option to run Linux as a Hyper-V client operating
system.

Symbol: HYPERV [=n]
Type  : tristate
Defined at drivers/hv/Kconfig:5
  Prompt: Microsoft Hyper-V client drivers
  Depends on: X86 [=y] && X86_LOCAL_APIC [=y] && HYPERVISOR_GUEST [=y] || ACPI [=n] && ARM64 && !CPU_BIG_ENDIAN
  Location:
    -> Device Drivers
      -> Microsoft Hyper-V guest support
        -> Microsoft Hyper-V client drivers (HYPERV [=n])
Selects: PARAVIRT [=y] && X86_HV_CALLBACK_VECTOR [=y] && OF_EARLY_FLATTREE [=n]

[totaam]

As of this week, MSYS2 has moved to python 3.12 and everthing should be in place:

$ python3
Python 3.12.7 (main, Nov  3 2024, 19:54:34)  [GCC 14.2.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.AF_HYPERV
<AddressFamily.AF_HYPERV: 34>
>>> exit()

[totaam]

The commit above adds hyper-v socket parsing to xpra urls, so now we can do:

xpra attach hyperv://${vmid}:20/

Which will convert the serviceid to 00000014-facb-11e6-bd58-64006a7986d3 before calling socket.connect. (serviceid = 10 = 0x14)
Or you can use the full syntax (here, again with the same linux vsock vmid):

xpra attach hyperv://${vmid}:/00000014-facb-11e6-bd58-64006a7986d3/

I would quite like to make it easier to locate the vmid.
this answer suggests using hcsdiag list or (Get-VM -Name $VMName).Id.

It seems odd that we have to register a new application "in order to use Hyper-V sockets", since the protocol + vmid + serviceid should fully describe the endpoint. It feels like an unnecessary hoop to go through.
And since this requires administrator privileges, not easily scriptable either.

Information for setting up vsock connections is in #983

[totaam]

Tested and working as of 6.3-r36890, available here: https://xpra.org/beta/windows/

Will follow up in #4418.