VirtualHPS

Looking Glass HoloPlayService replacement for virtual machines without direct access to the hardware.

Content

• Introduction
• HoloPlay API Variants
• Setup
• Alternative use-cases
• Debugging
• Windows-based holo apps

The problem

The official HoloPlay Service relies on a direct, low-level connection to the screen in order pull its lenticular lens calibration values over USB/HDMI. When we run HoloPlay Service in a virtual machine, it cannot reach the hardware, as the Looking Glass display is presented to the guest OS as a Generic PnP Monitor without any special properties.

The solution

VirtualHPS mirrors the API endpoints of HoloPlay Service, but with the option to use manually supplied calibration data. That means it works also in VMs, and it is currently the only way to play Windows-based holo apps on Apple Silicon Macs.

HoloPlay API Variants

1) Legacy ✅

This method is used by Unity apps built using the oldest HoloPlay SDK. Each app has means of direct communication with the display, something that was since then offloaded to HoloPlay Service. We cannot simulate the display itself, but luckily there is a built-in fallback (probably for debug reasons) where these apps check for the existence of a file C:\LKG_calibration\visual.json and, if found, use that instead. VirtualHPS makes sure that this file always matches the calibration values of the currently used display. You can identify apps using this method by the launch dialog that asks you to manually select the display and resolution.

2) NNG ✅

This is the current way that modern Unity, UE and HoloPlayCore apps get access to calibration data. The NNG-based CBOR-encoded inter-process endpoint is available at ipc:///tmp/holoplay-driver.ipc. Besides the lens calibration data this endpoint also provides render window location, which is why modern apps like the HoloPlay Studio don't ask you to manually choose a display.

3) Standard websocket ✅

This enpoint exists to enable Holoplay.js/Three.js integration in web browsers. I was debating whether to include this in VirtualHPS as web apps can be run on any platform and therefore there is no obvious benefit to running them in a virtual machine. Eventually I included it because it didn't take much effort to implement and it might be useful to somebody. This API endpoint runs at the address ws://127.0.0.1:11222/.

3) NNG websocket ☑️

I wasn't able to gather much about the inner workings or the purpose behind this special websocket, besided the fact that it runs at ws://127.0.0.1:11222/driver and uses the rep.sp.nanomsg.org websocket subprotocol. Currently not implemented in VirtualHPS.

Setup

Host system setup

• Verify that you have the "Default for Display" scaling option selected for the Looking Glass display in macOS' Display Preferences
• VM hypervisor capable of using multiple monitors. Tried and tested using Parallels Desktop.
• In the Parallels VM options enable:
  • Options > Full Screen > Use all displays in full screen
  • Hardware > Graphics > Best for external displays
• For non-Portait screens you need to install HoloPlay Service in order to be able to pull the calibration data (only needed for the initial setup).

Video guide

Portrait setup

1. Once your Windows VM is booted up, pass through the Portrait's mass storage device. It contains a calibration file from factory that VirtualHPS utilizes. No further setup needed.

Non-Portrait setup

1. Non-Portrait LG screens do not have a mass storage, therefore we have to manually supply calibration values on first run of VirtualHPS.
2. To get the calibration values, make sure that HoloPlay Service on your computer can see the display, and then paste the following code into your web browser's JavaScript console:

var ws = new WebSocket('ws://localhost:11222/'); ws.onmessage = function(event){alert(event.data)}

Note: Connection to local websocket doesn't work in Safari, so use Firefox, Brave or any other Chromium-based browser.

Alternatively, you can visit https://eka.hn/calibration_test.html and copy the provided JSON straight from the website.

3. On first launch of VirtualHPS enter the copied calibration data.

Alternative use-cases

My primary motivation for developing VirtualHPS was to allow the use of Looking Glass displays with applications running within virtual machines. However, there are at least two other cases where it might come handy:

1) Render farms

You might want to pre-render large amount of content for your display. VirtualHPS v1.0.1 added an option to render to a regular display, so you no longer have to have Looking Glass connected to the rendering machine. This allows to render volumetric scenes on multiple machines at the same time.

2) Multiple display management

Let's say you are a museum with 20 Looking Glass displays all over the exhibition floor. These displays play pre-rendered videos from network-connected thin clients. Every time you wanted to update the content in the past, you had to physically fetch each of the 20 screens and connect them to your computer one by one in order to have the calibration data available. With VirtualHPS that is no longer necessary. Create a calibration.txt file once for each of the displays and use them to render the content without the need to plug in each of the displays.

Debugging

Launch VirtualHPS.exe from a command line to receive debug messages.

Windows-based holo apps

If you are looking for some cool Windows holo apps, check out the following links:

1. Made With website
2. Library App store
3. Official Looking Glass Factory Discord server

Disclaimer

This work is not affiliated with Looking Glass Factory, Inc.

HoloPlay, Looking Glass, Portait and other potential trademarks belong to their respective owners.