Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebXR AR Light Estimation #2046

Merged
merged 7 commits into from
May 19, 2020

Conversation

Maksims
Copy link
Contributor

@Maksims Maksims commented May 3, 2020

WebXR Light Estimation is still in incubation, but already has experimental implementation in Chrome Canary 84 on Android, under the chrome://flags#webxr-incubations flag.

It provides estimation of lighting by underlying AR systems.
Current API has a spec for three paths:

  1. Reflection CubeMap
  2. L2 Spherical Harmonics
  3. Most prominent directional light

This PR implements basic interface to access most basic light estimation information: most prominent directional light. And Spherical Harmonics for ambient light.

We do not have support for Spherical Harmonics in engine, sadly. But the is internal shader code, that actually generates SH from IBL cubemap, and uses it as ambient light (exactly what we need). But it is not exposed as API.

Reflection cubemap will come as a separate PR.

New APIs:

// pc.XrManager
app.xr.lightEstimation // interface to access light estimation features

// pc.XrLightEstimation
lightEstimation.supported // true if device supports such feature. This information is only available during started XR session, currently it is not possible to find out if it is supported before starting XR session.
lightEstimation.available // true when light estimation data is available
lightEstimation.intensity // intensity of most prominent directional light
lightEstimation.color // color of most prominent directional light
lightEstimation.rotation // rotation of most prominent directional light
lightEstimation.sphericalHarmonics // Float32Array data, with 27 values, 9 of vec3's. Which represents L2 Spherical Harmonics

lightEstimation.start() // request light estimation information and start updating internal data. It will become available sometime later, and `available` property can be used to find out when, as well as `available` event.
lightEstimation.end() // stop light estimation

lightEstimation.on('available', function() { }); // fired when light estimation becomes available
lightEstimation.on('error', function(ex) { }); // fired when light estimation failed to start

Basic Example:

Initialize:

// start light estimation when XR session started
app.xr.on('start', function() {
    if (! app.xr.lightEstimation.supported)
        return; // light estimation not supported

    app.xr.lightEstimation.start();
});

On each update:

var lightEstimation = app.xr.lightEstimation;

if (lightEstimation.available) {
    entity.light.intensity = lightEstimation.intensity;
    entity.light.color = lightEstimation.color;
    entity.setRotation(lightEstimation.rotation);
}

Applying Spherical Harmonics:

if (lightEstimation.available) {
    material.ambientSH = lightEstimation.sphericalHarmonics;
    material.update();
}

Sample project for test

PSX_20200507_153358

I confirm I have signed the Contributor License Agreement.

@willeastcott willeastcott requested a review from a team May 4, 2020 00:31
@willeastcott willeastcott self-assigned this May 4, 2020
@willeastcott willeastcott added area: xr XR related issue enhancement labels May 4, 2020
@willeastcott willeastcott added this to the v1.128.0 milestone May 5, 2020
@willeastcott willeastcott merged commit 15e0e74 into playcanvas:master May 19, 2020
@Maksims Maksims deleted the webxr-light-estimation branch January 22, 2024 11:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area: xr XR related issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants