Project 5: Linshen Xiao

README.md

@@ -3,26 +3,89 @@ WebGL Clustered Deferred and Forward+ Shading
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* LINSHEN XIAO
+* Tested on: **Google Chrome 61.0.3163.100** on Windows 10, Intel(R) Core(TM) i7-6700HQ CPU @ 2.60GHz, 16.0GB, NVIDIA GeForce GTX 970M (Personal computer)
+
+## Overview
+
+In this project, I implemented 2 different rendering methods, including Clustered Forward+ and Clustered Deferred. For Clustered Forward+, I build a data structure to keep track of how many lights are in each cluster and what their indices are render the scene using only the lights that overlap a given cluster. For Clustered Deferred, I reuse clustering logic from Clustered Forward+ Store vertex attributes in g-buffer, read g-buffer in a shader to produce final output. Also, Blinn-Phong shading & Toon shading effects and simple optimization by reduce the number of g-buffers are implemented.
+
+## Features
+
+* Clustered Forward+
+	* Slice the frustrum into 15 * 15 * 15 clusters, save the lights for each clusters and render the scene using only the lights that overlap the given cluster.
+	* Most difficult part of this is to define if one light is in a cluster or not. Instead of check every light's position for each cluster, a better way is to judge the distance of light position to the plane for x, y & z, to find the smallest and largest index cluster for each dimension that the distance between it and the light position is smaller than light radius, and define all these cluster overlapped by the light. For z, it is simple because you can just consider the distance of light position's z coordinate to the z coordinate of each cluster. For x and y, we just need to find the distance from the point to the line on 2D plane, which is a dot product of the light position's xz or yz coordinate to the normal of the zy or xz plane.
+* Clustered Deferred
+	* Store attributes (normal, position, color) in g-buffer, read g-buffer in a shader to produce final output.
+* Effects
+	* Blinn-Phong shading (diffuse + specular);
+	* Toon shading implemented;
+* Optimizations
+	* Optimized g-buffer format - reduce the number of g-buffers: I pack color, position and normal together into 2 vec4s: gbuffers[0]: [col.x, col.y, col.z, normal.x], gbuffers[1]: [pos.x, pos.y, pos.z, normal.y](Use 2-component normals, normal is in view space). We can reconstruct normal in view space and transform it back to world space by * u_invViewMatrix. However, the reconstructed normal may not be accurate enough.
+
+## Results
 
 ### Live Online
 
-[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
+[https://githublsx.github.io/Project5-WebGL-Clustered-Deferred-Forward-Plus](https://githublsx.github.io/Project5-WebGL-Clustered-Deferred-Forward-Plus/)
 
 ### Demo Video/GIF
 
-[![](img/video.png)](TODO)
+![](img/result.gif)
+
+### Effects:
+
+#### Blinn-Phong shading:
+
+![](img/blinnphong.gif)
+
+#### Toon Shading:
+
+![](img/toon.gif)
+
+### Bebug:
+
+| Albedo | Normal |
+|-------------|--------------------------------|
+|![](img/aldeo.png)|![](img/normal.png)|
+
+| Position | Depth |
+|-------------|--------------------------------|
+|![](img/position.png)|![](img/depth.png)|
+
+## Performance Analysis
+
+### Comparision of Forward, Clustered Forward+ & Clustered Deferred
+
+![](img/diagram.png)
+
+| LightNumber | Forward | Clustered Forward+ | Clustered Deferred |
+|-------------|---------|--------------------|--------------------|
+| 100         | 105     | 42                 | 32                 |
+| 200         | 182     | 77                 | 53                 |
+| 300         | 303     | 133                | 68                 |
+| 400         | 370     | 167                | 86                 |
+| 500         | 476     | 250                | 100                |
+
+(Timing in milliseconds)
+
+As the number of light increase, it take longer time to render. We can also clearly tell that Clustered Deffered > Clustered Forward+ > Forward on efficiency. Instead of testing each fragment for every light in the scene in Forward, Clustered Forward+ divide frustrum into clusters and for each fragment, decide which cluster it's in and then only computed the final color with the light inside cluster, which greatly accerlate the process.
+
+### Clustered Deferred with 3 & 2 g-buffers
 
-### (TODO: Your README)
+![](img/diagram3.png)
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+| LightNumber | Clustered Deferred 3 g-buffers | Clustered Deferred 2 g-buffers |
+|-------------|--------------------------------|--------------------------------|
+| 100         | 32                             | 30                             |
+| 200         | 53                             | 48                             |
+| 300         | 68                             | 62                             |
+| 400         | 86                             | 83                             |
+| 500         | 100                            | 97                             |
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+(Timing in milliseconds)
 
+We can see that the difference with Clustered Deferred with 3 or 2 g-buffers is not very obvious. This is because though the number of the g-buffers is reduced, Clustered Deferred with 2 g-buffers needs extra time to reconstruct the normal.
 
 ### Credits
 

src/init.js

@@ -1,5 +1,5 @@
 // TODO: Change this to enable / disable debug mode
-export const DEBUG = true && process.env.NODE_ENV === 'development';
+export const DEBUG = false && process.env.NODE_ENV === 'development';
 
 import DAT from 'dat-gui';
 import WebGLDebug from 'webgl-debug';

src/main.js

@@ -9,7 +9,7 @@ const CLUSTERED_FORWARD_PLUS = 'Clustered Forward+';
 const CLUSTERED_DEFFERED = 'Clustered Deferred';
 
 const params = {
-  renderer: CLUSTERED_FORWARD_PLUS,
+  renderer: CLUSTERED_DEFFERED,
   _renderer: null,
 };
 

src/renderers/clustered.js

@@ -1,9 +1,20 @@
-import { mat4, vec4, vec3 } from 'gl-matrix';
+import { mat4, vec4, vec3, vec2 } from 'gl-matrix';
 import { NUM_LIGHTS } from '../scene';
 import TextureBuffer from './textureBuffer';
 
 export const MAX_LIGHTS_PER_CLUSTER = 100;
 
+function getDistance(ratio, lightPos){
+  //only consider 2D space
+  //for y: yz plane; for x: xz plane;
+  let temp = Math.sqrt(1 + ratio*ratio);
+  let a1 = 1 / temp;
+  let a2 = -ratio*a1;
+  let normal = vec2.create();
+  vec2.set(normal, a1, a2);
+  return vec2.dot(lightPos, normal);
+}
+
 export default class ClusteredRenderer {
   constructor(xSlices, ySlices, zSlices) {
     // Create a texture to store cluster data. Each cluster stores the number of lights followed by the light indices
@@ -27,6 +38,116 @@ export default class ClusteredRenderer {
       }
     }
 
+    var zInterval = (camera.far - camera.near)/this._zSlices;
+    var yzRatio = Math.tan(camera.fov / 2.0 * Math.PI / 180.0) * 2.0;
+    var xzRatio = yzRatio * camera.aspect;
+    var xInterval = xzRatio / this._xSlices;
+    var yInterval = yzRatio / this._ySlices;
+    var xStart = - xzRatio / 2.0;
+    var yStart = - yzRatio / 2.0;
+
+    for(let lightIndex = 0; lightIndex < NUM_LIGHTS; ++lightIndex)
+    {
+      //get light position
+      let lightPos = vec4.create();
+      vec4.set(lightPos, scene.lights[lightIndex].position[0], scene.lights[lightIndex].position[1], scene.lights[lightIndex].position[2], 1.0);
+      vec4.transformMat4(lightPos,lightPos,viewMatrix);
+      lightPos[2] *= -1.0;
+      let lightRadius = scene.lights[lightIndex].radius;
+      let zmin; let zmax;
+      let ymin; let ymax;
+      let xmin; let xmax;
+      let distance;
+
+      //find zmin
+      for(zmin = 0; zmin < this._zSlices; ++zmin)
+      {
+        distance = lightPos[2]-(camera.near + zmin * zInterval);
+        if(distance < lightRadius)
+        {
+          zmin = Math.max(0, zmin-1);
+          break;
+        }
+      }
+
+      //find zmax
+      for(zmax = zmin + 1; zmax < this._zSlices; ++zmax)
+      {
+        distance = lightPos[2]-(camera.near + zmax * zInterval);
+        if(distance < -lightRadius)
+        {
+          break;
+        }
+      }
+
+      //find xmin
+      for(xmin = 0; xmin < this._xSlices; ++xmin)
+      {
+        let lightPosxz = vec2.create();
+        lightPosxz = vec2.set(lightPosxz, lightPos[0], lightPos[2]);
+        distance = getDistance(xStart + xmin * xInterval, lightPosxz);
+        if(distance < lightRadius)
+        {
+          xmin = Math.max(0, xmin-1);
+          break;
+        }
+      }
+
+      //find xmax
+      for(xmax = xmin + 1; xmax < this._xSlices; ++xmax)
+      {
+        let lightPosxz = vec2.create();
+        lightPosxz = vec2.set(lightPosxz, lightPos[0], lightPos[2]);
+        distance = getDistance(xStart + xmax * xInterval, lightPosxz);
+        if(distance < -lightRadius)
+        {
+          break;
+        }
+      }
+
+      //find ymin
+      for(ymin = 0; ymin < this._ySlices; ++ymin)
+      {
+        let lightPosyz = vec2.create();
+        lightPosyz = vec2.set(lightPosyz, lightPos[1], lightPos[2]);
+        distance = getDistance(yStart + ymin * yInterval, lightPosyz);
+        if(distance < lightRadius)
+        {
+          ymin = Math.max(0, ymin-1);
+          break;
+        }
+      }
+
+      //find ymax
+      for(ymax = ymin + 1; ymax < this._ySlices; ++ymax)
+      {
+        let lightPosyz = vec2.create();
+        lightPosyz = vec2.set(lightPosyz, lightPos[1], lightPos[2]);
+        distance = getDistance(yStart + ymax * yInterval, lightPosyz);
+        if(distance < -lightRadius)
+        {
+          break;
+        }
+      }
+
+      for (let z = zmin ; z < zmax; ++z) {
+        for (let y = ymin; y < ymax; ++y) {
+          for (let x = xmin; x < xmax; ++x) {
+            let i = x + y * this._xSlices + z * this._xSlices * this._ySlices;
+            let lightCount = this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, 0)];
+            lightCount++;
+            if(lightCount<MAX_LIGHTS_PER_CLUSTER)
+            {
+              this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, 0)] = lightCount;
+              let row = Math.floor(lightCount / 4);
+              let pixel = lightCount - row * 4;
+              this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, row) + pixel] = lightIndex;
+            }
+          }
+        }
+      }
+    }
+
     this._clusterTexture.update();
   }
 }

src/renderers/clusteredDeferred.js

@@ -8,8 +8,9 @@ import QuadVertSource from '../shaders/quad.vert.glsl';
 import fsSource from '../shaders/deferred.frag.glsl.js';
 import TextureBuffer from './textureBuffer';
 import ClusteredRenderer from './clustered';
+import {MAX_LIGHTS_PER_CLUSTER} from './clustered';
 
-export const NUM_GBUFFERS = 4;
+export const NUM_GBUFFERS = 3;
 
 export default class ClusteredDeferredRenderer extends ClusteredRenderer {
   constructor(xSlices, ySlices, zSlices) {
@@ -21,21 +22,27 @@ export default class ClusteredDeferredRenderer extends ClusteredRenderer {
     this._lightTexture = new TextureBuffer(NUM_LIGHTS, 8);
 
     this._progCopy = loadShaderProgram(toTextureVert, toTextureFrag, {
-      uniforms: ['u_viewProjectionMatrix', 'u_colmap', 'u_normap'],
+      uniforms: ['u_viewMatrix', 'u_viewProjectionMatrix', 'u_colmap', 'u_normap'],
       attribs: ['a_position', 'a_normal', 'a_uv'],
     });
 
     this._progShade = loadShaderProgram(QuadVertSource, fsSource({
       numLights: NUM_LIGHTS,
       numGBuffers: NUM_GBUFFERS,
+      numxSlices: xSlices,
+      numySlices: ySlices,
+      numzSlices: zSlices,
+      numMaxLightsPerCluster: MAX_LIGHTS_PER_CLUSTER
     }), {
-      uniforms: ['u_gbuffers[0]', 'u_gbuffers[1]', 'u_gbuffers[2]', 'u_gbuffers[3]'],
+      uniforms: ['u_gbuffers[0]', 'u_gbuffers[1]', 'u_gbuffers[2]', 'u_lightbuffer', 'u_clusterbuffer'
+    , 'u_cameraFar', 'u_cameraNear', 'u_screenWidth', 'u_screenHeight', 'u_viewMatrix', 'u_invViewMatrix', 'u_viewProjectionMatrix'],
       attribs: ['a_uv'],
     });
 
     this._projectionMatrix = mat4.create();
     this._viewMatrix = mat4.create();
     this._viewProjectionMatrix = mat4.create();
+    this._invViewMatrix = mat4.create();
   }
 
   setupDrawBuffers(width, height) {
@@ -108,6 +115,7 @@ export default class ClusteredDeferredRenderer extends ClusteredRenderer {
     mat4.invert(this._viewMatrix, camera.matrixWorld.elements);
     mat4.copy(this._projectionMatrix, camera.projectionMatrix.elements);
     mat4.multiply(this._viewProjectionMatrix, this._projectionMatrix, this._viewMatrix);
+    mat4.invert(this._invViewMatrix, this._viewMatrix);
 
     // Render to the whole screen
     gl.viewport(0, 0, canvas.width, canvas.height);
@@ -123,6 +131,7 @@ export default class ClusteredDeferredRenderer extends ClusteredRenderer {
 
     // Upload the camera matrix
     gl.uniformMatrix4fv(this._progCopy.u_viewProjectionMatrix, false, this._viewProjectionMatrix);
+    gl.uniformMatrix4fv(this._progCopy.u_viewMatrix, false, this._viewMatrix);
 
     // Draw the scene. This function takes the shader program so that the model's textures can be bound to the right inputs
     scene.draw(this._progCopy);
@@ -154,6 +163,25 @@ export default class ClusteredDeferredRenderer extends ClusteredRenderer {
     gl.useProgram(this._progShade.glShaderProgram);
 
     // TODO: Bind any other shader inputs
+    gl.uniformMatrix4fv(this._progShade.u_viewMatrix, false, this._viewMatrix);
+    gl.uniformMatrix4fv(this._progShade.u_invViewMatrix, false, this._invViewMatrix);
+    gl.uniformMatrix4fv(this._progShade.u_viewProjectionMatrix, false, this._viewProjectionMatrix);
+
+    // Set the light texture as a uniform input to the shader
+    gl.activeTexture(gl.TEXTURE4);
+    gl.bindTexture(gl.TEXTURE_2D, this._lightTexture.glTexture);
+    gl.uniform1i(this._progShade.u_lightbuffer, 4);
+
+    // Set the cluster texture as a uniform input to the shader
+    gl.activeTexture(gl.TEXTURE5);
+    gl.bindTexture(gl.TEXTURE_2D, this._clusterTexture.glTexture);
+    gl.uniform1i(this._progShade.u_clusterbuffer, 5);
+
+    // Bind any other shader inputs
+    gl.uniform1f(this._progShade.u_screenWidth, canvas.width);
+    gl.uniform1f(this._progShade.u_screenHeight, canvas.height);
+    gl.uniform1f(this._progShade.u_cameraFar, camera.far);
+    gl.uniform1f(this._progShade.u_cameraNear, camera.near);
 
     // Bind g-buffers
     const firstGBufferBinding = 0; // You may have to change this if you use other texture slots

src/renderers/clusteredForwardPlus.js

@@ -6,6 +6,7 @@ import vsSource from '../shaders/clusteredForward.vert.glsl';
 import fsSource from '../shaders/clusteredForward.frag.glsl.js';
 import TextureBuffer from './textureBuffer';
 import ClusteredRenderer from './clustered';
+import { MAX_LIGHTS_PER_CLUSTER } from './clustered';
 
 export default class ClusteredForwardPlusRenderer extends ClusteredRenderer {
   constructor(xSlices, ySlices, zSlices) {
@@ -16,8 +17,13 @@ export default class ClusteredForwardPlusRenderer extends ClusteredRenderer {
 
     this._shaderProgram = loadShaderProgram(vsSource, fsSource({
       numLights: NUM_LIGHTS,
+      numxSlices: xSlices,
+      numySlices: ySlices,
+      numzSlices: zSlices,
+      numMaxLightsPerCluster: MAX_LIGHTS_PER_CLUSTER
     }), {
-      uniforms: ['u_viewProjectionMatrix', 'u_colmap', 'u_normap', 'u_lightbuffer', 'u_clusterbuffer'],
+      uniforms: ['u_viewMatrix', 'u_viewProjectionMatrix', 'u_colmap', 'u_normap', 'u_lightbuffer', 'u_clusterbuffer'
+    ,'u_cameraFar','u_cameraNear','u_screenWidth','u_screenHeight'],
       attribs: ['a_position', 'a_normal', 'a_uv'],
     });
 
@@ -64,6 +70,7 @@ export default class ClusteredForwardPlusRenderer extends ClusteredRenderer {
 
     // Upload the camera matrix
     gl.uniformMatrix4fv(this._shaderProgram.u_viewProjectionMatrix, false, this._viewProjectionMatrix);
+    gl.uniformMatrix4fv(this._shaderProgram.u_viewMatrix, false, this._viewMatrix);
 
     // Set the light texture as a uniform input to the shader
     gl.activeTexture(gl.TEXTURE2);
@@ -76,6 +83,10 @@ export default class ClusteredForwardPlusRenderer extends ClusteredRenderer {
     gl.uniform1i(this._shaderProgram.u_clusterbuffer, 3);
 
     // TODO: Bind any other shader inputs
+    gl.uniform1f(this._shaderProgram.u_screenWidth, canvas.width);
+    gl.uniform1f(this._shaderProgram.u_screenHeight, canvas.height);
+    gl.uniform1f(this._shaderProgram.u_cameraFar, camera.far);
+    gl.uniform1f(this._shaderProgram.u_cameraNear, camera.near);
 
     // Draw the scene. This function takes the shader program so that the model's textures can be bound to the right inputs
     scene.draw(this._shaderProgram);