RecordingMain.mp4
After some deliberation (primarily considering various artbooks and manga I have with me, and some artists like Yoshinori Kanada or Mortis Ghost who I think could be very interesting as inspiration yet it was surprisingly hard to find high-resolution images of the works I had in mind from them), I decided to try styling my project after my own sketches, such as those shown below. I don't normally get around to coloring when I draw, so as preparation I did some sketches out in colored pencil to get a better idea of how my sketch-coloring looks.
 |
 |
 |
|---|
Some details I wanted to potentially include:
- Very 'rough'/'loose' outlining—especially on things like hair I tend to have very variable weight and often gaps where the outline would not completely connect.
- Variable directions of hatching for filling in surfaces—often I seem to default to diagonal down-left strokes but depending on the shape or texture of the object I may vary (e.g. going along the flow of hair, or doing more curved strokes on rounded objects).
- Coloring shadows differently depending on the object casting it—e.g. if two different people cast a shadow on the same floor, they would have different colors of shadow. I'm not sure this is a consistent thing I've done before, but I did it some in my practice coloring for this and think it could be neat.
- Reflection after creating the shaders: I didn't end up directly including this concept (not sure Unity's pipeline is conducive to doing this) but I did use it as a basis for the variety of shadow colors used in the scene (note e.g. the blue shadows of the pages vs. the red shadows the books cast on the table.
- Relatedly, possibly having a bit of a seam around shadows to reflect my somewhat hasty coloring.
- Stronger saturation near contours—I think in the past as well I've been in the habit of coloring too lightly, which I've done a bit here, but noticed I tend to make bolder marks when I'm trying to basically highlight the form of something. See the coat in the first image above for instance, where the parts meant to emphasize the fabric folding about the body have darker lines.
- Didn't directly incorporate this concept, but was the inspiration to decide to incorporate rim lighting as that's a similar sort of effect.
Features added initially include multiple light support and rim highlighting, with the rim lighting being dependent on the presence of the lights throughout the scene, such that the color of the rim depends on the color of the incoming light beyond the object.
I've added a shadow texture that I hatched on paper then photographed, grayscaled, made tileable, and (within the shader graph) applied smoothstep to. I tried a few textures with varying density of lines but was only satisfied with the lighter one so far, but I'm hoping to potentially revisit the concept of heavier shadow textures and use some sort of transition between them if I can find a look I like enough.
My concept for the special surface shader is one that combines hatching in various directions, with the pattern distorting over time. I used Voronoi-based noise to divide the surface UVs into cells, such that per each cell I obtain a different value which is used to set the rotation of a hatching texture in screen space, which is as in my shadow effect smoothstepped and used to lerp color values. The input to the Voronoi noise function is perturbed by taking the result of a Perlin noise function which takes in the surface UVs and time (being a 3D noise function, but with two spatial dimensions and one temporal dimension as input), mapping it to be an angle of rotation for a vector, and adding that vector to the UVs being passed into the Voronoi function.
RecordingRimLight.mp4
The above video shows a clearer demonstration of the material properties, particularly the rim lighting, which are not necessarily evident in the wider view from the main turnaround video. A clearer view of the aforementioned special hatching effect can be seen in the close-up video in the following section.
RecordingClose.mp4
Two sets of outlines are created each using a sobel filter (drawing an outline if the total magnitude of the result is past a threshold value), one based on depth and one based on normals. The input coordinates to each of these are perturbed in a random direction with a random magnitude via a value from a Worley-based noise function, with the two outlines having different perturbations. Perlin noise is also used to add some variance to the thickness used for the sampling in the sobel filters. This gives the effect of inconsistent, gap-filled outlines as I had desired. The input to the Worley noise includes 3 dimensions: screen UV as well as time, with time being scaled and floored such that it gives abrupt leaps in input. This gives the lines a jumpy nature, which emphasizes the feeling of "sketchiness" in the result.
The colors of the scene are interpolated with two textures derived from photographs—one being a blank piece of paper, and the other a section of paper shaded with pencil; the colors from the scene become a tint for the pencil shading. Depth is used to interpolate these two textures such that especially far away areas are less colored, meaning that the background of the overall scene appears as a blank paper texture. I also included an effect where very lightly colored regions would be further interpolated with the blank texture, though the values I ultimately used for this scene only apply this feature very slightly.
The scene I applied this effect to is one of a desk with a pop-up book and various tiny clay doll-like figures spread about it. This is a reuse of models I previously created for a 3D modeling course, which were also based on sketches I made (as in the image below) and hence felt relevant as a showcase for the stylization.
Pressing the space bar alternates between the main "colored pencil" style and a monochrome "graphite pencil" style, with the exception that the "special" shader materials are kept as-is. This replaces the toon shading used with a shader that takes the same diffuse value as used for the color thresholds and uses it to interpolate between a set of six textures, all photos of pencil hatching of varying darkness (see e.g. the coffee mug in the above video for an example of the gradient). The post-process shader is also altered, such that the pencil shading texture it applies is no longer used (as it would be redundant with the hatching already drawn in the scene) and the blank background texture is replaced with the one used in this set of six textures (while they're both pictures of the same real-world paper, the lighting in the photos is inconsistent and hence this blank image better interpolates with the other shades used).
In this assignment, you will use a 2D concept art piece as inspiration to create a 3D Stylized scene in Unity. This will give you the opportunity to explore stylized graphics techniques alongside non-photo-realistic (NPR) real-time rendering workflows in Unity.
 |
 |
|---|---|
| 2D Concept Illustration | 3D Stylized Scene in Unity |
- Picking a Piece of Concept Art
- Interesting Shaders
- Outlines
- Full Screen Post Process Effect
- Creating a Scene
- Interactivity
- Extra Credit
After forking the repo, take a moment to watch this brief HW/Base Project Overview which goes over things that you're expected to bring over from the lab, and etc.
Choose a simple illustration to guide your stylization. Choose a relatively simple piece of art THAT INCLUDES OUTLINES. You might want to look through the rest of the homework instructions before committing to one. Here are some examples of styles that will work well. Feel free to choose one of these, but we encourage your to pick your own.
Disclaimer: Don't forget to identify and credit the artist who created the concept art : )
Emma Koch, an amazing 3D artist I happened to stumble upon on ArtStation produces incredible 2D-esque 3D art pieces. Some of the references I picked above were inspired directly from her work. I'd definitely check out her artstation for any inspiraiton if you want some! Link
Let's create some custom surface shaders for the objects in your scene, inspired by your concept art!
Take a moment to think about the main characteristics that you see in the shading of your concept art. What makes it look appealing/aesthetic?
- Is it the color palette? How are the different colors blending into each other? Is there any particular texture or pattern you notice?
- Are there additional effects such as rim or specular highlights?
- Are there multiple lights in the scene?
These are all things we want you to think about before diving into your shaders!
- Improved Surface Shader
- Create a surface shader inspired by the surface(s) in your concept art. Use the three tone toon shader you created from the Stylization Lab as a starting point to build a more interesting shader that fulfills all of the following requirements:
-
Multiple Light Support
- Follow the following tutorial to implement multiple light support.
-
Additional Lighting Feature
- Implement a Specular Highlight, Rim Highlight or another similarly interesting lighting-related effect
-
Interesting Shadow
- Create your own custom shadow texture!
- You can use whatever tools you have available! Digital art (Photoshop, CSP, Procreate, etc.), traditional art (drawing on paper, and then taking a photo/scan)-you have complete freedom!
- Make your texture seamless/tesselatable! You can do this through the following online tool: https://www.imgonline.com.ua/eng/make-seamless-texture.php
- Modify your shadows using this custom texture in a similar way to Puzzle 3 from the Lab
- Now, instead of using screen position, use the default object UVs!
- In the 3rd Puzzle of the Lab, the shadow texture was sampled using the Screen Position node. This time, let's use the object's UV coordinates to have the shadows conform to geometry. Hint: To get a consistent looking shadow texture scale across multiple objects, you're going to want some exposed float parameter, "Shadow Scale," that will adjust the tiling of the shadow texture. This will allow for per material control over the tiling of your shadow texture.
-
- Notice how in this artwork by Emma Koch, Link's shadow does not remain fixed in screen space as it is drawn via object UV coordinates.
- Create your own custom shadow texture!
-
Accurate Color Palette
- Do your best to replicate the colors/lighting of your concept art!
-
- Create a surface shader inspired by the surface(s) in your concept art. Use the three tone toon shader you created from the Stylization Lab as a starting point to build a more interesting shader that fulfills all of the following requirements:
- Special Surface Shader
- Let's get creative! Create a SPECIAL second shader that adds a glow, a highlight or some other special effect that makes the object stand out in some way. This is intended to give you practice riffing on existing shaders. Most games or applications require some kind of highlighting: this could be an effect in a game that draw player focus, or a highlight on hover like you see in a tool. If your concept art doesn't provide a visual example of what highlighting could look like, use your imagination or find another piece of concept art. Duplicate your shader to create a variant with an additional special feature that will make the hero object of your scene stand out. Choose one of the following three options:
-
Option 1: Animated colors -
- The above is a simple example of what an animated surface shader might do, eg flash through a bunch of different colors. Using at least two toolbox functions, animate some aspect of the surface shader to create an eye-catching effect. Consider how procedural patterns, the screen space position and noise might contribute.
- Useful tips to get started:
- Use the Time node in Unity's shader graph to get access to time for animation. Consider using a Floor node on time to explore staggered/stepped interpolation! This can be really helpful for selling the illusion of the animation feeling handdrawn.
-
Option 2: Vertex animation
- Similar to the noise cloud assignment, modify your object shader to animate the vertex positions of your object, eg. making an object sway or bob up and down to make it stand out. You should be able to figure out how to do this given the walkthrough so far, but if you need addition help, check out this tutorial.
-
Option 3: Another Custom Effect Tailored to your Concept Art
- If you'd like to do an alternative effect to Option 1, just make sure that your idea is roughly similar in scope/difficulty. Feel free to make an EdStem post or ask any TA to double check whether your effect would be sufficient.
-
- Let's get creative! Create a SPECIAL second shader that adds a glow, a highlight or some other special effect that makes the object stand out in some way. This is intended to give you practice riffing on existing shaders. Most games or applications require some kind of highlighting: this could be an effect in a game that draw player focus, or a highlight on hover like you see in a tool. If your concept art doesn't provide a visual example of what highlighting could look like, use your imagination or find another piece of concept art. Duplicate your shader to create a variant with an additional special feature that will make the hero object of your scene stand out. Choose one of the following three options:
Make your objects pop by adding outlines to your scene!
Specifically, we'll be creating Post Process Outlines based on Depth and Normal buffers of our scene!
-
Render Features! Render Features are awesome, they let us add customizable render passes to any part of the render pipeline. To learn more about them, first, watch the following video which introduces an example usecase of a renderer feature in Unity:
-
Next, let's explore the HW base code briely, and specifically, learn more about the "Full Screen Feature" that's included as part of your base project. There's a small part missing from "Full Screen Feature.cs" that's preventing it from applying any type of full screen shader to the screen. Your job is to solve this bug and in the process, learn how to create a Full Screen Shadergraph, and then have it actually affect the game view! Watch the following video to get a deeper break down of the Render Feature's code and some hints on what the solution may be.
-
Using what we've learnt about Render Features/URP as a base, let's now get access to the Depth and Normal Buffers of our scene!
- Unity's Universal Render Pipeline actually already provides us with the option to have a depth buffer, and so obtaining a depth buffer is a very simple/trivial process.
- This is not the case for a Normal Buffer, and thus, we need a render feature to render out the scene's normals into a render texture. Since the render feature for this has too much syntax specific fluff that's too Unity heavy and not very fun, I've provided a working render feature that renders objects' normals into a render texture in the /Render Features folder, called the "Normal Feature." There is also a shader provided, "Hidden/Normal Copy" or "Normal Copy.shader."
- Your task is to add the Normal Feature to the render pipeline, make a material off of the Normal Copy shader and then plug it into the Normal Feature, and finally, connect the render texture called "Normal Buffer" located in the "/Buffers" directory as the destination target for the render feature.
- Set the resolution of the Normal Buffer render texture to be equal to your game window resolution.
- Your task is to add the Normal Feature to the render pipeline, make a material off of the Normal Copy shader and then plug it into the Normal Feature, and finally, connect the render texture called "Normal Buffer" located in the "/Buffers" directory as the destination target for the render feature.
- Watch the following video for clarifications on both of these processes, and also, how to actually access and read the depth and normal buffers once we've created them.
-
Finally, using everything you've learnt about Render Features alongside the fact that we now have proper access to both Depth and Normal Buffers, let's create a Post Process Outline Shader!
-
We STRONGLY RECOMMEND watching at least one of these Incredibly Useful Tutorials before getting started on Outlines:
- NedMakesGames
- Robin Seibold
- Alexander Ameye
- Important Clarification/Note on the Tutorials:
- You will quickly notice after watching/reading any of these tutorials that many of them use a Render Feature to render out a single DepthNormals Texture that encodes both depth and normal information into a single texture. This optimization saves on performance but results in less accurate depth or normals information and is overall more confusing for a first time experience into Render Features. Thus, for this assignment, we will just be sticking to our approach of having separate Depth and Normal buffers.
-
Next, we will create a basic Depth and Normal based outline prototype that produces black outlines at areas of large depth and normal difference across the screen. - Explore different kinds of edge detection methods, including Sobel and Robert's Cross filters - Make sure the outline has adjustable parameters, such as width.
-
Let's get creative! Modify your outline to be ANIMATED and to have an appearance that resembles the outlines in your concept art / OR, if the outlines in your concept art are too plain, try to make your outline resemble crayon/pencil sketching/etc.
- Use your knowledge of toolbox functions to add some wobble, or warping or noise onto the lines that changes over time.
- In my example below, you might be able to notice that the internal Normal Buffer based edges actually don't have any warping/animation. I did this intentionally because I wanted the final look to still have some kind of structure. Thus, by doing the depth and normal outlines in separate passes, I'm able to have a variety of animated/non-animated outlines composited together : ) !
-
-
(OPTIONAL) If you're not satisfied with the look of your outlines and are looking for an extra challenge, after implementing depth/normal based post processing, you may explore non-post process techniques such as inverse hull edge rendering for outer edges to render bolder, more solid looking outlines for a different look.
- Check out Alexander Ameye's article on alternative methods of outline rendering in Unity: See Here
We're nearing the end!
Ok, now regardless of what your concept art looks like, using what you know about toolbox functions and screen space effects, add an appealing post-process effect to give your scene a unique look. Your post processing effect should do at least one of the following.
- A vingette that darkens the edges of your images with a color or pattern
- Color / tone mapping that changes the colorization of your renders. Here's some basic ideas, but please experiment
- A texture to make your image look like it's drawn on paper or some other surface.
- A blur to make your image look smudged.
- Fog or clouds that drift over your scene
- Whatever else you can think of that complements your scene!
Note: This should be easily accomplishable using what you should have already learnt about working with Unity's Custom Render Features from the Outline section!
Using Unity's controls, create a SUPER BASIC scene with a few elements to show off your unique rendering stylization. Be sure to apply the materials you've created. Please don't go crazy with the geometry -- then you'll have github problems if your files are too large. See here.
Note that your modelling will NOT be graded at all for this assignment. It is NOT expected that your scene will be a one-to-one faithful replica of your concept art. You are STRONGLY ENCOURAGED to find free assets online, even if they don't strongly resemble the geometry/objects present in your concept art. (TLDR; If you choose to model your own geometry for this project, be aware of the time-constraint and risk!)
Some example resources for finding 3D assets to populate your scene With:
As a finishing touch, let's show off the fact that our scene is rendered in real-time! Please add an element of interactivity to your scene. Change some major visual aspect of your scene on a keypress. The triggered change could be
- Party mode (things speed up, different colorization)
- Memory mode (different post-processing effects to color you scene differently)
- Fanart mode (different surface shaders, as if done by a different artist)
- Whatever else you can think of! Combine these ideas, or come up with something new. Just note, your interactive change should be at least as complex as implementing a new type of post processing effect or surface shader. We'll be disappointed if its just a parameter change. There should be significant visual change.
- Create at least one new material to be swapped in using a key press
- Create and attach a new C# script that listens for a key press and swaps out the material on that key press. Your C# script should look something like this:
public Material[] materials;
private MeshRenderer meshRenderer;
int index;
void Start () {
meshRenderer = GetComponent<MeshRenderer>();
}
void Update () {
if (Input.GetKeyDown(KeyCode.Space)){
index = (index + 1) % materials.Count;
SwapToNextMaterial(index);
}
}
void SwapToNextMaterial (int index) {
meshRenderer.material = materials[index % materials.Count];
}
- Attach the c# script as a component to the object(s) that you want to change on keypress
- Assign all the relevant materials to the Materials list field so you object knows what to swap between.
Explore! What else can you do to polish your scene?
- Implement Texture Support for your Toon Surface Shader with Appealing Procedural Coloring.
- I.e. The procedural coloring needs to be more than just multiplying by 0.6 or 1.5 to decrease/increase the value. Consider more deeply the relationship between things such as value and saturation in artist-crafted color palettes?
- Add an interesting terrain with grass and/or other interesting features
- Implement a Custom Skybox alongside a day-night cycle lighting script that changes the main directional light's colors and direction over time.
- Add water puddles with screenspace reflections!
- Any other similar level of extra spice to your scene : ) (Evaluated on a case-by-case basis by TAs/Rachel/Adam)
- Video of a turnaround of your scene
- A comprehensive readme doc that outlines all of the different components you accomplished throughout the homework.
- All your source files, submitted as a PR against this repository.
- Link to all my videos:
- Lab Video
- Very Helpful Creators/Videos from the internet