Compositing

Compositing is a fundamental technique in image synthesis. Conceptually, every write to a buffer that is interpreted as an image can fall in this category. I can highly recommend reading the paper "Compositing Digital Images" by Thomas Porter and Tom Duff, some of the OGs of computer graphics.

Blending in WebGL

In OpenGL terminology "blending" means that the output of the fragment shader also depends on the currently stored fragment. This behavior can be enabled by calling gl.enable(gl.BLEND). There are multiple options for the blending functionalities. When using blending with gl.blendFunc(sfactor, dfactor) the output will be calculated as: $$\mathbf{O} = \mathbf{S} _f \circ \mathbf{S} + \mathbf{D} _f \circ \mathbf{D}$$ where:

• $\mathbf{O}$ the output color vector
• $\mathbf{S}$ the source vector - the output color of the fragment shader
• $\mathbf{D}$ the destination vector - the currently stored color
• $\mathbf{S}_f$ the source factor vector
• $\mathbf{D}_f$ the destination factor vector
• $\circ$ the hadamarad product operator for vectors
• $\mathbf{S}_f$ and $\mathbf{D}_f$ are one of:

Constant	Factor RGB	Factor A
gl.ZERO	$\mathbf{0}$	$0$
gl.ONE	$\mathbf{1}$	$1$
gl.SRC_COLOR	$\mathbf{S} _{RGB}$	$\mathbf{S} _{A}$
gl.ONE_MINUS_SRC_COLOR	$\mathbf{1} - \mathbf{S} _{RGB}$	$1 - \mathbf{S} _{A}$
gl.DST_COLOR	$\mathbf{D} _{RGB}$	$\mathbf{D} _{A}$
gl.ONE_MINUS_DST_COLOR	$\mathbf{1} - \mathbf{D} _{RGB}$	$1 - \mathbf{D} _{A}$
gl.SRC_ALPHA	$\mathbf{S}_A$	$\mathbf{S}_A$
gl.ONE_MINUS_SRC_ALPHA	$\mathbf{1} - \mathbf{S}_A$	$1 - \mathbf{S}_A$
gl.DST_ALPHA	$\mathbf{D}_A$	$\mathbf{D}_A$
gl.ONE_MINUS_DST_ALPHA	$\mathbf{1} - \mathbf{D}_A$	$1 - \mathbf{D}_A$
gl.CONSTANT_COLOR	$\mathbf{C} _{RGB}$	$\mathbf{C} _{A}$
gl.ONE_MINUS_CONST_COLOR	$\mathbf{1} - \mathbf{C} _{RGB}$	$1 - \mathbf{C} _{A}$
gl.CONTANT_ALPHA	$\mathbf{C} _{A}$	$\mathbf{C} _{A}$
gl.ONE_MINUS_CONSTANT_ALPHA	$\mathbf{1} - \mathbf{C} _{A}$	$1 - \mathbf{C} _{A}$
gl.SRC_ALPHA_SATURATE	$\min( \mathbf{S}_A, \mathbf{1} - \mathbf{D}_A)$	$1$

The CONSTANT_COLOR can be set with gl.blendColor(r, g, b, a). The operation by which the source and destination are combined does not have to be addition. By using gl.blendEquation(mode) it can be set to one of

mode	function
gl.FUNC_ADD	$\mathbf{S} _{\circ} + \mathbf{D} _{\circ}$
gl.FUNC_SUBTRACT	$\mathbf{S} _{\circ} - \mathbf{D} _{\circ}$
gl.FUNC_REVERSE_SUBTRACT	$\mathbf{D} _{\circ} - \mathbf{S} _{\circ}$
gl.MIN	$\min(\mathbf{S}, \mathbf{D})$
gl.MAX	$\max(\mathbf{S}, \mathbf{D})$

where $\mathbf{S} _{\circ} = \mathbf{S} _{f} \circ \mathbf{S}$ and $\mathbf{D} _{\circ} = \mathbf{D} _{f} \circ \mathbf{D}$. This extends the equation to

$$\mathbf{O} = \mathbf{func}( \mathbf{S}_f \circ \mathbf{S}, \mathbf{D}_f \circ \mathbf{D})$$ for gl.FUNC_ADD, gl.FUNC_SUBTRACT andgl.FUNC_REVERSE_SUBTRACT and to $$\mathbf{O} = \mathbf{func}(\mathbf{S}, \mathbf{D})$$ for gl.MIN and gl.MAX.

OpenGL and WebGL also provide the option to set the factors and function separately for the alpha component by using blendFuncSeparate(srcRGB, dstRGB, srcAlpha, dstAlpha) and blendEquationSeparate(modeRGB, modeAlpha) respectively. This extends the equation to

$$\mathbf{O} = (\mathbf{func} _{RGB}( \mathbf{S} _{f _{RGB}} \circ \mathbf{S} _{RGB}, \mathbf{D} _{f _{RGB}} \circ \mathbf{D} _{RGB}), \mathrm{func}_A( \mathbf{S} _{f _A}\mathbf{S}_A, \mathbf{D} _{f _A}\mathbf{D}_A))$$

or a respective form for gl.MIN and gl.MAX without the factor. This leaves us with $15 \cdot 15 \cdot 5 \cdot 15 \cdot 15 \cdot 5 = 1.265.625$ possible combinations for blending parameters, many of them producing the same output. In practice most of them do not make any sense and even worse: those are still not enough to produce the all essential blending operations.

Blend Over

The blend over operation is the "natural" operation when layering one transparent image over another. There are a thew cases to consider with the blend over operation.

Straight vs Premultiplied Alpha

Color Model	RGB	A
straight alpha	the albedo of the pixel	the opacity of this layer
premultiplied alpha	the emission of the pixel	the occlusion of the layer beneath

While premultiplied alpha is less intuitive, it has some major advantages, as well mathematically and conceptionally, as we will see in the next sections.

Straight Alpha

With straight alpha the RGB components represent the albedo of the surface and the alpha component represents the opacity of the surface. That is a relatively intuitive representation as it resembles the idea of thin colored plastic sheets.

Opaque Destination

If the alpha of the destination is known to be $1$ the mathematical model is $$\mathbf{O} = (\mathbf{S}_A \mathbf{S} _{RGB} + (1 - \mathbf{S}_A) \mathbf{D} _ {RGB}, 1)$$ which translates to

gl.blendEquation(gl.FUNC_ADD);
gl.blendFuncSeperate(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA, gl.ZERO, gl.ONE);

Transparent Destination

If the alpha of the destination is not $1$, a variety of interpretations is possible. Usually a model as described in Compositing Digital Images is employed to deal with this case, as the result is very intuitive. The geometric interpretation of the model is:

which yields:

$\displaystyle \mathbf{O}_A = \mathbf{S}_A + (1 - \mathbf{S}_A)\mathbf{D}_A$

$\displaystyle \mathbf{O} _{RGB} = \frac{\mathbf{S}_A \mathbf{S} _{RGB} + (1 - \mathbf{S}_A) \mathbf{D}_A \mathbf{D} _{RGB}}{\mathbf{O}_A}$

This cannot be translated to a GL blend function, but must be implemented manually using a shader and a third buffer.

Premultiplied Alpha

With premultiplied alpha the RGB components represent the emission of the surface and the alpha component represents the occlusion of the layer beneath. This can be considered less intuitive as there is no haptic concept in the physical world resembling these properties. However when considering the blend over function you start seeing the advantages of premultiplied alpha. To convert from a straight alpha color to a premultiplied alpha color, the RGB values are multiplied by the alpha:

$$\mathbf{C} _{RGB _{pre}} = \mathbf{C}_A \mathbf{C} _{RGB _{straight}}$$

Blending Over Premultiplied Destination

Applying this to the RGB equation for the blend over model, we get:

$\displaystyle \mathbf{O} _{RGB _{pre}} = \mathbf{O}_A \frac{ \mathbf{S}_A \mathbf{S} _{RGB _{straight}} + (1 - \mathbf{S}_A) \mathbf{D}_A \mathbf{S} _{RGB _{straight}}} {\mathbf{O}_A}$

which simplifies to:

$\mathbf{O} _{RGB _{pre}} = \mathbf{S} _{RGB _{pre}} + (1 - \mathbf{S}_A) \mathbf{D} _{RGB _{pre}} $

The equation for the blended alpha component is the same for premultiplied and straight alpha. Since the form for both RGB and alpha are the same for premultiplied alpha, both can be described using one equation:

$\mathbf{O} = \mathbf{S} + (1 - \mathbf{S}_A) \mathbf{D}$

which translates to:

gl.blendEquation(gl.FUNC_ADD);
gl.blendFunc(gl.ONE, gl.ONE_MINUS_SRC_ALPHA);