In this chapter, we will focus on techniques that work directly with the pixels in a framebuffer. These techniques typically involve multiple passes. An initial pass produces the pixel data and subsequent passes apply effects or further processes those pixels. To implement this, we often make use of the ability provided in OpenGL for rendering directly to a texture or set of textures (refer to the Rendering to a texture recipe in Chapter 5, Using Textures).
The ability to render to a texture, combined with the power of the fragment shader, opens up a huge range of possibilities. We can implement image processing techniques such as brightness, contrast, saturation, and sharpness by applying an additional process in the fragment shader prior to output. We can apply convolution filters such as edge detection, smoothing (blur), or sharpening. We'll take a closer look at convolution filters in the recipe on edge detection.
A related set of techniques involves rendering additional information to textures beyond the traditional color information and then, in a subsequent pass, further processing that information to produce the final rendered image. These techniques fall under the general category that is often called deferred shading.
In this chapter, we'll look at some examples of each of the preceding techniques. We'll start off with examples of convolution filters for edge detection, blur, and bloom. Then, we'll move on to the important topics of gamma correction and multisample anti-aliasing. Finally, we'll finish with a full example of deferred shading.
Most of the recipes in this chapter involve multiple passes. In order to apply a filter that operates on the pixels of the final rendered image, we start by rendering the scene to an intermediate buffer (a texture). Then, in a final pass, we render the texture to the screen by drawing a single fullscreen quad, applying the filter in the process. You'll see several variations on this theme in the following recipes.
![]()
Edge detection is an image processing technique that identifies regions where there is a significant change in the brightness of the image. It provides a way to detect the boundaries of objects and changes in the topology of the surface. It has applications in the field of computer vision, image processing, image analysis, and image pattern recognition. It can also be used to create some visually interesting effects. For example, it can make a 3D scene look similar to a 2D pencil sketch, as shown in the following image. To create this image, a teapot and torus were rendered normally, and then an edge detection filter was applied in a second pass:
The edge detection filter that we'll use here involves the use of a convolution filter, or convolution kernel (also called a filter kernel). A convolution filter is a matrix that defines how to transform a pixel by replacing it with the sum of the products between the values of nearby pixels and a set of pre-determined weights. As a simple example, consider the following convolution filter:
The 3 x 3 filter is shaded in gray, superimposed over a hypothetical grid of pixels. The numbers in bold represent the values of the filter kernel (weights), and the non-bold values are the pixel values. The values of the pixels could represent grayscale intensity or the value of one of the RGB components. Applying the filter to the center pixel in the gray area involves multiplying the corresponding cells together and summing the results. The result would be the new value for the center pixel (25). In this case, the value would be (17 + 19 + 2 * 25 + 31 + 33), or 150.
Of course, in order to apply a convolution filter, we need access to the pixels of the original image and a separate buffer to store the results of the filter. We'll achieve this here by using a two-pass algorithm. In the first pass, we'll render the image to a texture, and then in the second pass, we'll apply the filter by reading from the texture and send the filtered results to the screen.
One of the simplest convolution-based techniques for edge detection is the so-called Sobel operator. The Sobel operator is designed to approximate the gradient of the image intensity at each pixel. It does so by applying two 3 x 3 filters. The results of the two are the vertical and horizontal components of the gradient. We can then use the magnitude of the gradient as our edge trigger. When the magnitude of the gradient is above a certain threshold, we assume that the pixel is on an edge.
The 3 x 3 filter kernels used by the Sobel operator are shown in the following equation:
If the result of applying Sx is sx and the result of applying Sy is sy, then an approximation of the magnitude of the gradient is given by the following equation:
If the value of g is above a certain threshold, we consider the pixel to be an edge pixel and we highlight it in the resulting image.
In this example, we'll implement this filter as the second pass of a two-pass algorithm. In the first pass, we'll render the scene using an appropriate lighting model, but we'll send the result to a texture. In the second pass, we'll render the entire texture as a screen-filling quad, and apply the filter to the texture.
![]()
Set up a framebuffer object (refer to the Rendering to a texture recipe in Chapter 5, Using Textures) that has the same dimensions as the main window. Connect the first color attachment of the FBO to a texture object in texture unit zero. During the first pass, we'll render directly to this texture. Make sure that the mag and min filters for this texture are set to GL_NEAREST. We don't want any interpolation for this algorithm.
Provide vertex information in vertex attribute zero, normals in vertex attribute one, and texture coordinates in vertex attribute two.
The following uniform variables need to be set from the OpenGL application:
- Width: This is used to set the width of the screen window in pixels
- Height: This is used to set the height of the screen window in pixels
- EdgeThreshold: This is the minimum value of g squared required to be considered on an edge
- RenderTex: This is the texture associated with the FBO
Any other uniforms associated with the shading model should also be set from the OpenGL application.
![]()
To create a shader program that applies the Sobel edge detection filter, perform the following steps:
- The vertex shader just converts the position and normal to camera coordinates and passes them along to the fragment shader.
- The fragment shader applies the reflection model in the first pass, and applies the edge detection filter in the second pass:
in vec3 Position; in vec3 Normal;
uniform int Pass; // Pass number // The texture containing the results of the first pass layout( binding=0 ) uniform sampler2D RenderTex; uniform float EdgeThreshold; // The squared threshold // Light/material uniforms... layout( location = 0 ) out vec4 FragColor; const vec3 lum = vec3(0.2126, 0.7152, 0.0722); vec3 blinnPhong( vec3 pos, vec3 norm ) {
// ... } // Approximates the brightness of a RGB value. float luminance( vec3 color ) { return dot(lum, color); }
vec4 pass1() { return vec4(blinnPhong( Position, normalize(Normal) ),1.0); }
vec4 pass2() { ivec2 pix = ivec2(gl_FragCoord.xy); float s00 = luminance( texelFetchOffset(RenderTex, pix, 0, ivec2(-1,1)).rgb); float s10 = luminance( texelFetchOffset(RenderTex, pix, 0, ivec2(-1,0)).rgb); float s20 = luminance( texelFetchOffset(RenderTex, pix, 0, ivec2(-1,-1)).rgb); float s01 = luminance( texelFetchOffset(RenderTex, pix, 0, ivec2(0,1)).rgb); float s21 = luminance( texelFetchOffset(RenderTex, pix, 0, ivec2(0,-1)).rgb); float s02 = luminance( texelFetchOffset(RenderTex, pix, 0, ivec2(1,1)).rgb); float s12 = luminance( texelFetchOffset(RenderTex, pix, 0, ivec2(1,0)).rgb); float s22 = luminance( texelFetchOffset(RenderTex, pix, 0, ivec2(1,-1)).rgb); float sx = s00 + 2 * s10 + s20 - (s02 + 2 * s12 + s22); float sy = s00 + 2 * s01 + s02 - (s20 + 2 * s21 + s22); float g = sx * sx + sy * sy; if( g > EdgeThreshold ) return vec4(1.0); else return vec4(0.0,0.0,0.0,1.0); } void main() { if( Pass == 1 ) FragColor = pass1();
if( Pass == 2 ) FragColor = pass2(); }
In the render function of your OpenGL application, follow these steps for pass #1:
- Select FBO, and clear the color/depth buffers
- Set the ...
