A system for improved shadowing of images using a multiple pass, depth buffer approach includes rendering a scene from the perspective of a light source to construct a shadow depth map in a rasterization buffer. The system computes depth values for the two nearest geometric primitives to the light source for pixels, and stores these depth values in the rasterization buffer. Once the shadow map is constructed, it is stored in shared memory, where it can be retrieved for subsequent rendering passes. The two depth values for each element in the shadow map can be used in combination with a global bias to eliminate self-shadowing artifacts and avoid artifacts in the terminator region. The system supports linear or higher order filtering of data from the shadow depth map to produce smoother transitions from shadowed and un-shadowed portions of an image. In addition, the system supports the re-use of the shadow map and shadowed images for more than one frame.