In a passive identification apparatus for identifying a predetermined individual member of a television viewing audience in a monitored viewing area, a video image of a monitored viewing area is captured. A template matching score is provided for an object in the video image. A Eigenface recognition score is provided for an object in the video image. These scores may be provided by comparing objects in the video image to reference files. The template matching score and the Eigenface recognition score are fused to form a composite identification record from which a viewer may be identified. Body shape matching, viewer tracking, viewer sensing, and/or historical data may be used to assist in viewer identification. The reference files may be updated as recognition scores decline.