A video security system (10) monitors a premise (F) to detect unwanted intrusions onto the premises. A plurality of cameras (22) located about the premise supply video images of scenes to a processor (12) located which processes the images to detection motion in a scene and classify the source of the motion. Only if the source is determined be one of a predetermined class of causes, is an indication provided to an alarm unit (16). The alarm unit, which is also connected to a plurality of conventional sensors (S1-S3) is responsive to the indication to cause the processor to transmit authenticated video images of the scene in which the motion is detected to a central station (CS). There, a video server (102), in conjunction with an alarm computer (104), enables the images to be displayed at a selected workstation (106) for viewing by an operator (O). Besides video, audio and relevant site data is also made available to the operator at the workstation. The operator's responsibility is to alert appropriate authorities. By not providing an indication of an intrusion unless the intrusion has been previously confirmed on-site, false and unwanted alarms are prevented without reducing but the probability of detecting an intrusion.