A driver assistance system for a vehicle includes an imaging device disposed at an exterior portion of a vehicle. With the imaging device disposed at the exterior portion of the vehicle, the imaging device has a field of view at least one of rearward of the vehicle and sideward of the vehicle. A control utilizes edge detection in processing image data captured by the imaging device. Responsive at least in part to processing of captured image data by the control, the control determines that an object is present in the field of view of the imaging device. The control is operable to determine that the imaging device is misaligned. The control, responsive to a determination of misalignment of the imaging array sensor, is operable to at least partially compensate for the determined misalignment of the imaging device.