Over recent years, the demand for face video analytics in surveillance cameras is increasing dramatically due to the need for real-time event detection in commercial and security applications. The main objective of this paper is to implement robust face detection and identification system on a programmable resource constrained digital video processor. In order to reduce the unnecessary computations involved in Viola and Jones (V-J) face detector, we have proposed skin detection based search window reduction technique. After that, to identify the detected faces, we have proposed sparse prototypes based face identification algorithm which can address partial occlusion and expression variation issues effectively. Furthermore, within the face identification framework, a simple rejection ratio has been proposed to reject the invalid test faces. Experimental results of the developed embedded vision system demonstrate that the proposed methods can achieve satisfactory performance in uncontrolled settings. © 2015 Inderscience Enterprises Ltd.