This paper presents a scheme for object detection and tracking in heterogeneous sensor network environment. The main objective is to provide a solution based on Wireless and Multimedia Sensor Networks (W&MSN) for monitoring and tracking of object (person/vehicle) in secured area. The multi-tier, heterogeneous sensor network adapted for efficient usage of image data. The object detection is carried out with background subtraction technique. The detected blob region is taken as input for extracting the features based on Haar wavelet. The feature extraction is followed by joint boosting algorithm to classify as interested object or not. The object detection is combined with Kalman Filter to accurately track the movement of desired objects in the given area. This approach provides better detection and tracking of person even in the presence of occlusion and multiple persons in the environment. © Springer-Verlag Berlin Heidelberg 2012.