To achieve the goal of intelligent motion perception effort has been spent on visual object tracking, which is one of the most important research topics in computer vision. Visual video tracking includes object detection and tracking which are closely related process. Object detection involves verifying the presence of object and object tracking is monitoring object's spatial and temporal changes in images sequences. A major challenge in object tracking is the occlusion of the target object by other objects in the scene. In this paper we have proposed an automatic object tracking system to track occluded and Non-occluded object in the videos using SIFT (Scale Invariant Feature Transform) and Kalman filter. The objects in the image sequences can be identified with the help of invariant features extracted using SIFT algorithm. SIFT algorithm exhibits poor performance in the event of occlusion. However the occluded objects can be tracked using Kalman filter since Kalman filter optimally estimates the position of the object in the current frame using the information obtained from the previous frame. The performance of our method is evaluated using videos captured using webcam, downloaded videos from the web and some videos from visual tracker Benchmark dataset and we have achieved better recall and precision for the proposed tracking system than SIFT based tracking system. © 2017 IEEE.