In distracting and cluttered environments object tracking in long duration videos is quite challenging task for computer vision. The two main difficulties in real world object tracking are partial occlusion and illumination variations. Conventional methods are based on annotation of the object in the first frame, the tracker task is to estimate the target locations using same annotations in successive video frames. But these methods fail in automatic detection and tracking of object in video. This failure task can be rectified by using real time segmentation, object detection and tracking. The aim of the survey for object racking in video based on image segmentation and pattern matching is to emphasis the robustness and accuracy. We have clearly given survey on real time segmentation, feature extraction and block matching using distance measures for real time object tracking. This survey also provides an overview of the present growth of research by summarizing promising challenges for further research. © 2017 Pushpa Publishing House, Allahabad, India.