The video sequences provide more information than the still images about how objects and scenarios change over time. However, video needs more space for storage and wider bandwidth for transmission. Hence, more challenges are encountered in retrieval and event detection in large data sets during the visual tracking. In the proposed method, the object planes are segmented properly and the motion parameters are derived for each plane to achieve a better compression ratio. Most of the existing tracking algorithms in dynamic scenes consider the target alone and the background information are often ignored. Therefore, they are failed to track the target. In order to optimize the existing system, a robust visual tracking algorithm is to be developed which will adapt the drastic changes of target appearance without background influence. The initial occlusion of non target objects in the background can effectively be addressed by the integration of multiple cues and spatial information in target representation. With the combination of motion information and detection methods, the target can be reacquired when complete occlusion of target occurs. © 2012 IEEE.