Background subtraction is a very popular approach for foreground segmentation in a still scene image. A common approach is to perform background subtraction, which identifies moving objects from the portion of a video frame that differs significantly from a background model. Each video frame from the sequence is compared against a reference frame. There are several problems that a good background subtraction algorithm must resolve. To achieve robust background subtraction is required to have reliable and effective background estimation. In the existing system the background reference image is arbitrarily chosen. This paper presents a novel background estimation algorithm based on improved mode algorithm to obtain static background reference frame from input image. Evaluation of the background extraction is presented in this paper. The experimental results of using background image estimated by background estimation method are faster and accurate in background subtraction. The major goal is to obtain a clean static background reference image using a real-time video. © Springer International Publishing Switzerland 2016.