The paper presents an efficient and reliable approach to automatic people segmentation, tracking and counting, used in surveillance systems. The initial novel background extraction algorithm uses an improved mode algorithm to obtain the static and novel fuzzy background subtraction approach regions for background subtraction. We also develop a complete framework to evaluate a new edge flow vector based segmentation processes. Tracking segmented people is a dynamic cluster assignment problem between two consecutive frames, and it is solved by a fuzzy-based rule system for tracking and people counting and is applied to people surveillance. Experimental results suggest that the proposed method can achieve good results in both counting accuracy and execution speed. © 2015 The Authors.