Background modelling is a fundamental task in video analytics. This paper presents an adaptive background modelling for real-time indoor and outdoor videos. This proposed method treats background modelling as an optimisation problem and, it fetches multiple peaks from the histogram of the video frame and optimises them using a multi-swarm technique. The background is successfully adapted whenever there is a change in the environment as well as a number of objects in the background. The experimental result of foreground extraction confirms the effectiveness and robustness of the proposed background modelling technique against the various background modelling approaches. Copyright © 2019 Inderscience Enterprises Ltd.