The energy efficiency is considered as a major issue due to large power consumption of eNBs in heterogeneous cellular networks. In this paper, a novel relay station (RS) deployment scheme and base station (BS) sleep mode algorithm is proposed to minimize the power consumption of eNBs. Initially, the RSs are deployed to cover the entire area of a cell. It is due to the purpose of providing service when the BS is in sleep mode. Then the network traffic of each cell is measured with Erlang B and C probability measure. If the network traffic is low, then the BS is decided to put into sleep mode which reduces the power consumption. For that, the corresponding RS are selected to handover the active mobile users (MUs). Then, the network traffic is estimated for each RS and the RS without MU becomes sleep mode in order to reduce power consumption further. The proposed model of cellular network reduces the power consumption by applying sleep mode algorithm for both BS and RS based on the measured network traffic. The power consumed by the entire network is measured and compared with the network without sleep mode. The evaluation results show the efficiency of our proposed work. © 2019, © 2019 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.