Human activity recognition is an active and interesting field in computer vision from past decades. The objective of the system is to identify human activities using different sensors such as cameras, wearable devices, motion and location sensors, and smartphones. The human actions are automatically identified through their physical activities in human–computer interaction. Determining the human action in an uncontrolled environment is a challenging task in human activity recognition system. In this paper, a novel approach is proposed to recognize human actions effectively in an uncontrolled environment. A frame for the video segment is selected by temporal superpixel, which acts as the input image for the model. Convolutional neural network techniques are applied to extract the features and recognize the human activities from the image. The proposed method has experimented on KTH database and it shows the performance of the method in terms of accuracy. However, the proposed method has attained better accuracy when compared to the existing methods. © 2020, Springer Nature Singapore Pte Ltd.