Facial expression recognition is an active research area for which the research community has presented a number of approaches due to its diverse applicability in different real-world situations such as real-time suspicious activity recognition for smart security, monitoring, marketing, and group sentiment analysis. However, developing a robust application with high accuracy is still a challenging task mainly due to the inherent problems related to human emotions, lack of sufficient data, and computational complexity. In this paper, we propose a novel, cost-effective, and energy-efficient framework designed for suspicious activity recognition based on facial expression analysis for smart security in law-enforcement services. The Raspberry Pi camera captures the video stream and detects faces using the Viola Jones algorithm. The face region is pre-processed using Gabor filter and median filter prior to feature extraction. Oriented FAST and Rotated BRIEF (ORB) features are then extracted and the support vector machine (SVM) classifier is trained, which predicts the known emotions (Angry, Disgust, Fear, Happy, Neutral, Sad, and Surprise). Based on the collective emotions of the faces, we predict the sentiment behind the scene. Using this approach, we predict if a certain situation is hostile and can prevent it prior to its occurrence. The system is tested on three publically available datasets: Cohen Kande (CK+), MMI, and JAFEE. A detailed comparative analysis based on SURF, SIFT, and ORB is also presented. Experimental results verify the efficiency and effectiveness of the proposed system in accurate recognition of suspicious activity compared to state-of-the-art methods and validate its superiority for enhancing security in law enforcement services. © 2018 Elsevier Inc.