The rapid development of Information and Communication Technology and the growing number of devices connected to the Internet make the Internet of Things (IoT) as a promising technology for a new breed of applications. The Routing Protocol for Low-Power and Lossy Networks (RPL) is a widely applied open standard protocol for IoT networks. The RPL routing is highly vulnerable to routing attacks due to the constrained nodes. The attacks on the RPL aim to disrupt the optimal protocol structure and significantly deteriorating network performance. Secure RPL routing schemes attempt to derive a high-level abstract of RPL operations through network simulation traces and apply it as a reference to differentiate the malicious behavior. The RPL specifications include all the states and transitions with its corresponding statistics. However, the malicious activities around a node enforce it to initiate the unnecessary state transition, and thus, the legitimate nodes are equally treated as malicious in dynamic IoT network scenarios. Hence, this work proposes a game theoretic model based anomaly Intrusion Detection System (IDS) to detect the RPL attacks and verify and confirm their malicious activities. This study formulates the Game models based Anomaly Intrusion Detection System (GAIDS) for RPL security. The proposed approach consists of two interrelated formulations, such as a stochastic game for attack detection and evolutionary game for attack confirmation. The stochastic game model formulates the activities of the standard RPL rules as a zero-sum stochastic game. The stochastic game estimates the payoff by observing the states, transitions between them, and their statistics. However, there is a possibility to model legitimate players as malicious, due to the nature of RPL. Thus, the proposed GAIDS scheme implements the evolutionary game theoretic framework on clustered network topology for the attack verification. By synchronizing the results of the stochastic game of neighboring players, it differentiates the legitimate players from the suspected list successfully. As a result, the GAIDS isolates the detected attackers and maintains the routing performance. The simulation results demonstrate that the detection accuracy and throughput of the proposed gaming model based anomaly IDS is substantially high and outperforms the existing scheme. © 2019, Springer Science+Business Media, LLC, part of Springer Nature.