Completely Automated Public Turing test to tell Computers and Humans Apart abbreviated as CAPTCHAs are extensively used security systems on the Internet that prohibit automated software from misusing online resources. They achieve it by asking humans to accomplish a task that automated systems cannot yet perform, such as identifying distorted characters, recognising images, etc. While increasing the difficulty level for the bots, often the designs end up getting more difficult for the users which violates the basic Usability concept of the Human Computer Interaction. There is no definite evaluative framework for CAPTCHAs to assess its usability. The paper aims to develop an automated tool that takes into account quantitative metrics to evaluate the usability of CAPTCHAs, based on statistical measures like count of confusing characters, line count, distortion, etc; an application which takes a CAPTCHA as the input and outputs the usability rank of that particular CAPTCHA. © 2017 IEEE.