Recognition of sign languages has gained reasonable interest by the researchers in the last decade. An accurate sign language recognition system can facilitate more accurate communication of deaf and dumb people. The wide variety of Indian Sign Language (ISL) led to more challenging learning process. In the current work, three novel methods was reported to solve the problem of recognition of ISL gestures effectively by combining Neural Network (NN) with Genetic Algorithm (GA), Evolutionary algorithm (EA) and Particle Swarm Optimization (PSO) separately to attain novel NN-GA, NN-EA and NN-PSO methods; respectively. The input weight vector to the NN has been optimized gradually to achieve minimum error. The proposed methods performance was compared to NN and the Multilayer Perceptron Feed-Forward Network (MLP-FFN) classifiers. Several performance metrics such as the accuracy, precision, recall, F-measure and kappa statistic were calculated. The experimental results established that the proposed algorithm achieved considerable improvement over the performance of existing works in order to recognize ISL gestures. The NN-PSO outperformed the other approaches with 99.96 accuracy, 99.98 precision, 98.29 recall, 99.63 F-Measure and 0.9956 Kappa Statistic. © Springer International Publishing Switzerland 2017.