The objective of this work is to demonstrate the feasibility of excitation source information obtained by non-parametric Vector Quantization (VQ) for speaker recognition task. Linear Prediction (LP) residual is used as the representation of excitation source information. The LP residual is subjected to non-parametric VQ during training. The codebooks are built for different codebook sizes. The testing of these codebooks using the LP residual of testing speech data indeed demonstrates that a codebook of sufficiently large size uniquely represents the speaker and provides appreciable performance. The speaker recognition system built using conventional Mel Frequency Cepstral Coefficients (MFCCs) representing vocal tract information combines well with the proposed speaker recognition system using excitation source information to provide improved performance. On a set of randomly chosen 30 speakers from the TIMIT database, the proposed system provides 75%, MFCC based system provides.