Biometrics is indeed becoming an important solution for any highly secured system. Voice is one of the biometric parameters that can use for a person identification and verification. In this paper, a small foot-print, low power embedded system is proposed and implemented using Beagle Bone Black (BBB). Hidden Markov Model (HMM) based speaker recognition system is implemented. Mel-Frequency Cepstrum Coefficients (MFCC) is used as features to identify the speaker. Each speaker is modelled as one HMM. The verification of the speaker voice is done using Viterbi decoder. The embedded system for Voice Biometric system is successfully implemented for a limited number of speakers and the accuracy is verified to be as almost 100%.