This paper describes an experimental study to compare the various steps involved in Hidden Markov Model (HMM) and Hidden Semi Markov Model (HSMM) approaches for recognition and synthesis, respectively. It presents different aspects of the two systems, bringing up the commonalities and differences among them. Further, to demonstrate the usefulness of Markov models, an HMM based recognition system and HSMM based synthesis system are implemented for Assamese handwritten numerals. All the experiments are carried out by using Assamese numeral data written by native writers. The experimental result proves the potential of Markov model applied to both handwriting recognition and synthesis system. © 2016 IEEE.