This work describes the development of a scheme for retrieving spoken documents in a remote fashion stored on a voice server. The spoken documents are recorded and indexed based on the frequency of occurrence of isolated keywords and are stored on the voice server. An isolated word recognizer (IWR) is developed for recognizing the identified keywords spoken in isolated fashion. The IWR employs foreground speech segmentation to selected speech regions against complex background audio. For retrieving spoken documents related to any of the isolated keywords, the user calls to the voice server and gives test speech for required isolated word. The IWR recognizes the spoken word and matches with the indexed database. The retrieved spoken documents are played via a voice activated interactive voice response (IVR) system based on the feedback from the user. © 2013 IEEE.