Fruits are sources of many nutrients essential for leading a healthy life. Growing fruits has been the mainstay of rural economy and has emerged as an indispensable part of agriculture over the world. Apart from vitamins and minerals this sector significantly contributes for the economy of the nation. Such fruits are affected by some bacteria and fungus resulting in small black spots, Alternaria brown spot, melanose, greasy spot on fruits at all stages of its growth. In this paper special emphasis is placed on diagnosis and identification of diseases. Thresholding segmentation separates the affected part of the fruit images and the features of the segmented region is extracted by Local Binary Pattern (LBP) method and is used as input data in the feed forward back propagation neural network to classify the bacterial and fungal disease of fruit images. © 2019 IEEE.