Image classifiers are largely adopted to categorize a pool of images or patterns in a databank, match category of a query image and to retrieve similar images to query from the category. Fuzzy ARTMAP (FAM) architecture have been widely included for pattern classification in various applications. The major constraint that limits the application of FAM network is category proliferation problem. That is the architecture has the tendency to increase the network size. The issue is because of noisy data, order of presenting training data and/or overlapping categories. In this paper, we propose a new methodology, DE-FAM to handle category proliferation problem by reducing the quantity of categories in the trained FAM architectures. The enhanced generalized performance, reduction in network size and influence of the proposed algorithm in computational cost is demonstrated by adopting the algorithm for image classification and retrieval. Furthermore the comparison of DE-FAM with other algorithms that address the category proliferation problem illustrate the advantages of DE-FAM. © 2019, Springer Science+Business Media, LLC, part of Springer Nature.