Indexing of medical images for efficient management is gaining importance with its continuously growing database size. The popular text-based image indexing method is time-consuming. Content-based labeling uses mathematical models for automatic indexing, offers higher speed and unbiased performance. Widely used image features are computed from pixel intensity and its distribution pattern. Pixel intensity depends on imaging modality. It introduces differences in the content of the same image when acquired using different modalities. In this article, we have proposed an intensity independent descriptor, Information Count and Distribution Matrix (ICDM). It is formulated to eliminate modality dependency and decrease the mathematical load. ICDM is formed using the foreground information of an image. Three primary (and two auxiliary) features are extracted from this matrix and used for brain image indexing. Brain computed tomography images are indexed by more than 96% accuracy using the proposed features. The formation of ICDM is proposed to reduce the semantic gaps between human and computer vision. © 2020 Wiley Periodicals, Inc.