The use of formal concept analysis (FCA) derives knowledge from any underlying information system in the form of concept lattices and a set of association rules. However, huge contexts increase the complexities of deriving concept lattices and their association rules. Consequently, the task of discovering knowledge and mining association rules becomes a challenging problem. Researchers have handled this problem with matrix decomposition techniques to approximate the original context which is perhaps not best suitable, because the linear combination of vectors do not yield meaningful interpretations in real-life contexts. To overcome this problem, in this article we propose a novel approach using the CUR matrix decomposition technique which decomposes the original context in terms of dimensionally reduced low-rank matrices of actual columns and rows. The main distinction of the CUR decomposition method from others is that it maintains better structural properties of the original matrix. So the use of CUR decomposition in FCA reduction techniques could assist us in retrieving the highly important information from the datasets. The proposed method is illustrated with the use of real-time medical diagnosis reports. Furthermore, the performance of the proposed method is tested on the large synthetic contexts. © 2019, © 2019 Taylor & Francis Group, LLC.