Whales play an important role in ocean ecosystem by maintaining a stable food chain. Whales continued to be endangered animals though they don't have direct predators. To ensure the survival of this endangered species, marine biologists are tracking them to know their status and health of the species at all times. Manual recognition of whale is most tricky and hence automated system helps biologist to develop conservation strategies for different species of whale. This can be primarily achieved through individual whale recognition and tracking their behavior by analyzing the data collected. In this paper we have proposed a method for finding the matching pairs of whales by analyzing the caudal fin images of whales in the data set. For finding the matching pairs we have segmented the caudal fins from the background images using GrabCut and FloodFill algorithm. From the segmented images key features are extracted using Scale Invariant Feature Transform (SIFT) and key features are matched using the FLANN (Fast Library for Approximate Nearest Neighbors) matcher. Finally the matched pairs are ranked using the confidence values.