In recent years Biometric based Authentication systems have gained more attention due to frequent fraudulent attacks. Hence, this study aims at developing a multi-modal, multi-sensor based Person Authentication System (PAS) using Joint Directors of Laboratories (JDL) fusion model. This study investigates the need for multiple sensors, multiple recognition algorithms and multiple fusion levels and their efficiency for a Person Authentication System (PAS) with face, fingerprint and iris biometrics. The proposed system considers several environmental factors in the design. If one sensor is not functional, others contribute to the system making it fault-tolerant. Robustness has been tactfully administered to the processing module by employing different efficient algorithms for a given modality. Selection of the recognition algorithms is rooted on the attributes of the input and multiplicity has been employed to establish a unanimous decision. Information fusion at various levels has been introduced. A multitude of decisions are fused locally to decide the weight for a particular modality. Algorithms are tagged with weights based on their recognition accuracy. Weights are assigned to sensors based on their identification accuracy. Adaptability is incorporated by modifying the weights based on the environmental conditions. All local decisions are then combined to result in a global decision about the person. The final aggregation concludes whether 'The Person is Authenticated or not'.