For a sustainable environment, geopolymer (GPO) paste can be used in the construction industry instead of Portland cement. Nowadays, sustainable construction and high-efficacy composites are demanding. Therefore, in the present investigation, the mechanical and microstructural efficacy of carbon-fiber-reinforced fly ash-based GPO with different percentages of nano-sodium dioxide (NS) were studied. The investigated percentages of NS were 0%, 1%, 2%, 3%, and 4%. For all the samples, the carbon fiber content was kept the same at 0.5% by weight. Different percentages of NS for all five fabricated GPO composite pastes were assessed with scanning electron microscopy (SEM). Various mechanical parameters of GPO—the compressive strength, toughness modulus, hardness, toughness indices, impact strength, fracture toughness, flexural strength, and elastic modulus—were evaluated. The results revealed that the use of 3% NS was the most effective for ameliorating the mechanical, microstructural, and fracture behavior of GPO. The use of 3% NS in carbon-fiber-reinforced GPO paste showed the maximum improvements of 22%, 46%, 30%, 40%, 14%, 38.4%, 50.2%, 31%, and 64% for the compressive strength, flexural strength, elastic modulus, toughness modulus, hardness, compressive stiffness, bending stiffness, fracture toughness, and impact strength, respectively. The SEM study showed that the inclusion of NS improved the microstructure and delivered a denser GPO paste by improving the interfacial zones and quickening the polymerization reaction.