Free space optical (FSO) communication system is a wireless communication technology which possess advantages such as huge license free bandwidth and high data carrying capacity among others. It has proved to be a viable solution towards the radio frequency spectrum crunch and hence has been applied in cellular back haul, earth to satellite links, deep space and defense among others. However, FSO performance is adversely affected by atmospheric turbulence and weather conditions. Different channel models are used to estimate the performance of FSO communication system. Meanwhile, different mitigation techniques have been proposed and analyzed over the years for improvement of FSO communication. Of late, hybrid modulation schemes have proved to be another research direction. In this article, we review the FSO channel models as well as hybrid modulation schemes that have been recently introduced to further improve the FSO performance. The conditional probability of error for the commonly used FSO modulation schemes are presented. Finally, the expressions for the bit error for the hybrid modulation schemes are also given. © 2021, The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature.