Peak-to-average power ratio (PAPR) is found to be a significant role in multiple input, multiple output (MIMO) orthogonal frequency division multiplexing (OFDM) systems. Partial transmit sequence (PTS) is one of the effective methods to reduce the PAPR in OFDM signals. However, finding an optimum phase factor in PTS scheme is considered to be a significant issue. In order to overcome this problem, we use orthogonal initialization method to identify the initial population for enhanced Artificial Bee Colony (ABC) algorithm. In addition, PTS technique uses exhaustive search method to find the better combination of phase factors. This would also increase the computational complexity of the system. In order to overcome this issue, a newly suboptimal method is also used in the enhanced ABC algorithm. Finally, we have combined the space time block coding (STBC) and the PTS based enhanced ABC algorithm to reduce the PAPR and bit error rate (BER). The orthogonal initialization method and suboptimal method in ABC algorithm greatly reduces the PAPR and BER in the OFDM system. MATLAB software is used to run this simulation. © 2017 Springer-Verlag GmbH Germany