In this paper, a multi-carrier cognitive decision engine based on a binary particle swarm optimization with a non-linear decreasing inertia-weight (NDI-BPSO) is presented. Our main goal is to solve the optimization problem of transmitter parameters in different wireless communication modes for cognitive radio systems (CRSs), especially for the transmitter in communication systems based on the environment sensing. In the new algorithm, the multi-carrier cognitive decision engine based on an NDI-BPSO algorithm can mitigate the local extreme points effectively and reduce the oscillation phenomenon in the process of optimization. We apply the NDI-BPSO to the cognitive orthogonal frequency division multiplexing (OFDM) system to determine the best parameters to obtain good performances in different communication modes. The simulation results show that the proposed multi-objective cognitive decision engine, which has a high fitness value and strong robustness for different communication modes, is better than the existing engines. The novel NDI-BPSO algorithm achieves the objective of parameter optimization effectively. © 2018 John Wiley & Sons, Ltd.