The focus of this paper is to investigate the use of improved reinforcement learning (IRL) in feedback control to improve the closed-loop performance of nonlinear systems. The classical nonlinear cargo ship steering control has been employed to test the performance of IRL. The Markov decision processes (MDP) based reinforcement learning suffers from trade-off between exploration versus exploitation and discretization for continuous control. To address these issues of discretization and exploration, nonlinear adaptive discretization is proposed to obtain more states near the desired state and less spacing states as we move away from the desired state. Such technique provides better control near the desired state which is essential to prevent overshoot and achieve good tracking performance while keeping total states bare minimum. This solves the problem of exploration partly, but also creates control issues around less density states. Around such states IRL leverages cubic spline interpolation in conjunction with K-means clustering to find smoother control actions. At the end of IRL the experience function has been added to reduce the search space for similar problems. The experience function reduces the upper and lower bounds of states in consecutive runs if desired states could be reached within new limit bounds. Simulation results show excellent performance of IRL based controller for ship steering control, noise tolerance and disturbance rejection. © 2017 Technical Committee on Control Theory, CAA.