During the learning process, every agent's action affects the interaction with the environment based on the agent's current knowledge and future knowledge. The agent must therefore have to choose between exploiting its current knowledge or exploring other alternatives to improve its knowledge for better decisions in the future. This paper presents critical analysis on a number of exploration strategies reported in the open literatures. Exploration strategies namely random search, greedy, ε-greedy, Boltzmann Distribution (BD), Simulated Annealing (SA), Probability Matching (PM) and Optimistic Initial Values (OIV) are implemented to study on their performances on a multi-agent foraging task modeled. © 2011 Springer-Verlag.