Adaptive swarm behavior acquisition by a neuro-fuzzy system and reinforcement learning algorithm

Purpose - The purpose of this paper is to present a neuro-fuzzy system with a reinforcement learning algorithm (RL) for adaptive swarm behaviors acquisition. The basic idea is that each individual (agent) has the same internal model and the same learning procedure, and the adaptive behaviors are acquired only by the reward or punishment from the environment. The formation of the swarm is also designed by RL, e.g. temporal difference (TD)-error learning algorithm, and it may bring out a faster exploration procedure comparing with the case of individual learning. Design/methodology/approach - The internal model of each individual composes a part of input states classification by a fuzzy net, and a part of optimal behavior learning network which adopting a kind of RL methodology named actor-critic method. The membership functions and fuzzy rules in the fuzzy net are adaptively formed online by the change of environment states observed in the trials of agent's behaviors. The weights of connections between the fuzzy net and the action-value functions of actor which provides a stochastic policy of action selection, and critic which provides an evaluation to state transmission, are modified by TD-error. Findings - Simulation experiments of the proposed system with several goal-directed navigation problems are accomplished and the results show that swarms are successfully formed and optimized routes are found by swarm learning faster than the case of individual learning. Originality/value - Two techniques, i.e. fuzzy identification system and RL algorithm, are fused into an internal model of the individuals for swarm formation and adaptive behavior acquisition. The proposed model may be applied to multi-agent systems, swarm robotics, metaheuristic optimization, and so on.