GymSlug: Deep Reinforcement Learning Toward Bio-inspired Control Based on Aplysia californica Feeding

Developing robots with animal-like flexibility, adaptability, and robustness remains challenging. However, the neuromuscular system of animals can provide bioinspiration for robotic controller design. In this work, we have developed a bio-inspired simulation environment, GymSlug, for reinforcement learning of motor control sequences based on our prior models of feeding behavior in the marine mollusk Aplysia californica. Using a range of model-free deep reinforcement learning algorithms, we train agents capable of producing motor neural control sequences, muscle activities, and feeding apparatus behavior that are qualitatively similar to behaviors observed in the animal during swallowing of unbreakable seaweed. The robustness of the trained agent is demonstrated by its ability to adapt to a previously unseen environment with breakable seaweed of varying strength. In addition, the environment can be easily reconfigured to train agents for additional tasks, including effective egestion of inedible objects. Our extensible simulation environment provides a platform for developing novel controllers to test biological hypotheses, learn control policies for neurorobotic models, and develop new approaches for soft robotic grasping control inspired by Aplysia.

Automated summary

This study develops a bio-inspired simulation environment, GymSlug, based on the feeding behavior model of Aplysia, to train agents using deep reinforcement learning algorithms. The trained agent can produce motor neural control sequences, muscle activities, and feeding apparatus behavior that are similar to behaviors observed in the animal. The agent also demonstrates robustness in adapting to different seaweed environments.