Scheduling of Twin Automated Stacking Cranes Based on Deep Reinforcement Learning

Effective scheduling of twin automated stacking cranes (ASCs) in automated storage yard is critical to maximize operational efficiency. While Deep Reinforcement Learning (DRL) is promising in solving NP-hard scheduling problems, twin ASCs scheduling is challenging due to its unique properties including sequence-dependent setup and potential ASC interferences. In this paper, we propose a novel DRL method to learn high-quality policy for scheduling twin ASCs. We propose a Markov Decision Process model that enables the DRL agent to learn to minimize makespan and possible interferences. Based on the problem characteristics, we design a self-attention based neural architecture to effectively capture the relationships between containers under certain block state. Experiments show that the agent can learn high-quality polices from training instances, and produce better solutions with reasonable computational efficiency compared with manually designed scheduling rules and metaheuristic. Moreover, the policies show remarkable generalization ability on unseen instances with different distributions or scales