We propose an actor-critic reinforcement learning (RL) algorithm for the optimal execution problem. We consider the celebrated Almgren-Chriss model in continuous time and formulate a relaxed stochastic control problem for execution under an entropy regularized mean-quadratic variation objective....