Chen, X.; Diao, D.; Chen, H.; Yao, H.; Piao, H.; Sun, Z.; Yang, Z.; Goebel, R.; Jiang, B.; Chang, Y. The Sufficiency of Off-Policyness and Soft Clipping: PPO Is Still Insufficient According to an Off-Policy Measure. AAAI 2023, 37, 7078-7086.