Unity has published ML-Agents 2.x which can do reinforcement learning on Unity environment. Here is the GitHub Repo.
To set everything up, we first need to install Unity. I have installed 2020.3.18f1 LTS version to my Ubuntu 20.04 from Unity Hub.
Then, we need to clone the ML-Agents 2.x GitHub Repo, for example, to ~/code/ml-agents/.
Create a new Unity project, drag the folder ~/code/ml-agents/Project/Assets/ML-Agents/Examples to the Unity Assets, right beside the Scenes folder.
Then, we need to install some packages to Unity. Click on the menu Window->Package Manager, select Packages: In Project, click the + on its left, Add package from disk..., and choose the package.json in ~/code/ml-agents/com.unity.ml-agents/ folder, install it. Click the + and Add package from disk... again, and choose the package.json in ~/code/ml-agents/com.unity.ml-agents.extensions/ folder, install it as well. 
We also need another package from Unity Registry. Search for Input System, and install that. 
We would notice that there are still errors in the console. It seems that there are some incompatible code in the example PushBlockWithInput. I simply delete that example in Assets. (There is one folder called Assets/Examples/SharedAssets, it must be included.)
Now, if we open the scene Assets/Examples/3DBall/Scenes/3DBall.unity, we can click play to see the pre-trained model controling the robots. (Although there are still many error messages. ;) 