Unity has published ML-Agents 2.x which can do reinforcement learning on Unity environment. Here is the GitHub Repo.
To set everything up, we first need to install Unity. I have installed 2020.3.18f1 LTS version to my Ubuntu 20.04 from Unity Hub.
Then, we need to clone the ML-Agents 2.x GitHub Repo, for example, to
Create a new Unity project, drag the folder
~/code/ml-agents/Project/Assets/ML-Agents/Examples to the Unity Assets, right beside the
Then, we need to install some packages to Unity. Click on the menu
Package Manager, select
Packages: In Project, click the
+ on its left,
Add package from disk..., and choose the
~/code/ml-agents/com.unity.ml-agents/ folder, install it. Click the
Add package from disk... again, and choose the
~/code/ml-agents/com.unity.ml-agents.extensions/ folder, install it as well.
We also need another package from
Unity Registry. Search for
Input System, and install that.
We would notice that there are still errors in the console. It seems that there are some incompatible code in the example
PushBlockWithInput. I simply delete that example in
Now, if we open the scene
Assets/Examples/3DBall/Scenes/3DBall.unity, we can click play to see the pre-trained model controling the robots. (Although there are still many error messages. ;)