![Page 4: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/4.jpg)
ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose...
21
Wismut Labs OpenAI + DotA 2
Transcript of ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose...
![Page 1: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/1.jpg)
Wism
ut
Labs
OpenAI + DotA 2
![Page 2: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/2.jpg)
Wism
ut
Labs
DotA (Defense of the Ancients) 2 Gameplay
•
•
•
•
•
•
•
![Page 3: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/3.jpg)
Wism
ut
Labs
DotA and StarCraft II Challenges [1, 2, 3]
• → →
•
•
•
•
![Page 5: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/5.jpg)
Wism
ut
Labs
OpenAI DotA Approach [1]
•
•
•
•
•
γ
•
![Page 6: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/6.jpg)
Wism
ut
Labs
OpenAI DotA ‘Cheats’ [1, 3]
•
• →
•
•
•
•
•
• →
•
•
![Page 7: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/7.jpg)
Wism
ut
Labs
OpenAI Five Network Architecture [1, 6]
![Page 8: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/8.jpg)
Wism
ut
Labs
OpenAI Five Network Architecture [1, 6]
![Page 9: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/9.jpg)
Wism
ut
Labs
OpenAI Five Model Structure [1]
•
•
•
•
•
•
•
•
•
•
![Page 10: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/10.jpg)
Wism
ut
Labs
OpenAI Five Exploration [1]
•
•
• →
•
•
• →
→
•
•
•
![Page 11: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/11.jpg)
Wism
ut
Labs
OpenAI Rapid [1]… a general-purpose RL training system
•
•
•
•
•
•
•
•
•
→
![Page 12: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/12.jpg)
Wism
ut
Labs
Proximal Policy Optimization [1, 4, 5]
•
• →
• →
•
•
→
![Page 13: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/13.jpg)
Wism
ut
Labs
Proximal Policy Optimization [1, 4, 5]
•
•
•
•
• →
![Page 14: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/14.jpg)
Wism
ut
Labs
Proximal Policy Optimization [1, 4, 5]
•
•
•
• →
•
•
→
![Page 15: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/15.jpg)
Wism
ut
Labs
Proximal Policy Optimization [1, 4, 5]
•
•
•
•
•
•
• →
![Page 16: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/16.jpg)
Wism
ut
Labs
Proximal Policy Optimization [1, 4, 5]
•
• →
•
![Page 17: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/17.jpg)
Wism
ut
Labs
Proximal Policy Optimization [1, 4, 5]
•
•
•
![Page 18: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/18.jpg)
Wism
ut
Labs
Proximal Policy Optimization [1, 4, 5]
•
→
![Page 19: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/19.jpg)
Wism
ut
Labs
Transfer Learning for RL [1]
•
•
•
→
![Page 20: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/20.jpg)
Wism
ut
Labs
Open Challenges & Moving Forward
•
•
•
•
•
![Page 21: ut s OpenAI + DotA 2kanmy/courses/6101_1820/s13.pdf · ut s OpenAI Rapid[1] … a general-purpose RL training system . ut s Proximal Policy Optimization[1, 4, 5]](https://reader034.fdocuments.us/reader034/viewer/2022042419/5f35cf86d6d56226871ad0c0/html5/thumbnails/21.jpg)
Wism
ut
Labs
Reference Materials
