Extracting information from gameplay videos using machine learning techniques and its varieties

Thumbnail Image
Luo, Zijin
Riedl, Mark
Associated Organization(s)
Organizational Unit
Organizational Unit
Supplementary to
The ability to extract sequences of game events for high-resolution e-sport games has traditionally required access to the game’s engine. This difficulty serves as a barrier to groups who don’t possess this access. It is possible to apply deep learning to derive these logs from gameplay video, but it requires computational power that serves as an additional barrier. These groups would benefit from access to these logs, such as small e-sport tournament organizers who could better visualize gameplay to inform both audience and commentators. In this dissertation, we present a combined solution to reduce the required computational resources and time to apply a convolutional neural network (CNN) to extract events from e-sport gameplay videos. This solution consists of techniques to train CNN faster and methods to execute predictions more quickly. These techniques expand the types of machines capable of training and running these models, which in turn extends access to extracting game logs with this approach. We evaluate the approaches in the domain of DOTA2, one of the most popular competitive e-sport games. Our results demonstrate our approach outperforms standard backpropagation baselines.
Date Issued
Resource Type
Resource Subtype
Undergraduate Thesis
Rights Statement
Rights URI