Deep Coaching: A Deep Learning Approach for Human Action Recognition in Photos and Videos of Athletic Motions

Abstract

This thesis serves as a survey of deep learning methods applied to the problem of human action recognition in photos and videos, specifically photos and videos of sports. The overarching goal was to start working towards developing models that might help to better understand and improve the technique of athletic motions. As a first step in this process, we sought to create models capable of differentiating between a variety of human actions. For this purpose, we created three new human action recognition datasets, one comprised of photos and the other two comprised of videos. We test multiple different deep learning methods on each dataset and analyze their performance. This serves a twofold purpose. First, it allows for a preliminary investigation as to which methods are effective in this context and which are not. Second, it establishes benchmark performance on these new datasets. In general, we find that more basic models demonstrate superior performance compared to relatively complex models, suggesting that there is significant room for improvement as far as the implementation of complex deep learning models for evaluating photos and video of athletics is concerned.