Optimal Commenting: Predictive Analytics On NYTimes Data

Abstract

The internet has revolutionized the distribution of news media, and further, allows users to quickly publicize responses to online news media through public commenting. The New York Times online commenting platform also allows registered users to \recommend" comments, thereby crowdsourcing a measure of comment quality. This research attempts to discover relationships between comment recommendation count, comment text and other associated metadata (e.g. newspaper section, time posted) by conducting an in-depth exploration of the New York Times comment dataset (2005 - 2013). In this paper, we review descriptive statistics of the dataset and applicable methods for metadata sourcing, text vectorization and supervised learning. We nd recommendation prediction is best mod- eled in terms of classi cation using the Naive Bayes learning algorithm. We are able to incorporate metadata features using classi er stacking, a form of ensemble learning, to boost performance. We then discuss the results in the broader context of user-generated internet content and crowdsourcing measures of content quality.