Methods of Pair Selection in Pairwise Comparisons for Efficient List Ranking

Abstract

This thesis implements and analyzes various pair selection methods for pairwise comparisons in list rankings. To compare these methods, this thesis utilizes the Bradley- Terry model to score and rank the list of items for each method. It first scores the items in the existing dataset, creating a set of “true scores” that is used measure the error of the rankings from each method implemented. The random, epsilon-greedy, epsilon-greedy within the top 10, minimizing overlap with exploration, and minimizing overlap within the top 10 pair selection methods are then implemented. The average error of item scores over 100 trials for varying numbers of questions indicates how quickly the scores converge to the true scores. Results indicate that a variation of the epsilon-greedy algorithm, the epsilon-greedy top 10 method, as well as a method that minimizes overlap with a small amount of random pair selection, perform better than random pair selection itself, while the traditional epsilon-greedy method as well as a method that minimizes overlap within the top 10 ideas, with a small amount of random pair selection, perform much worse than random pair selection alone.