The Little Wind Farm that Could: A Comparative Analysis of Lookahead Policies for Energy Storage Problems

Abstract

In this thesis, we explore how to make the best sequential decisions when faced with variations of a stochastic energy storage optimization problem. Each day, smart grid managers need to satisfy the energy demands of a load with wind energy from a wind farm, energy from a rechargable storage device, and energy from an electricity grid. However, this problem is rife with uncertainty, since both the supply of wind energy and grid electricity prices are highly stochastic.

We create seven variations of this problem designed to bring out the features of different policies which make decisions every hour about what combination of energy sources to use to satisfy the power demand, how much energy to store, and how much to buy from the grid in order to maximize profits over a day. For each problem, we evaluate the performance of the policies over a large number of simulated trials to identify the policy which best optimizes revenue when fitted to the problem's electricity market. These problems demonstrate that each class of policies is best for particular problem characteristics. In the process, we combine tools from dynamic programming, approximate dynamic programming, Monte Carlo methods and reinforcement learning.