CSTIO Seminar with Gar Goei Loke (Durham University)

Optimize-via-Predict: Realizing out-of-sample optimality in data-driven optimization

Abstract

We examine a stochastic formulation for data-driven optimization wherein the decision-maker is not privy to the true distribution, but has knowledge that it lies in some hypothesis set and possesses a historical data set, from which information about it can be gleaned. We define a prescriptive solution as a decision rule mapping such a data set to decisions. As there does not exist prescriptive solutions that are generalizable over the entire hypothesis set, we define out-of-sample optimality as a local average over a neighbourhood of hypotheses, and averaged over the sampling distribution. We prove sufficient conditions for local out-of-sample optimality, which reduces to functions of the sufficient statistic of the hypothesis family. We present an optimization problem that would solve for such an out-of-sample optimal solution, and does so efficiently by a combination of sampling and bisection search algorithms. Finally, we illustrate our model on the newsvendor model, and find strong performance when compared against alternatives in the literature. There are potential implications of our research on end-to-end learning and Bayesian optimization.

About Dr Gar Goei Loke

Gar Goei recently joined the Department of Management and Marketing at Durham as an associate professor in November 2023. Previously, he was an assistant professor at the Rotterdam School of Management, and prior to that, a visiting assistant professor at the National University of Singapore Business School. Gar Goei’s research centres on the technical areas of robust optimization and data-driven optimization. He is looking at ways to harmonize machine learning, statistics and optimization. He finds applications in areas of healthcare operations management, energy and water, supply chain management and service operations management. His research has been published in Operations Research and Manufacturing & Service Operations Management. Prior to joining academia, he spent 5 years as a data scientist, and then leading teams of data scientists in the Singapore Government. He remains affiliated to the core data science team in the Singapore Government, and provides consultancy and training on a voluntary basis.