 1.
DataPooling in Stochastic Optimization.
with Nathan Kallus
 Under Review
(Submitted May 2019).
 [Abstract]
[SSRN]
[Code]

Managing largescale systems often involves simultaneously solving thousands of unrelated stochastic optimization problems, each with limited data. Intuition suggests one can decouple these unrelated problems and solve them separately without loss of generality. We propose a novel datapooling algorithm called ShrunkenSAA that disproves this intuition. In particular, we prove that combining data across problems can outperform decoupling, even when there is no a priori structure linking the problems and data are drawn independently. Our approach does not require strong distributional assumptions and applies to constrained, possibly nonconvex, nonsmooth optimization problems such as vehiclerouting, economic lotsizing or facility location. We compare and contrast our results to a similar phenomenon in statistics (Stein’s Phenomenon), highlighting unique features that arise in the optimization setting that are not present in estimation. We further prove that as the number of problems grows large, ShrunkenSAA learns if pooling can improve upon decoupling and the optimal amount to pool, even if the average amount of data per problem is fixed and bounded. Importantly, we highlight a simple intuition based on stability that highlights when and why datapooling offers a benefit, elucidating this perhaps surprising phenomenon. This intuition further suggests that datapooling offers the most benefits when there are many problems, each of which has a small amount of relevant data. Finally, we demonstrate the practical benefits of datapooling using real data from a chain of retail drug stores in the context of inventory management.

 2.
The Value of Personalized Pricing.
with Adam Elmachtoub and Michael Hamilton
 Under Review
(Updated Dec. 2019).
 [Abstract]
[SSRN]
[Code]
 **Finalist in the 2018 INFORMS Service Science Best Paper Award**
 **Accepted in The 15th Conference on Web and Internet Economics (WINE), 2019**

Increased availability of highquality customer information has fueled interest personalized pricing strategies, i.e., strategies that predict an individual customer’s valuation for a product and then offer a customized price tailored to that customer. While the appeal of personalized pricing is clear, it may also incur large costs in the form of market research, investment in information technology and analytics expertise, and branding risks. In light of these tradeoffs, in this work, we study the value of personalized pricing over simpler pricing strategies, such as charging a single price to all customers. In the first part of the work, we provide tight, closedform upper bounds on the ratio of personalized pricing profits to singlepricing profits that depend on simple statistics of the valuation distribution. These bounds shed light on the types of markets for which personalized pricing has the most potential. In the second part of the work, we use these bounds to study the two key assumptions underlying personalized pricing: (i) the firm can charge a distinct price to each customer and (ii) the firm can perfectly predict customer valuations. Specifically, we bound the ratio of personalized pricing profits to the profits from a ksegmentation strategy where the firm is omniscient but can only charge customers one of k prices, and the ratio of personalized pricing profits to the profits of a featurebased pricing strategy, where the firm can charge a continuum of prices, but is no longer omniscient. These bounds help quantify the value of the operational capability of charging distinct prices and the value of additional predictive accuracy, respectively. Finally, we provide a general framework for computing an essentially tight bound on the ratio of personalized pricing profits to singlepricing profits in terms of the mean, support, and a generalized moment of the distribution

 3.
SmallData, LargeScale Linear Optimization.
with Paat Rusmevichientong
 Management Science
(Forthcoming).
 [Abstract]
[SSRN]
[Code]

Optimization applications often depend upon a huge number of uncertain parameters. In many contexts, however, the amount of relevant data per parameter is small, and hence, we may only have imprecise estimates. We term this setting – where the number of uncertainties is large, but all estimates have low precision – the “smalldata, largescale regime.” We formalize a model for this new regime, focusing on optimization problems with uncertain linear objectives. We show that common datadriven methods may perform poorly in this new setting, despite their provably good performance in the traditional largesample regime. Such methods include sample average approximation, “estimatethenoptimize” policies, datadriven robust optimization, and certain regularized policies.
We then propose a novel framework for selecting a datadriven policy from a given policy class. Like the aforementioned datadriven methods, our policy enjoys provably good performance in the largesample regime. Unlike, these methods, however, we show
that in the smalldata, largescale regime, our datadriven policy performs comparably to an oracle bestinclass policy, provided the policy class and estimates satisfy some mild conditions. We specialize and strengthen this result for linear optimization problems and two natural policy classes: the first inspired by the empirical Bayes literature in statistics and the second by the regularization literature in optimization and machine learning. For both classes, we show that the suboptimality gap between our proposed policy and the oracle policy decays exponentially fast in the number of uncertain parameters, even for a fixed amount of data. Thus, these policies retain the strong largesample performance of traditional methods, and additionally enjoy provably strong performance in the smalldata, largescale regime. Numerical experiments confirm the significant benefits of our methods.

 4.
Maximizing Intervention Effectiveness.
with Brian Rongqing Han, SongHee Kim, and Hyung Paek
 Management Science
(Forthcoming).
 [Abstract]
[SSRN]
 **Finalist in the 2018 Pierskalla Best Paper Award**
 **Finalist in the 2018 POMS College of Healthcare and Operations Management (CHOM) Best Paper Competition**

Frequently, policymakers seek to roll out an intervention previously proven effective in a research study, perhaps subject to resource constraints. However, since different subpopulations may respond differently to the same treatment, there is no a priori guarantee that the intervention will be as effective in the targeted population as it was in the study. How then should policymakers target individuals to maximize intervention effectiveness? We propose a novel robust optimization approach that leverages evidence typically available in a published study. Our approach is tractable – realworld instances are easily optimized in minutes with offtheshelf software – and flexible enough to accommodate a variety of resource and fairness constraints. We compare our approach with current practice by proving performance guarantees for both approaches, which emphasize their structural differences. We also prove an intuitive interpretation of our model in terms of regularization, penalizing differences in the demographic distribution between targeted individuals and the study population. Although the precise penalty depends on the choice of uncertainty set, we show that for special cases we can recover classical penalties from the covariate matching literature on causal inference. Finally, using real data from a large teaching hospital, we compare our approach to common practice in the particular context of reducing emergency department utilization by Medicaid patients through case management. We find that our approach can offer significant benefits over common practice, particularly when the heterogeneity in patient response to the treatment is large.

 5.
NearOptimal Bayesian Ambiguity Sets for Distributionally Robust Optimization.
 Management Science
(Mar. 2019).
 [Abstract]
[10.1287/mnsc.2018.3140]
[Optimization Online]
[Code]

We propose a Bayesian framework for assessing the relative strengths of datadriven ambiguity sets in distributionally robust optimization (DRO) when the underlying distribution is defined by a finitedimensional parameter. The key idea is to measure the relative size between a candidate ambiguity set and a specific, asymptotically optimal set. This asymptotically optimal set is provably the smallest convex ambiguity set that satisfies a particular Bayesian robustness guarantee with respect to a given class of constraints as the amount of data grows large. In other words, it is a subset of any other convex set that satisfies the same guarantee. Using this framework, we prove that
existing, popular ambiguity sets based on statistical confidence regions are significantly larger than the asymptotically optimal set with respect to constraints that are concave in the ambiguity– the ratio of their sizes scales with the square root of the dimension of the ambiguity. By contrast, we construct new ambiguity sets that are tractable, satisfy our Bayesian robustness guarantee with finite data and are only a small, constant factor larger than the asymptotically optimal set; we call these sets “Bayesian nearoptimal.” We further prove that, asymptotically, solutions to DRO models with our Bayesian nearoptimal sets enjoy frequentist robustness properties, despite their smaller size. Finally, our framework yields guidelines for practitioners for selecting between competing ambiguity set proposals in DRO. Computational evidence in portfolio allocation using real and simulated data confirms that our framework, although motivated by asymptotic analysis in a Bayesian setting, provides practical insight into the performance of various DRO models with finite data under frequentist assumptions.

 6.
DataDriven Robust Optimization.
with Dimitris Bertsimas and Nathan Kallus
 Mathematical Programming
(Feb. 2017).
 [Abstract]
[10.1007/s1010701711258]
[Code]
 **Finalist in the 2013 George Nicholson Student Paper Prize**

The last decade has seen an explosion in the availability of data for operations research applications as part of the Big Data revolution. Motivated by this data rich paradigm, we propose a novel schema for utilizing data to design uncertainty sets for robust optimization using statistical hypothesis tests. The approach is flexible and widely applicable, and robust optimization problems built from our new sets are computationally tractable, both theoretically and practically. Furthermore, optimal solutions to these problems enjoy a strong, finitesample probabilistic guarantee. We also propose concrete guidelines for practitioners and illustrate our approach with applications in portfolio management and queueing. Computational evidence confirms that our datadriven sets significantly outperform conventional robust optimization techniques whenever data is available.

 7.
Robust Sample Average Approximation.
with Dimitris Bertsimas and Nathan Kallus
 Mathematical Programming
(Jun. 2017).
 [Abstract]
[doi:10.1007/s101070171174z]
 **Winner of the 2013 MIT Operations Research Center Best Student Paper Award**

Sample average approximation (SAA) is a widely approach to datadriven decisionmaking under uncertainty. Under mild assumptions, SAA is both tractable and enjoys strong asymptotic performance guarantees. Similar guarantees, however, do not typically hold in finite samples. In this paper, we propose a modification of SAA, which we term Robust SAA, which retains SAA’s tractability and asymptotic properties and, additionally, enjoys strong finitesample performance guarantees. The key to our method is linking SAA, distributionally robust optimization, and hypothesis testing of goodnessoffit. Beyond Robust SAA, this connection provides a unified perspective enabling us to characterize the finite sample and asymptotic guarantees of various other datadriven procedures that are based upon distributionally robust optimization. We present examples from inventory management and portfolio allocation, and demonstrate numerically that our approach outperforms other datadriven approaches in these applications.

 8.
A Comparison of Monte Carlo Tree Search and Mathematical Optimization for Large Scale Dynamic Resource Allocation.
with Dimitris Bertsimas, John D. Griffith, Mykel Kochenderfer, Velibor Misic, and Robert Moss
 European Journal of Operations Research
(Dec. 2017).
 [Abstract]
[doi:10.1016/j.ejor.2017.05.032]

Dynamic resource allocation (DRA) problems are an important class of dynamic stochastic optimization problems that arise in a variety of important realworld applications. DRA problems are notoriously difficult to solve to optimality since they frequently combine stochastic elements with intractably large state and action spaces. Although the artificial intelligence and operations research communities have independently proposed two successful frameworks for solving dynamic stochastic optimization problems—Monte Carlo tree search (MCTS) and mathematical optimization (MO), respectively—the relative merits of these two approaches are not well understood. In this paper, we adapt both MCTS and MO to a problem inspired by tactical wildfire and management and undertake an extensive computational study comparing the two methods on large scale instances in terms of both the state and the action spaces. We show that both methods are able to greatly improve on a baseline, problemspecific heuristic. On smaller instances, the MCTS and MO approaches perform comparably, but the MO approach outperforms MCTS as the size of the problem increases for a fixed computational budget.

 9.
DataDriven Estimation in Equilibrium Using Inverse Optimization.
with Dimitris Bertsimas and Ioannis Ch. Paschalidis
 Mathematical Programming
(Sep. 2014).
 [Abstract]
[doi:10.1007/s1010701408194]
[arXiv:1308.3397]
[Code]

Equilibrium modeling is common in a variety of fields such as game theory and transportation science. The inputs for these models, however, are often difficult to estimate, while their outputs, i.e., the equilibria they are meant to describe, are often directly observable. By combining ideas from inverse optimization with the theory of variational inequalities, we develop an efficient, datadriven technique for estimating the parameters of these models from observed equilibria. We use this technique to estimate the utility functions of players in a game from their observed actions and to estimate the congestion function on a road network from traffic count data. A distinguishing feature of our approach is that it supports both parametric and nonparametric estimation by leveraging ideas from statistical learning (kernel methods and regularization operators). In computational experiments involving Nash and Wardrop equilibria in a nonparametric setting, we find that a) we effectively estimate the unknown demand or congestion function, respectively, and b) our proposed regularization technique substantially improves the outofsample performance of our estimators.

 10.
A Course on Advanced Software Tools for Operations Research and Analytics.
with Iain Dunning, Angela King, Jerry Kung, Miles Lubin and Jon Silberholz
 INFORMS Transactions on Education
(Jan. 2015).
 [Abstract]
[ited.2014.0131]
[Code/Materials]

In the age of big data analytics, it is increasingly important for researchers and practitioners to be familiar with methods and software tools for analyzing large data sets, formulating and solving largescale mathematical optimization models, and sharing solutions using interactive media. Unfortunately, advanced software tools are seldom included in curricula of graduatelevel operations research (OR) programs. We describe a course consisting of eight threehour modules intended to introduce Master’s and PhD students to advanced software tools for OR: Machine Learning in R, Data Wrangling, Visualization, Big Data, Algebraic Modeling with JuMP, HighPerformance and Distributed Computing, Internet and Databases, and Advanced Mixed Integer Linear Programming (MILP) Techniques. For each module, we outline content, provide course materials, summarize student feedback, and share lessons learned from two iterations of the course. Student feedback was very positive, and all students reported that the course equipped them with software skills useful for their own research. We believe our course materials could serve as a template for the development of effective OR software tools courses and discuss how they could be adapted to other educational settings.

 11.
Inverse Optimization: A New Perspective on the BlackLitterman Model.
with Dimitris Bertsimas and Ioannis Ch. Paschalidis
 Operations Research
(Nov. 2012).
 [Abstract]
[doi:10.1287/opre.1120.1115]

The BlackLitterman (BL) model is a widely used asset allocation model in the financial industry. In this paper, we provide
a new perspective. The key insight is to replace the statistical framework in the original approach with ideas from inverse
optimization. This insight allows us to significantly expand the scope and applicability of the BL model. We provide a
richer formulation that, unlike the original model, is flexible enough to incorporate investor information on volatility and
market dynamics. Equally importantly, our approach allows us to move beyond the traditional meanvariance paradigm of
the original model and construct “BL”type estimators for more general notions of risk such as coherent risk measures.
Computationally, we introduce and study two new “BL”type estimators and their corresponding portfolios: a mean variance
inverse optimization (MVIO) portfolio and a robust mean variance inverse optimization (RMVIO) portfolio. These two
approaches are motivated by ideas from arbitrage pricing theory and volatility uncertainty. Using numerical simulation
and historical backtesting, we show that both methods often demonstrate a better riskreward tradeoff than their BL
counterparts and are more robust to incorrect investor views.
