Prof. C. F. Jeff Wu, gave a lecture titled “A hierarchical expected improvement method for Bayesian optimization” at AMSS on 9, September 2020.

The Expected Improvement (EI) method is a widely-used Bayesian optimization method, which makes use of a fitted Gaussian process model for efficient black-box optimization. However, one key drawback of EI is that it is overly greedy in exploiting the fitted Gaussian process model, which results in suboptimal solutions. We propose a new hierarchical EI (HEI) framework, which makes use of a hierarchical Gaussian process model. HEI preserves a closed-form acquisition function, and corrects the over-greediness of EI by encouraging exploration. Under certain prior specifications, we prove the global convergence of HEI over a broad function space, and derive global convergence rates under smoothness assumptions on the objective function. We then introduce hyperparameter estimation methods which allow HEI to mimic a fully Bayesian procedure while avoiding expensive Markov-chain Monte Carlo sampling. Numerical experiments and a toy semiconductor optimization application show the improvement of HEI over existing black-box optimization methods.

C. F. Jeff Wu is Professor and Coca Cola Chair in Engineering Statistics at the School of Industrial and Systems Engineering, Georgia Institute of Technology. He was elected a Member of the National Academy of Engineering (2004), and a Member (Academician) of Academia Sinica (2000). A Fellow of American Society for Quality, of Institute of Mathematical Statistics, of INFORMS, and of American Statistical Association. He received the COPSS Presidents’ Award in 1987, COPSS Fisher Lecture Award in 2011, Deming Lecture in 2012. He has won other awards, including the Shewhart Medal (2008), the Pan Wenyuan Technology Award (2008), Class of 1934 Distinguished Professor Award and Sigma Xi Monie A. Ferst Award both at Georgia Institute of Technology in 2020. He was the 1998 Mahalanobis Memorial Lecturer at the Indian Statistical Institutes, received the inaugural Akaike Memorial Lecture Award in 2016 sponsored by the Japan Statistical Society and the Institute of Statistical Mathematics, Tokyo, the 2017 Box Medal from ENBIS, and an honorary doctor degree at the University of Waterloo.