Efficient Global Optimization: Motivation, Variations, and Applications

A popular optimization method of a black box objective function is Efficient Global Optimization (EGO), also known as Sequential Model Based Optimization, SMBO, with kriging and expected improvement. EGO is a sequential design of experiments aiming at gaining as much information as possible from as few experiments as feasible by a skillful choice of the factor settings in a sequential way. In this paper we will introduce the standard procedure and some of its variants. In particular, we will propose some new variants like regression as a modeling alternative to kriging and two simple methods for the handling of categorical variables, and we will discuss focus search for the optimization of the infill criterion. Finally, we will give relevant examples for the application of the method. Moreover, in our group, we implemented all the described methods in the publicly available R package mlrMBO.