CEU Electronic Theses and Dissertations, 2020
| Author | Kujbus, Marcell |
|---|---|
| Title | A Model Confidence Set extension of grid search in hyper-parameter optimization |
| Summary | Hyper-parameter optimization is a thoroughly investigated discipline in optimization theory, statistics and machine learning. While it is crucial to configure the hyper-parameters properly, the most widely used grid search technique has some hidden issues that impacts the efficiency of the learning algorithm. Such an issue is that the mentioned method does not take statistical significance of out-of-sample forecasting performance into consideration when choosing the hyper-parameter yielding the minimal loss. The following paper proposes an extension of this approach to make the modeling more robust to random fluctuation. The Model Confidence Set algorithm terminates at a superior set of hyper-parameters that are statistically indistinguishable at a given a significance level with respect to the loss their generated models produce. It is shown empirically and theoretically, that averaging over the predictions of such a superior set is more efficient with respect to out-of-sample loss than taking the prediction of a single realization of the ”minimum -yieldingȁ d; hyper-parameter. The theoretical argument is based on showing a specific example, where the limitations of the grid search method are reached. The empirical study examines the predictability of the evolution of the novel Covid-19 infectious disease in New York state, U.S.A. Extending the famous SEIR dynamic system with gamma distributed latent and infectious periods gives rise to hyper-parameter optimization in disease prediction. |
| Supervisor | Dr. Tamás Ferenci |
| Department | Mathematics MSc |
| Full text | https://www.etd.ceu.edu/2020/kujbus_marcell.pdf |
Visit the CEU Library.
© 2007-2021, Central European University