Mean Estimation and Regression Under Heavy-Tailed Distributions: A Survey

Authors: Gábor Lugosi and Shahar Mendelson

Foundations of Computational Mathematics, Vol. 19, 1145–1190, August, 2019

We survey some of the recent advances in mean estimation and regression function estimation. In particular, we describe sub-Gaussian mean estimators for possibly heavy-tailed data both in the univariate and multivariate settings. We focus on estimators based on median-of-means techniques but other methods such as the trimmed mean and Catoni's estimator are also reviewed. We give detailed proofs for the cornerstone results. We dedicate a section on statistical learning problems--in particular, regression function estimation--in the presence of possibly heavy-tailed data.