Statistical Machine Learning for Large and Unstructured Data

Course Overview

We introduce a combination of cutting-edge data analysis methods for analyzing large and unstructured data. The goal is for attendees to become familiar with methods lying at the frontier of research and applications and how to apply them in practice. We will discuss several technical details on how these methods and algorithms work. We focus on problems where the premiere goal is not a prediction but one wants to obtain statistically valid inferences on parameters, hypotheses, or probabilistic forecasts. We emphasize Bayesian methodology as a natural probabilistic machine learning framework for such tasks. Recent advances in the statistical, Bayesian, and machine learning literature will be the core of the lectures.

We start introducing the basics of probabilistic modeling, emphasizing how Bayesian learning allows us to build models (painlessly) describing complex dependencies while quantifying uncertainty in the unknown parameters. Then, we discuss the high-dimensional linear regression model, introducing penalized likelihood and Bayesian methods for high-dimensional regression. Rather than using these methods for prediction, we will discuss how to use them for inference. In particular, we will look at treatment effect estimation. Finally, we discuss latent discrete variable models helpful in analyzing unstructured data, with a main focus on text.

Parallel to the methodologies, we will introduce modern computational methods and programming software that allow one to deploy efficiently the methods discussed and showcase how to do so via case studies. Here, we will describe the technical details of the algorithms and some publicly available implementations. We will emphasize some applications in Economics and the Social Sciences, although the presented ideas are widely applicable to other fields, and we also present examples from disciplines such as Biomedicine. The practical session will mostly be based on R, but most techniques discussed have a Python implementation as well.