R-miss-tastic

The ability to easily collect and gather a large amount of data from different sources can be seen as an opportunity to better understand many processes. It has already led to breakthroughs in several application areas. However, due to the wide heterogeneity of measurements and objectives, these large databases often exhibit an extraordinary high number of missing values. Hence, in addition to scientific questions, such data also present some important methodological and technical challenges for data analysts. This tutorial gives an overview of the missing values literature as well as the recent improvements that caught the attention of the community due to their ability to handle large matrices with large amount of missing entries. The methods presented in this tutorial are illustrated on medical, environmental and survey data.

• Slides
• Lecture notes
• Tutorial
• Data used in tutorial.

Missing values are ubiquitous in the practice of data analysis. In this series of lectures, we will start by presenting classical methods for handling missing data (simple imputation, multiple imputation, likelihood-based methods) developed in an inferential framework, where the objective is to best estimate parameters and their variance in the presence of missing data.We will emphasize very powerful methods of simple and multiple imputation based on low-rank approximations that can be applied to heterogeneous data (quantitative, categorical). We will then present recent results in a supervised learning framework. A striking result is that naive imputation strategies (such as mean imputation) can be optimal, as the supervised learning method does the hard work. The fact that such a simple approach can be relevant may have important consequences in practice. We will also discuss how missing value modeling can be easily incorporated into tree models, such as gradient boosted trees, resulting in a learner that has been shown to perform very well, including in challenging non-random missingness settings.Notebooks will be presented. Finally, we will briefly present how such results are useful in the context of causal inference with missing values in the covariates.

• Slides

This course focuses on the theory and methods for missing data analysis. Topics include maximum likelihood estimation under missing data, EM algorithm, Monte Carlo computation techniques, imputation, Bayesian approach, propensity scores, semi-parametric approach, and non-ignorable missing data.

• Introduction
• Likelihood-based approach
• Computation
• Imputation
• Multiple imputation
• Propensity Score approach
• Nonignorable missing data

This course formally introduces methodologies for handling missing data in statistical analyses. It covers naive methods, missing-data assumptions, likelihood-based approaches, Bayesian and multiple imputation approaches, inverse-probability weighting, pattern-mixture models, sensitivity analysis and approaches under nonignorable missingness. Computational tools such as the Expectation-Maximization algorithm and the Gibbs’ sampler will be introduced. This course is intended for students who are interested in methodological research.
Course syllabus

• Lecture 1: syllabus, motivating examples
• Lecture 2: general setup, notation, missing-data mechanisms
• Lecture 3: naive methods: complete-case analysis and imputation
• Lecture 4: R session 1
• Lecture 5: likelihood-based methods
• Lecture 6: the EM algorithm
• Lecture 7: R session 2 (setup), R script
• Lecture 8: introduction to Bayesian inference
• Lecture 9: Gibbs sampling, ignorability under Bayesian inference, data augmentation
• Lecture 10: multiple imputation
• Lecture 11: R session 3 (setup), R script
• Lecture 12: inverse probability weighting
• Lecture 13: introduction to (weighted generalized) estimating equations
• Lecture 14: R session 4 (setup), R script
• Lecture 15: identifiability, nonignorability, pattern-mixture models
• Lecture 16: pattern-mixture models (continued), sensitivity analysis

Exercices/Homework

• Homework 0 (after Lecture 1)
• Homework 1 (after Lecture 4)
• Homework 2 (after Lecture 7)
• Homework 3 (after Lecture 11)
• Homework 4 (after Lecture 14)

This short course on multiple imputation gives an overview of missing data problems, various solutions to tackle them as well as their limitations. It introduces to MI inferences and provides details on implementation and application of MI.

• Multiple Imputation
• Example using mice

These two videos can be viewed independently or as a complement to the above tutorial on Imputation using principal components as they provide detailed explanation on how to use the functions of the missMDA package to visualize and analyze missing values and how to perform (multiple) imputation.

• Handling missing values in PCA
• Handling missing values in MCA

This keynote talk gives an overview of different approaches for inference and prediction tasks. A striking result for the latter is that the widely-used method of imputing with the mean prior to learning can be consistent.

• A missing value tour in R (video)
• Slides of the talk

This course recalls basic concepts of surveys and data collection before discussing how to handle unit non-response and item non-response in surveys.

• Traitement des données manquantes dans les Enquêtes

In follow-up studies different types of outcomes are typically collected for each subject. These include longitudinally measured responses (e.g., biomarkers), and the time until an event of interest occurs (e.g., death, dropout). Often these outcomes are separately analyzed, but in many occasions it is of scientific interest to study their association. This type of research question has given rise in the class of joint models for longitudinal and time-to-event data. These models constitute an attractive paradigm for the analysis of follow-up data that is mainly applicable in two settings: First, when focus is on a survival outcome and we wish to account for the effect of endogenous time-dependents covariates measured with error, and second, when focus is on the longitudinal outcome and we wish to correct for non-random dropout. This course is aimed at applied researchers and graduate students, and will provide a comprehensive introduction into this modeling framework. It provides explanation when these models should be used in practice, which are the key assumptions behind them, and how they can be utilized to extract relevant information from the data. Emphasis is given on applications, and after the end of the course participants will be able to define appropriate joint models to answer their questions of interest.

• Joint Modelling of Longitudinal and Time to Event Data

This tutorial gives a short overview about methods for missing data in time series in R in general and subsequently introduces the imputeTS package. The imputeTS package is specifically made for handling missing data in time series and offers several functions for visualization and replacement (imputation) of missing data. Based on usage examples it is shown how imputeTS can be used for time series imputation.

• How to deal with Missing Data in Time Series and the imputeTS package

While the problem of missing values in the covariates has been considered very early in the causal inference literature, it remains difficult for practitioners to know which method to use, under which assumptions the different approaches are valid and whether the tools developed are also adapted to more complex data, e.g., for high-dimensional or mixed data. This talk provides a rigorous classification of existing methods according to the main underlying assumptions, which are based either on variants of the classical unconfoundedness assumption or relying on assumptions about the mechanism that generates the missing values. It also highlights two recent contributions on this topic: first an extension of classical doubly robust estimators that allows handling of missing attributes and second an approach to causal inference based on variational autoencoders in the case of latent confounding.

• Video
• Slides of the talk

The estimation of count data, such as bird abundance, is an important task in many disciplines and can be used for instance by ecologists for species conservation. Collecting count data is often subject to inaccuracies and missing data due to the nature of the counted object and due to multiplicity of actors/sensors collecting the data over more or less long periods of time. Methods such as Correspondence Analysis or Generalized Linear Models can be used to estimate these missing values and allow a more accurate analyses of the count data. The objective of this project is to investigate the abundance for the Eurasian Coot, which is mainly observed in the mediterranean part of North-Africa, and its relation to external geographical and meteorological factors. First, different methods are compared in terms of accuracy, using R packages glm, Rtrim, Lori and missMDA. Afterwards, external factors and their impact on bird abundance are examined and finally the temporal trend is investigated to determine whether the Eurasian coot is declining or not.
This project was carried out in collaboration with the Research Institute for the conservation of Mediterranean wetlands, the association Les Amis des Oiseaux (Friends of the birds) and the Office National de la Chasse et de la Faune Sauvage (National Agency for Hunting and Wildlife).

• Estimation of species abundance using log-linear models (slides)
• Estimation of species abundance using log-linear models (project report)

• useR! tutorial on handling missing values

  (Julie Josse & Nicholas Tierney, 2018)

• Handling missing values in PCA
• Handling missing values in MCA

• mice vignette: Ad hoc methods and mice

  (Stef van Buuren and Gerko Vink, 2018)

• Other mice vignettes

  (Stef van Buuren and Gerko Vink, 2018)

• Multiple imputation with the mice package

  (Nicole Erler, NIHES course on multiple imputation, 2018)

• Multiple imputation in complex settings (using mice, JointAI, smcfs, jomo)

  (Nicole Erler, NIHES course on multiple imputation, 2018)

• Example using mice

  (Jerry Reiter, 2018)

• missForest vignette: Using the missForest Package

  (Daniel J. Stekhoven, 2012)