NIPS 2011 Workshop on
Copulas in Machine Learning

December 16th, 2011, Sierra Nevada, Spain
Hotel Melia Sierra Nevada, Genil room

Overview Schedule Abstract, slides, papers Submissions

Keynote: Everything You Always Wanted to Know About Copulas and Were Afraid to Ask

The speaker will give an introduction to dependence modelling based on copulas. Basic concepts will first be presented in a data analytic perspective. Broad strategies for copula model building will then be described, and some of the most common constructions will be critically reviewed. In the second part of his talk, the speaker will show how graphical tools and nonparametric inference techniques can be used for copula model fitting and validation. Procedures will be illustrated with concrete data; special attention will be paid to phenomena exhibiting joint extreme behavior.

Slides (pdf) Introductory paper (pdf)

Invited: Exploiting Copula Parameterizations in Graphical Model Construction and Learning

Graphical models and copulas are two sets of tools for multivariate analysis. Both are in some sense pathways to the construction of multivariate distributions using modular representations. The former focuses on languages to express conditional independence constraints, factorizations and efficient inference algorithms. The latter allows for the encoding of some marginal features of the joint distribution (univariate marginals, in particular) directly, without resorting to an inference algorithm. In this talk we exploit copula parameterizations in two graphical modeling tasks: parameterizing decomposable models and building proposal distributions for inference with Markov chain Monte Carlo; parameterizing directed mixed graph models and providing simple estimation algorithms based on composite likelihood methods.

Slides (pdf)

Tutorial: Introduction to Vine Models

In this talk, we introduce the main concepts of vine pair-copula constructions. This framework uses bivariate copulas as building blocks to obtain higher-dimensional distributions. As these bivariate copulas can be selected from a wide range of families, the vine approach leads to a more flexible model compared to traditional approaches. For the estimation of vine pair-copulas, we introduce a mathematically elegant framework that joins (a) graph theory, to determine the dependency structure of the data, and (b) maximum-likelihood estimation, to fit bivariate copulas.

Web site        Slides (pdf)

High-dimensional Copula Constructions in Machine Learning: An Overview

With the ``discovery'' of copulas by machine learning researchers, several works have emerged that focus on the high-dimensional scenario. This talk will provide a brief overview of these works and cover tree-averaged distributions (Kirshner), the nonparanormal (Liu, Lafferty and Wasserman), copula processes (Wilson and Ghahramani), kernel-based copula processes (Jaimungal and Ng), and copula networks (Elidan). Special emphasis will be given to the high level similarities and differences between these works.

Slides (pdf)

Copula Mixture Model for Dependency-seeking Clustering

We introduce a Dirichlet prior mixture of meta-Gaussian distributions to perform dependency-seeking clustering when co-occurring samples from different data sources are available. The model extends Bayesian mixtures of Canonical Correlation Analysis clustering methods to multivariate data distributed with arbitrary continuous margins. Using meta-Gaussian distributions gives the freedom to specify each margin separately and thereby also enables clustering in the joint space when the data are differently distributed in the different views. The Bayesian mixture formulation retains the advantages of using a Dirichlet prior. We do not need to specify the number of clusters and the model is less prone to overfitting than non-Bayesian alternatives. Inference is carried out using a Markov chain sampling method for Dirichlet process mixture models with non-conjugate prior adapted to the copula mixture model. Results on different simulated data sets show significant improvement compared to a Dirichlet prior Gaussian mixture and a mixture of CCA model.

Slides (pdf) Poster (pdf)

Expectation Propagation for the Estimation of Conditional Bivariate Copulas

We present a semi-parametric method for the estimation of the copula of two random variables X and Y when conditioning to an additional covariate Z. The conditional bivariate copula is described using a parametric model fully specified in terms of Kendall's tau. The dependence of the conditional copula on Z is captured by expressing tau as a function of Z. In particular, tau is obtained by filtering a non-linear latent function, which is evaluated on Z, through a sigmoid-like function. A Gaussian process prior is assumed for the latent function and approximate Bayesian inference is performed using expectation propagation. A series of experiments with simulated and real-world data illustrate the advantages of the proposed approach.

Slides (pdf)

Robust Nonparametric Copula Based Dependence Estimators

A fundamental problem in statistics is the estimation of dependence between random variables. While information theory provides standard measures of dependence (e.g. Shannon-, Renyi-, Tsallis-mutual information), it is still unknown how to estimate these quantities from i.i.d. samples in the most efficient way. In this presentation we review some of our recent results on copula based nonparametric dependence estimators and demonstrate their robustness to outliers both theoretically in terms of finite-sample breakdown points and by numerical experiments in independent subspace analysis and image registration.

Slides (pdf)


  • New Classes of Copula Gaussian Graphical Models
    Justin Dauwels, Hang Yu, Xueou Wang, Xu Zhang and Shiyan Xu
    Spotlight (PDF)

  • Applications of Copulas in Neuroscience
    Arno Onken, Steffen Grunewalder, Matthias Munk and Klaus Obermayer
    Spotlight (PDF)

  • Modeling Cell Populations in High Content Screening using Copulas
    Edouard Pauwels
    Spotlight (PDF)

  • Transfer Learning with Copulas
    David Lopez Paz and Jose Miguel Hernandez-Lobato
    Spotlight (PDF)

  • Copula functions for learning multimodal densities with non-linear dependencies
    Ashutosh Tewari, Madhusudana Shashanka, Michael Giering
    Spotlight (PDF) Poster (PDF)

  • C-Vines
    Nicole Kraemer and Ulf Schepsmeier
    Spotlight (PDF)

  • Copulas for Context Sensitive Classification Ashish Kapoor
    Spotlight (PDF)