N1H111SM's Miniverse

Contrastive Predictive Coding

字数统计: 125阅读时长: 1 min
2020/05/28 Share


  • paper “Contrastive Predictive Coding”

The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models. We use a probabilistic contrastive loss which induces the latent space to capture information that is maximally useful to predict future samples.

Predictive coding在信号处理领域是常见的unsupervised learning方法. 本文的方法流程为:

  • First, we compress high-dimensional data into a much more compact latent embedding space in which conditional predictions are easier to model.
  • Secondly, we use powerful autoregressive models in this latent space to make predictions many steps in the future.
  • Finally, we rely on Noise-Contrastive Estimation for the loss function.