- paper “Contrastive Predictive Coding”
The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models. We use a probabilistic contrastive loss which induces the latent space to capture information that is maximally useful to predict future samples.
Predictive coding在信号处理领域是常见的unsupervised learning方法. 本文的方法流程为：
- First, we compress high-dimensional data into a much more compact latent embedding space in which conditional predictions are easier to model.
- Secondly, we use powerful autoregressive models in this latent space to make predictions many steps in the future.
- Finally, we rely on Noise-Contrastive Estimation for the loss function.