On this episode of Bit of A Tangent, we discuss the emerging field of self-supervised machine learning. This is an immensely exciting area of active research in machine learning and AI - one which most people haven’t even heard about yet! We build up to the intuition for the topic by covering supervised and unsupervised learning; autoencoders and dimensionality reduction, and exploring how these techniques could be applied to Gianluca’s Quantified Self n=1 sleep quality dataset. We culminate in a detailed discussion of the state-of-the-art Contrastive Predictive Coding model, and how it allows us to learn about the structure of the world, without tonnes of labelled training data!

--------

Shownotes:

--------

Bit of a Tangent on Twitter (www.twitter.com/podtangent) and Instagram (instagram.com/podtangent/)

Summer school on Computational Neuroscience: http://imbizo.africa/

Control problem in AI: https://intelligence.org/stanford-talk/

Coordination problem: https://conceptually.org/concepts/coordination-problems

Deep learning overview: https://lilianweng.github.io/lil-log/2017/06/21/an-overview-of-deep-learning.html

t-SNE explained: https://mlexplained.com/2018/09/14/paper-dissected-visualizing-data-using-t-sne-explained/

Variational autoencoders explained: https://anotherdatum.com/vae.html

Self-supervised learning by fast.ai: https://www.fast.ai/2020/01/13/self_supervised/

CPC model papers on Arxiv: https://arxiv.org/pdf/1807.03748.pdf https://arxiv.org/pdf/1905.09272.pdf

Blog posts explaining CPC: https://lilianweng.github.io/lil-log/2019/11/10/self-supervised-learning.html

https://yann-leguilly.gitlab.io/post/2019-09-29-representation-learning-with-contrastive-predictive-coding/

https://mf1024.github.io/2019/05/27/contrastive-predictive-coding/