📒
PaperNotes
  • PAPER NOTES
  • Meta-Learning with Implicit Gradient
  • DARTS: Differentiable Architecture Search
  • Meta-Learning of Neural Architectures for Few-Shot Learning
  • Towards Fast Adaptation of Neural Architectures with Meta Learning
  • Editable Neural Networks
  • ANIL (Almost No Inner Loop)
  • Meta-Learning Representation for Continual Learning
  • Learning to learn by gradient descent by gradient descent
  • Modular Meta-Learning with Shrinkage
  • NADS: Neural Architecture Distribution Search for Uncertainty Awareness
  • Modular Meta Learning
  • Sep
    • Incremental Few Shot Learning with Attention Attractor Network
    • Learning Steady-States of Iterative Algorithms over Graphs
      • Experiments
    • Learning combinatorial optimization algorithms over graphs
    • Meta-Learning with Shared Amortized Variational Inference
    • Concept Learners for Generalizable Few-Shot Learning
    • Progressive Graph Learning for Open-Set Domain Adaptation
    • Probabilistic Neural Architecture Search
    • Large-Scale Long-Tailed Recognition in an Open World
    • Learning to stop while learning to predict
    • Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift
    • Learning to Generalize: Meta-Learning for Domain Generalization
  • Oct
    • Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization
    • Network Architecture Search for Domain Adaptation
    • Continuous Meta Learning without tasks
    • Learning Causal Models Online
    • Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples
    • Conditional Neural Progress (CNPs)
    • Reviving and Improving Recurrent Back-Propagation
    • Meta-Q-Learning
    • Learning Self-train for semi-supervised few shot classification
    • Watch, Try, Learn: Meta-Learning from Demonstrations and Rewards
  • Nov
    • Neural Process
    • Adversarially Robust Few-Shot Learning: A Meta-Learning Approach
    • Learning to Adapt to Evolving Domains
  • Tutorials
    • Relax constraints to continuous
    • MAML, FO-MAML, Reptile
    • Gradient Descent
      • Steepest Gradient Descent
      • Conjugate Gradient Descent
  • KL, Entropy, MLE, ELBO
  • Coding tricks
    • Python
    • Pytorch
  • ml
    • kmeans
Powered by GitBook
On this page
  • Motivation
  • Model
  • Reference

Was this helpful?

  1. Oct

Conditional Neural Progress (CNPs)

ICML 18 10-23-2020

PreviousMeta-Dataset: A Dataset of Datasets for Learning to Learn from Few ExamplesNextReviving and Improving Recurrent Back-Propagation

Last updated 4 years ago

Was this helpful?

Motivation

CNPs combine benefits of NNs and GPs:

  • the flexibility of stochastic processes such as GPs

  • structured as NNs and trained via Gradient Descent from data directly

Model

Define two sets:

Our goal is: given some observations, we want to be able to make predictions at unseen target inputs at test time. Just like supervise learning.

The architecture of our model captures this task:

Key properties of the model:

  • CNPs are permutation invariant in O and T.

Reference

we have a function f(xi)=yif(x_i)=y_if(xi​)=yi​ with input xix_ixi​ and output yiy_iyi​ .

f()f()f() is drawn from PPP , a distribution over functions.

Observations: O={(xi,yi)}O = \{(x_i, y_i)\}O={(xi​,yi​)}

Targets: T={xj}T = \{x_j\}T={xj​}

rir_iri​ are the representations of the pairs {(xi,yi)}\{(x_i, y_i)\}{(xi​,yi​)}

rrr is the overall representation obtained by summing all rir_iri​

hθh_{\theta}hθ​ and gθg_{\theta}gθ​ are NNs

ϕi\phi_iϕi​ parametrizes the output distribution (either a Gaussian or a categorical distribution)

CNPs are conditional distributions over functions trained to model the empirical conditional distributions of functions f∼Pf\sim Pf∼P .

scalable, achieving a running time complexity of O(m+n)O(m+n)O(m+n) for making m predictions with n observations.

https://arxiv.org/pdf/1807.01613.pdf
https://vimeo.com/312299226
https://www.martagarnelo.com/projects
https://github.com/deepmind/neural-processes