> For the complete documentation index, see [llms.txt](https://lichangbin.gitbook.io/paper_notes/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://lichangbin.gitbook.io/paper_notes/oct/conditional-neural-progress.md).

# Conditional Neural Progress (CNPs)

## Motivation

![](/files/-MKaKHaNr16uEApMPncN)

CNPs combine benefits of NNs and GPs:

* the flexibility of stochastic processes such as GPs
* structured as NNs and trained via Gradient Descent from data directly

## Model

![](/files/-MKaHXgFo4bTz9Ftznpc)

we have a function $$f(x\_i)=y\_i$$ with input $$x\_i$$ and output $$y\_i$$ .

$$f()$$ is drawn from $$P$$ , a distribution over functions.

Define two sets:

* Observations: $$O = {(x\_i, y\_i)}$$ 
* Targets: $$T = {x\_j}$$ 

Our goal is: given some observations, we want to be able to make predictions at unseen target inputs at test time. Just like supervise learning.

The architecture of our model captures this task:

![](/files/-MKaIhXYoxS_eNbOEnsS)

* $$r\_i$$ are the representations of the pairs $${(x\_i, y\_i)}$$ 
* $$r$$ is the overall representation obtained by summing all $$r\_i$$ 
* $$h\_{\theta}$$ and $$g\_{\theta}$$ are NNs
* $$\phi\_i$$ parametrizes the output distribution (either a Gaussian or a categorical distribution)

#### Key properties of the model:

* CNPs are conditional distributions over functions trained to model the empirical conditional distributions of functions $$f\sim P$$ .
* CNPs are permutation invariant in O and T.
* scalable, achieving a running time complexity of $$O(m+n)$$ for making m predictions with n observations.

## Reference

* <https://arxiv.org/pdf/1807.01613.pdf>
* <https://vimeo.com/312299226>
* <https://www.martagarnelo.com/projects>
* <https://github.com/deepmind/neural-processes>