> For the complete documentation index, see [llms.txt](https://lichangbin.gitbook.io/paper_notes/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://lichangbin.gitbook.io/paper_notes/oct/meta-learning-acquisition-functions-for-transfer-learning-in-bayesian-optimization.md).

# Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization

ICLR 20  10-2-2020

## Motivation

NOTE: read this paper again in details. It looks a novel direction

**Transferring knowledge across tasks to improve data-efficiency** is one of the open key challenges in the field of **global black-box optimization**. (how about not black box?) 

limitation of current method:

Readily available algorithms are typically designed to be universal optimizers and, therefore, often **suboptimal for specific tasks**. 

**Method in the paper:**

We propose **a novel transfer learning method to obtain customized optimizers** within the well-established framework of **Bayesian optimization**, allowing our algorithm to utilize the proven generalization capabilities of Gaussian processes surrogate model. 

**Using reinforcement learning to meta-train an acquisition function (AF)** on a set of related tasks, the proposed method learns to **extract implicit structural information** and to exploit it for improved data-efficiency.

## Goal : Global Black-Box Optimization problem

![](https://1687130946-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MEnQbUIupyAn8eMmrmG%2F-MKaKVTb8_m8voaUp7_r%2F-MKaYQ9gO2Xre24ZAR9E%2Fimage.png?alt=media\&token=bfcfdef8-eaf0-4d82-aa8f-729ad766b71a)

Usually Bayesian Optimization (BO) is used for the problem

* Probabilistic surrogate model (e.g., GP) to interpolate between data points
* Sampling strategy (acquisition function, AF) based on surrogate model

Transfer learning is used to increase the data-efficiency by transferring knowledge across task instances

##  MetaBO

Retain the proven structure of BO, keep the powerful GP surrogate model

![](https://1687130946-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MEnQbUIupyAn8eMmrmG%2F-MKaKVTb8_m8voaUp7_r%2F-MKaZP15KwjD4CSi60L4%2Fimage.png?alt=media\&token=a8b33fa7-68a7-431a-9670-bef0c29c4cb8)

Replace the AF part with neural AF to obtain task-specific AFs by transfer learning

![](https://1687130946-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MEnQbUIupyAn8eMmrmG%2F-MKaKVTb8_m8voaUp7_r%2F-MKaZeuWlzmP3mxV6Mq-%2Fimage.png?alt=media\&token=3f031d99-0602-4fa3-8591-6def6e20157e)

Train neural AFs using RL: so no need for gradients of $$f\in \mathcal{F}$$ 

## Reference

* <https://openreview.net/forum?id=ryeYpJSKwr>
* <https://iclr.cc/virtual_2020/poster_ryeYpJSKwr.html>
* <https://metalearning-cvpr2019.github.io/assets/CVPR_2019_Metalearning_Tutorial_Frank_Hutter.pdf>
*