📒
PaperNotes
  • PAPER NOTES
  • Meta-Learning with Implicit Gradient
  • DARTS: Differentiable Architecture Search
  • Meta-Learning of Neural Architectures for Few-Shot Learning
  • Towards Fast Adaptation of Neural Architectures with Meta Learning
  • Editable Neural Networks
  • ANIL (Almost No Inner Loop)
  • Meta-Learning Representation for Continual Learning
  • Learning to learn by gradient descent by gradient descent
  • Modular Meta-Learning with Shrinkage
  • NADS: Neural Architecture Distribution Search for Uncertainty Awareness
  • Modular Meta Learning
  • Sep
    • Incremental Few Shot Learning with Attention Attractor Network
    • Learning Steady-States of Iterative Algorithms over Graphs
      • Experiments
    • Learning combinatorial optimization algorithms over graphs
    • Meta-Learning with Shared Amortized Variational Inference
    • Concept Learners for Generalizable Few-Shot Learning
    • Progressive Graph Learning for Open-Set Domain Adaptation
    • Probabilistic Neural Architecture Search
    • Large-Scale Long-Tailed Recognition in an Open World
    • Learning to stop while learning to predict
    • Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift
    • Learning to Generalize: Meta-Learning for Domain Generalization
  • Oct
    • Meta-Learning Acquisition Functions for Transfer Learning in Bayesian Optimization
    • Network Architecture Search for Domain Adaptation
    • Continuous Meta Learning without tasks
    • Learning Causal Models Online
    • Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examples
    • Conditional Neural Progress (CNPs)
    • Reviving and Improving Recurrent Back-Propagation
    • Meta-Q-Learning
    • Learning Self-train for semi-supervised few shot classification
    • Watch, Try, Learn: Meta-Learning from Demonstrations and Rewards
  • Nov
    • Neural Process
    • Adversarially Robust Few-Shot Learning: A Meta-Learning Approach
    • Learning to Adapt to Evolving Domains
  • Tutorials
    • Relax constraints to continuous
    • MAML, FO-MAML, Reptile
    • Gradient Descent
      • Steepest Gradient Descent
      • Conjugate Gradient Descent
  • KL, Entropy, MLE, ELBO
  • Coding tricks
    • Python
    • Pytorch
  • ml
    • kmeans
Powered by GitBook
On this page
  • Motivation
  • Overview
  • A schematic of our problem setting and approach
  • Reference

Was this helpful?

  1. Sep

Adaptive Risk Minimization: A Meta-Learning Approach for Tackling Group Shift

9-28-2020

Motivation

A fundamental assumption of most machine learning algorithms is that the training and test data are drawn from the same underlying distribution.

However, this assumption is violated in almost all practical applications: machine learning systems are regularly tested on data that are structurally different from the training set, either due to temporal correlations, particular end users, or other factors.

structural shift or group shift

Imagine an image classification system: After training on a large database of past images, this system is deployed to specific end users. Each user takes photos with differing cameras, locations, and subjects, leading to shift in the input distribution.

This test scenario must be carefully considered when building machine learning systems for real world applications.

Overview

We aim to get both robustness and performance.

To do so, we introduce a new assumption to tackling group shift — that we get the unlabeled test data at once in a batch (only available at test time)

Different from in domain adaptation and transductive learning settings

  • we do not need to specify the test distribution at training time

  • we are not limited to a single test distribution

This assumption is reasonable in many standard supervised learning setups.

Key idea: Rather than trying to be robust, we train a meta-learning algorithm to adapt to group distribution shift with unlabeled data.

We acquire such models by learning to adapt to training batches sampled according to different sub-distributions (groups, tasks), which simulate structural distribution shifts that may occur at test time.

adaptive risk minimization (ARM), a formalization of this setting that lends itself to meta-learning methods.

A schematic of our problem setting and approach

NOTE: update the note based on the newer version of the paper.

Reference

PreviousLearning to stop while learning to predictNextLearning to Generalize: Meta-Learning for Domain Generalization

Last updated 4 years ago

Was this helpful?

Introduction to Federated Learning:

https://arxiv.org/pdf/2007.02931.pdf
https://github.com/henrikmarklund/arm
https://sites.google.com/view/adaptive-risk-minimization
https://www.youtube.com/watch?v=STxtRucv_zo&list=PLvOO0btloRns6egXueiRju4DXQjNRJQd5&index=4