> For the complete documentation index, see [llms.txt](https://lichangbin.gitbook.io/paper_notes/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://lichangbin.gitbook.io/paper_notes/editable-neural-networks.md). # Editable Neural Networks ## Motivation it is crucially important to correct model mistakes quickly as they appear during training the neural network. In this work, we investigate the problem of neural network editing — how one can efficiently patch a mistake of the model on a particular sample, without influencing the model behavior on other samples. Namely, we propose Editable Training, a model-agnostic training technique that encourages fast editing of the trained model. ## Editing Neural Networks * $$f(x,\theta)$$ : a neural network * $$\mathcal{L}\_{base}(\theta)$$ : task-specific objective loss function The goal is to change model's predictions on a subset of inputs, corresponding to misclassified objects, without affecting other inputs, by changing the model parameters $$\theta$$ . An editor function could be used to formalized this: $$\hat{\theta}=Edit(\theta,l\_e)$$ , with a constraint: $$l\_e(\hat{\theta})\leq0$$ * $$\hat{\theta}$$ is the changed parameters For example, multi-class classification. * $$l\_e(\hat{\theta}) = \max\_{y\_i} (\log p(y\_i|x, \hat{\theta}) - \log p(y\_{ref}|x,\hat{\theta}))$$ where $$y\_{ref}$$ is the desired label. if under the constraint: $$l\_e(\hat{\theta})\leq0$$ , The constraint: $$l\_e(\hat{\theta})\leq0$$ is satisfied iff $$\arg \max\_{y\_i} \log p(y\_i|x, \hat{\theta})=y\_{ref}$$ So the goal is how to design the editor neural network. ## Reference: * * *