Neutral residues: revisiting adapters for model extension
LLM;model extension
Yes
No ethics review needed.
- It would be useful to clarify the introduction and method section to make it more clear what the exact contributions are. Especially in the method section, it is unclear which aspects of the approach are novel.
- A variety of architecture choices were made based on preliminary experiments that are not shown in the paper. It would be useful to include these results in the appendix to support these decisions.
- The experiments show some improvement on the trade-off between the original domain and the adaptation domain.
- The idea of optimizing for sparse output if the input follows the training distribution is interesting and seems like a plausible way to maintain the original model performance.
This paper proposes neutral residues, an improvement on adapters that allows for domain adaptation while preserving the model performance in the original domain. Neutral residues are additional feed-forward gated adapter blocks added to the model, which are optimized such that the if the input is in the pretraining distribution, the adapter output is sparse. The paper studies the effect of factors such as percent of data from the original distribution, adapter architecture, adapter initialization, and adapter training loss.

In experiments for English and multilingual models with French and German finetuning datasets, neutral residues show some improvement over other domain adaptation approaches (full fine-tuning, LoRA, and vanilla adapters) in terms of the trade-off between retaining the model's original knowledge (English perplexity and benchmarks) and learning the new domain (French/German perplexity and benchmarks).
- Other than the use case provided in the experiments, when is this approach useful instead of something like LoRA or fine-tuning? It seems like the application in the experiments is for a very specific use case where one large domain adaptation would need to be applied, but in real-world settings there are often multiple downstream tasks that would need to be adapted to.
- The additional 20% of parameters seems very high, especially for larger model sizes. It would be valuable to see the results for other domain adaptation methods with varying numbers of additional parameters in Table 3 to provide stronger evidence for the method.
- The experiments would be strengthened by timing comparisons during training and inference. It is not clear to me what the computational cost of this approach is when compared to the other domain adaptation approaches.
Yes
No ethics review needed.
What are the languages and datasets used to train 'Transformer Multilingual' described in Appendix A?
The paper addresses an important question: how do we extend a pretrained language model to a new language, without hurting original performance.
This paper proposes a new training recipe, with data, architecture, and loss innovations, for adapting a pretrained language model to a new language.
1. More languages are needed to validate the claims. Currently the extensions considered are French and German, which are arguably much more similar to English, syntax- and lexicon-wise, than many other human languages. To show the effectiveness of the proposed method, the authors should consider evaluating on languages that are known to be under-represented (_e.g._, tasks from the XTREME-UP dataset).
2. The assumption of access to a 'similar [pretraining] distribution' (Sec 3) is unrealistic in many cases. However given access to the original checkpoint, there are ways to mitigate forgetting with anchors (e.g., [Agarwal _et al._ (2024)](https://arxiv.org/abs/2306.13649).) The authors should evaluate whether such approaches are effective.
Yes
No ethics review needed.
1. is there ablations about different activation choices?Why did you use Silu and Elu activations? 
2. does your method still works best if the amount of training data is less than what's used in the experiments?
1. this paper addresses the problem of efficient adaptation of LLMs to new knowledge without forgetting, which is an important problem for practical usages of these models.
2. the proposed architecture is relatively novel, although the presentation is lacking.
3. The paper includes thorough evaluations of factors like initialization, data mixing, and architecture choices.
this paper proposes a new adapter architecture that is designed to extend a pretrained model to new domain/language by continue training on new data mixture while freezing the backbone model. The goal is to improve the model performance on new language while incurring minimal forgetting on the pretraining domain/language. The adapter contains several gating mechanism as seen in Figure 2 of the paper. Experiments are done comparing to LoRA, vanilla adapters, full fine-tuning on both open-sourced and closed-sourced models which shows that the proposed method has the best trade-off.
1. this paper is not very well-written so it is difficult to fully asses the content. Section 3 discusses adapter gating and local loss, but I still don't fully understand what each component is like. It is better to write down how the input is transformed through the adapter layer using math formulas.
2. there are some architecture choices that are not clearly explained. Why did you use Silu and Elu activations? 
3. on line 315 the authors mentions that the training batch size is 64 and 8, which is quite small. This might make full fine-tuning more unstable. This might not be a fair comparison between different methods. 
4. In table 8 the authors show the trade-off between different learning rates, but it's not clear what data mixture it's using. The percentage of new data can affect the conclusion too.
We add new knowledge to a pretraining large language model without forgetting its original capabilities
@inproceedings{
anonymous2024neutral,
title={Neutral residues: revisiting adapters for model extension},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=8ZPLn3GCDb},
note={under review}
}
We address the problem of extending a pretrained large language model to a new domain that was not seen at training time, like adding a language for which the original model has seen no or little training data.  Popular solutions like fine-tuning or low-rank adaptation are successful at domain adaptation, but formally they do not add any extra capacity and degrade the performance in the original domain. 

Our paper analyzes this extension problem under three angles: data, architecture and training procedure, which are advantageously considered jointly. In particular, we improve adapters and make it possible to learn an entire new language while ensuring that the output of the neural network is  almost unchanged in the original domain. For this purpose, we modify the new residual blocks in a way that leads each new residual block to output near-zeros in the original domain. 

This solution of neutral residues, which borrows architectural components from mixture of experts, is effective: with only 20% extra learnable weights compared to an original model trained on English, we get results that are significantly better than concurrent approaches (fine-tuning, low-rank or vanilla adapters) in terms of the trade-off between learning a new language and not forgetting English.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
LLM model extension
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/4dac17773423f8a4947783f212f262717fe1d1c7.pdf
transfer learning, meta learning, and lifelong learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Neutral residues: revisiting adapters for model extension
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission