PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patches
Large Language Models;NLP;Efficiency;Fine-tuning;Efficient Fine-tuning;Portability
Yes
No ethics review needed.
- Theoretically, the proposed method $\theta’ + \Delta \theta_i$ introduces error compared to $\theta’ + \Delta \theta_i’$, potentially leading to worse performance. However, in Table 2, a large proportion of $\theta’ + \Delta \theta_i$ results outperform $\theta’ + \Delta \theta_i’$ (the fine-tuned updated LLM). Could the authors elaborate on potential reasons contributing to this improvement?

- What does “# Trainable Parameters” signify in Table 6? Why does the fine-tuning model have zero trainable parameters, while the proposed training-free approach has a large number of trainable parameters?

- In line with the discussion in Weaknesses, could the authors investigate the impact of the approximation error introduced by the proposed method? For instance, providing metrics similar to those in Table 1 for all datasets, along with the corresponding performance gains, could offer a clearer understanding of how this factor contributes to overall performance.
The proposed method is based on the observation that residual parameter updates are negligible compared to the naive update, which is both intriguing and empirically proven effective.
This paper proposes a training-free personalization approach for evolving LLMs. Specifically, the proposed PortLLM utilizes LoRA parameters from finetuning older models to approximate updated models. Extensive experiments validate the effectiveness of the proposed approach.
The paper’s core claim is that the residual matrix $\Delta \theta_i’ - \Delta \theta_i$ is negligible in value compared to the naive update $\theta’ + \Delta \theta_i$. Although the authors provide both theoretical and empirical justifications for this, the Lemma (lines 266-269) only establishes that $\text{rank}(\Delta \theta_i’ - \Delta \theta_i) \ll \text{rank}(\theta’ + \Delta \theta_i)$, which does not directly imply that the former is negligible in value relative to the latter. While this statement is supported by empirical results in Table 1, a theoretical proof or an estimated error bound would strengthen the justification, as this could represent a significant proportion of the error for this method.
Yes
No ethics review needed.
1.	In Table 2, why does your method show such a large improvement on the WNLI task compared to the Fine-tuned LLM and Fine-tuned Updated LLM?

2.	Line 344: There is a notation error for the fine-tuned model, it should be $\theta_i$.

3.	Are the two columns of values in Table 6 reversed?
1. The framework proposes a training-free alternative to recurrent fine-tuning, which could be useful for applications with limited resources.
2. The portability across model versions and architectures has potential, especially in scenarios where repeated fine-tuning is computationally prohibitive.
This paper introduces PORTLLM, a framework designed to maintain the effectiveness of domain-specific adaptations in Large Language Models (LLMs) across updates without requiring further fine-tuning. The framework leverages task-specific model patches, derived as the difference between a fine-tuned model and its base (pre-trained) version, to transfer knowledge from one version of an LLM to the next. Experiments demonstrate the method’s effectiveness in enhancing the performance of updated LLMs on various tasks. The approach is claimed to offer computational efficiency and portability across model architectures.
1. The core idea of leveraging parameter differences between fine-tuned and pre-trained models (task vectors) has been explored in prior work, such as Ilharco et al. (2022) on task arithmetic. PORTLLM builds on this concept, but without substantial theoretical or empirical advancements, the framework’s contribution may be seen as incremental rather than groundbreaking. In light of the established work on task vectors, PORTLLM’s innovation could be questioned.

Ilharco, Gabriel, et al. “Editing models with task arithmetic.” arXiv preprint arXiv:2212.04089 (2022).

2. From the results in Tables 2 and 4, it is evident that the PortLLM method is effective in enhancing the performance of the Updated LLM. However, in most cases, it does not surpass the performance of the Fine-tuned LLM. Moreover, since the PortLLM approach requires access to the parameters of the Fine-tuned LLM to calculate $\Delta \theta$ (task vector), applying PortLLM depends on the existence of a Fine-tuned model. In this setup, if PortLLM cannot consistently outperform the Fine-tuned LLM in terms of performance, its practical application value becomes limited.
Yes
No
No ethics review needed.
Hope the authors can expand the experiments based on the identified weaknesses and revise the writing and theoretical analysis accordingly.
1. The training-free framework proposes the reuse of existing adapters after updating large language models and validates its feasibility, effectively reducing training costs.

2. Extensive experiments across multiple datasets and architectures provide strong evidence for the method’s effectiveness. The paper encompasses a broad range of tasks and models, demonstrating its generalizability and robustness across diverse scenarios.
This paper hypothesizes that the rank of adapters trained with LoRA for a specific downstream task is significantly lower than that of adapters obtained through continued pre-training of large language models using LoRA. The authors experimentally validate this hypothesis and further demonstrate that for pre-trained large language models, using the original adapter fine-tuned for a specific downstream task with LoRA does not lead to significant performance degradation after continued pre-training; in fact, there may even be an improvement in performance on downstream tasks. The performance is comparable to that achieved by training new adapters on the updated large language model. This conclusion suggests that after continued pre-training, the original adapters can still be effectively utilized, significantly reducing training costs.
1. The background and related work sections contain excessive irrelevant content.

2. In the experimental setup, the fine-tuning for downstream tasks is limited to a rank of 8 without further experimentation.

3. In the experimental setup, the continued pre-training involves only the use of different pre-training datasets for training, without addressing the scenario claimed by the authors: that large language models are updated periodically. In such updates, there may be conflicts between new knowledge that develops over time and existing knowledge, potentially leading to significant changes in the model. The authors did not consider this situation.

4. In the theoretical analysis section, it is better to check the statement that $ \( \theta' = \theta + \Delta\theta \)$ ensures $ \( \text{rank}(\theta') \geq \text{rank}(\theta) \) $.
Yes
No ethics review needed.
See above.
- A training-free framework for transferring domain-specific knowledge between evolving LLM versions through portable model patches
Theoretical analysis justifying why model patches work effectively, showing certain terms can be neglected to create simplified, training-free patches
- Comprehensive empirical validation across multiple tasks ranging from QA to complex reasoning and different model architectures
- Significant efficiency gains compared to LoRA fine-tuning
This paper presents PORTLLM, a novel training-free framework that enables the transfer of personalized knowledge between different versions of evolving LLMs. The key innovation is the creation of lightweight model patches that can be seamlessly applied to updated LLMs without requiring fine-tuning. The framework is extensively evaluated across seven tasks and multiple model architectures, demonstrating comparable or superior performance to LoRA fine-tuning while significantly reducing computational costs. The authors also provide theoretical justification for the portability of their model patches.
I don't identify major weakness, but I do have a few concern regarding the experimental setting:
- In the experiments, seems the authors only focus on small-scale open-source LLMs. However, from my understanding, it could also be more beneficial for large-scale commercial LLMs. 
- Following the previous setting, a few related works adaptation related works might be considered as related works besides LoRA, LMaaS (Sun et al., 2022), kNN-Adapter (Huang et al., 2023), CombLM (Ormazabal et al., 2023), IPA (Lu et al., 2023), Proxy-Tuning (Liu et al., 2024), and BBox-Adaptor (Sun et al., 2024).
- It is interesting to observe that in some cases it even performs better than fine-tuned models. More explanations on potential reasons could benefit this work.
- Echo my previous questions on experimental settings, I think a better evaluation benchmarks lie in (1) personalization tasks, like LaMP; (2) domain-specific adaptation tasks.
- Any thoughts on rigorous theoretical bounds on performance guarantees?
@inproceedings{
anonymous2024portllm,
title={Port{LLM}: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patches},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=gyHoR6uFhU},
note={under review}
}
As large language models (LLMs) increasingly shape the AI landscape, fine-tuning pretrained models has become more popular than in the pre-LLM erafor achieving optimal performance in domain-specific tasks. However, pretrained LLMs such as ChatGPT are periodically evolved (i.e., model parameters are frequently updated), making it challenging for downstream users with limited resources to keep up with fine-tuning the newest LLMs for their domain application. Even though fine-tuning costs have nowadays been reduced thanks to the innovations of parameter-efficient fine-tuning such as LoRA, not all downstream users have adequate computing for frequent personalization. Moreover, access to fine-tuning datasets, particularly in sensitive domains such as healthcare, could be time-restrictive, making it crucial to retain the knowledge encoded in earlier fine-tuned rounds for future adaptation. In this paper, we present PORTLLM, a training-free framework that (i) creates an initial lightweight model update patch to capture domain-specific knowledge, and (ii) allows a subsequent seamless plugging for the continual personalization of evolved LLM at minimal cost. Our extensive experiments cover seven representative datasets, from easier question-answering tasks {BoolQ, SST2} to harder reasoning tasks {WinoGrande, GSM8K}, and models including {Mistral-7B,Llama2, Llama3.1, and Gemma2}, validating the portability of our designed model patches and showcasing the effectiveness of our proposed framework. For instance, PORTLLM achieves comparable performance to LoRA fine-tuning with reductions of up to 12.2× in GPU memory usage. Finally, we provide theoretical justifications to understand the portability of our model update patches, which offers new insights into the theoretical dimension of LLMs’ personalization.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Large Language Models NLP Efficiency Fine-tuning Efficient Fine-tuning Portability
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/4a586df13477309b401d3028b4dc9c6adbd430d8.pdf
other topics in machine learning (i.e., none of the above)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/bd32be74ae1dff540e2f82b2a6d604fb11be3ec8.zip
PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patches
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission