Soft-TransFormers for Continual Learning
Soft-Transformers;Continual Learning;Well-initialized Lottery Ticket Hypothesis (WLTH)
Yes
No ethics review needed.
Please refer to my above weaknesses.
1. The presentation sounds well.

2. The final experimental results present competitive even SoTA on some evaluated tasks.

3. Implementing the method on the Prompt Pool, which can divide the prompt pool into several groups and enable different group prompts to acquire different knowledge sounds reasonable while applying soft-network or binary masks on pre-trained weights can further boost the training.
In this paper, the authors attempt to solve the continual learning problem via a soft transformer idea, which adaptively selects subnetworks (specified by learnable masking for pre-trained parameters) and soft network tasked for detailed tasks in CL. During the sequential learning, the model will be trained to achieve a task-specific soft network based on the task-agnostic pre-trained model and sparsely activate some weights in the layers or deactivate someone. Besides, the authors support some theory proofs based on Well-initialized Lottery Ticket Hypothesis to analyze the rationale behind the improvements. Implemented on baselines, like DualPrompt, L2P, and L2p-PGP, they obtain competitive even SoTA performances.
1. Somehow, I believe such soft-network or subnetwork ideas have been widely studied among the communities, like model compression, model purification, or PEFT. One can think of employing the extra prompt vectors or using a learnable masking strategy, forcing the model to fit the downstream task while the pre-trained weights are fixed to hold the generalizability. Moreover, the prompt pool construction mechanism has been widely studied by DualPrompt, L2P.

2. I am curious about the training efficiency and inference latency since they utilize a large subnetwork or softnework compared with the baseline. 

3. Missing the tunable parameter comparisons with baselines and existing methods.
Yes
No ethics review needed.
Please refer to the weakness.
The overall presentation is clear.
The experimental results demonstrate the effectiveness of the proposed method.
This work proposed sub-network over the prompt tuning technique for continuous learning. 
The overall presentation is clear but the motivation and the novelty is unclear.
Experimental results demonstrate superior performance to other methods.
Firstly, the motivation is unclear.  In lines 84-85, the paper stated that " It cannot capture all the nuances of uncorrelated sequential tasks wildly if the task significantly differs from what the CL model was pre-trained initially on". It seems this work aims to address the reliance on a well pretraining, but the proposed method also require a well-initialized network as stated in lines 96-99. 
Besides, in lines 224-225, the task-specific prompts can be regarded as the explicit task-specific fine-tuning.

Secondly, the proposed method seems to be a combination of parameter-efficient fine-tuning method, e.g., prompt + LoRA, prompt + Adapter. How much performance improvement would be caused by adding LoRA or adapters for specific tasks in the baseline method。
The superior performance seems contributed by more learnable parameters.

The experimental results requires more analysis. 
(i) Explain the ``Upper bound''. In fact, since the proposed method modifies the model architecture, the ``Upper bound''  cannot be the same as the other paper.
(ii) Why the proposed method achieves higher performance than the ``Upper bound''? 
(iii) Why the proposed method achieves higher performance when using DualPrompt as baseline while the improvement over the L2P seems smaller. 
(iv). In table 1, why the performance under the  20-Split-CIFAR100 is higher than  10-Split-CIFAR100? In general, the more incremental learning step, the weaker the performance due to forgetting. I suspect that the improved performance is due to learning more parameters. The number of parameters should be reported.
Yes
No ethics review needed.
I recommend the authors address these concerns by providing detailed descriptions of the soft mask initialization, selection, and optimization processes, as well as methods for managing storage and computational costs with an increasing number of tasks. Clarifying the underlying mechanisms of mask composition and task differentiation would greatly enhance the transparency and reproducibility of the method.
1. The paper includes extensive experiments across various datasets and tasks, providing comprehensive evidence for the method’s effectiveness.
2. The paper is well-written, with clear organization and logical flow that effectively communicates complex ideas.
This paper presents an approach that combines Prompt techniques with learnable soft masks to address catastrophic forgetting in continual learning.  Inspired by Well-initialized Lottery Ticket Hypothesis (WLTH), which provides suboptimal fine-tuning solutions, this paper proposes a fully fine-tuned continual learning (CL) method referred to as Soft-TransFormers (Soft-TF).
### 1. Insufficient Technical Explanation

**1.1 Lack of Detailed Specification on Soft Mask \( m \) Initialization, Selection, and Optimization**  
The paper suggests that the soft mask \( m \) is task-specific, adapting the subnetwork structure based on each task's needs. However, it lacks essential details on how these masks are initialized, selected, and optimized for each task. For instance, are there specific initialization strategies, criteria for mask selection, or task-transition strategies? Such missing details make it challenging to understand the core contribution of the paper, especially as the optimization of these masks is crucial for retaining knowledge in continual learning.

**1.2 Potential Growth in Parameters Due to Task-Specific Masks**  
As each task requires an independent soft mask \( m \), the storage cost may grow exponentially with the number of tasks. This is particularly concerning in scenarios with a large number of continual learning tasks, where storage costs could become prohibitive. The paper should discuss how to mitigate this issue.

**1.3 Lack of Clarity on the Setting of \( \alpha_t \)**  
The parameter \( \alpha_t \) is referenced but not clearly explained in terms of how it is set or updated for each task. How does the method ensure that the sum of \( \alpha_t \) across tasks equals 1, and what role does this parameter play in mask composition? Detailed explanations and possible constraints for \( \alpha_t \) are necessary for comprehending how the method balances task adaptation and parameter sharing.

**1.4 Redundant Calculation of \( t_x \) in Algorithm 2**  
Algorithm 2 calculates \( t_x \) twice, raising the question of whether the indices for the E-Prompt and the learned subnetwork are identical. This redundancy lacks justification, and without clear reasoning, readers may question the necessity of these duplicate operations. A clarification is needed regarding the purpose of this repeated calculation and the roles of E-Prompt and learned subnetworks in relation to \( t_x \).

**1.5 Increased Inference Cost Due to Subnetwork Selection**  
According to Algorithm 2, selecting the learned subnetwork for a specific task requires backpropagation for subnetwork selection, which can significantly increase inference costs. This may limit the method's applicability in real-time or resource-constrained environments. 

### 2. Unclear Core Mechanism

**2.1 Ambiguity in the Combination Mechanism of Soft Mask \( m \)**  
The paper does not clarify why and how soft masks \( m \) can be combined based on different tasks. Ensuring each task's mask is unique and effectively isolated from other tasks is vital to avoid task interference and catastrophic forgetting. The paper lacks an explanation of how such task differentiation is achieved in practice. This is a crucial detail for understanding the validity of the proposed method's mechanism.

**2.2 Ensuring Task-Specific Mask Differentiation**  
The authors do not provide details on how task-specific masks are designed to ensure they differ across tasks. If there is overlap or similarity between masks for different tasks, there is a high risk of interference, which could exacerbate forgetting. Further clarification on how each task’s mask maintains uniqueness would strengthen the proposed method.
Fully fine-tuned continual learning (CL) Model, Soft-TransFormers (Soft-TF) inspired by Well-initialized Lottery Ticket Hypothesis (WLTH)
@inproceedings{
anonymous2024softtransformers,
title={Soft-TransFormers for Continual Learning},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=o9icFTlKi4},
note={under review}
}
Inspired by Well-initialized Lottery Ticket Hypothesis (WLTH), which provides suboptimal fine-tuning solutions, we propose a novel fully fine-tuned continual learning (CL) method referred to as Soft-TransFormers (Soft-TF). Soft-TF sequentially learns and selects an optimal soft-network or subnetwork for each task. During sequential training in CL, Soft-TF jointly optimizes the weights of sparse layers to obtain task-adaptive soft (real-valued) networks or subnetworks (binary masks), while keeping the well-pre-trained layer parameters frozen. In inference, the identified task-adaptive network of Soft-TF masks the parameters of the pre-trained network, mapping to an optimal solution for each task and minimizing Catastrophic Forgetting (CF) - the soft-masking preserves the knowledge of the pre-trained network. Extensive experiments on Vision Transformer (ViT) and CLIP demonstrate the effectiveness of Soft-TF, achieving state-of-the-art performance across various CL scenarios, including Class-Incremental Learning (CIL) and Task-Incremental Learning (TIL), supported by convergence theory.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Soft-Transformers Continual Learning Well-initialized Lottery Ticket Hypothesis (WLTH)
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/ce336bdb3b3f22b08e7ed0a04f12ddecc24bc921.pdf
transfer learning, meta learning, and lifelong learning
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Soft-TransFormers for Continual Learning
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission