Archilles' Heel in Semi-open LLMs: Hiding Bottom against Recovery Attacks
Semi-open Model;Closed-sourcing Approach
Yes
No ethics review needed.
Please refer to the weaknesses.
1. The paper introduces SCARA, a method that selectively closes only the bottom layers of semi-open large language models (LLMs) to enhance customizability while maintaining resilience against recovery attacks.
2. It provides a theoretical analysis of the existence of a transition layer in transformer-based models.
The paper introduces SCARA, a selective closed-sourcing approach for designing semi-open large language models (LLMs) that enhance customizability while maintaining resilience against recovery attacks. The authors develop an algorithm that strategically keeps only a few bottom layers closed-source, ensuring model flexibility without compromising security. They theoretically demonstrate a "transition layer" within deep transformer models, showing that recovery errors in layers before this point lead to recovery failure, while errors in later layers have a limited impact. SCARA estimates the optimal number of layers to hide using a novel metric based on initial recovery loss, bypassing the need for fine-tuning. The method is applied to five models ranging from 1.3B to 70B parameters, tested across six downstream tasks and sixteen recovery benchmarks. Results show that SCARA improves downstream performance while requiring over ten times fewer closed-source parameters than baselines, achieving improvements, especially in domain-specific tasks like Financial, with 30% higher performance on Llama2-70B. SCARA maintains comparable resilience against recovery attacks.
1. **Unclear Motivation for Semi-Open Models:** The market is dominated by closed-source models and fully open-source models. If customization needs are already addressed by existing fine-tuning services provided for closed-source models (e.g., API-based fine-tuning on closed models like GPT-4), it would be insightful to understand the specific motivations and advantages driving the development of a semi-open architecture. 
2. **The threat model is not clear.** The threat model concerning recovery attacks on semi-open LLMs is insufficiently defined. The paper does not clearly specify the adversary's capabilities, such as the extent of access to the model's architecture, parameters, or outputs. This lack of clarity makes it challenging to assess the effectiveness of the proposed SCARA method in mitigating such threats.
3. **Insufficient Details on SCARA's Implementation:** The description of SCARA's methodology is vague, particularly regarding the fine-tuning-free metric used to determine which layers to keep closed-source. The paper does not provide a clear explanation of how this metric is calculated, the data required, or the computational resources involved etc.
4. **Evaluation minors:** While the authors present experimental results across multiple models and tasks, the evaluation lacks depth. The paper does not offer a comprehensive analysis of SCARA's performance compared to existing methods, nor does it explore potential trade-offs between customizability and security.
Yes
No ethics review needed.
n/a
The evaluation seems comprehensive. It was easy to follow the problem setup and the method.
The paper proposes a method for identifying which layers in the model that, if recovered by an adversary in an iterative layer recovery process, will make subsequent layers easier to recover.
I question the threat model that the authors are introducing. I don't think there's any chance of stealing an LLM through any method that exists no matter how semi-closed/semi-open it is. The only methods that have been proposed that can do something like this specifically target the embedding layer.

It seems like the main insight of the paper, that hiding the earlier layers in the model is more impactful than hiding later layers because if an attacker wants to recover the model they'll pay an accuracy error scaling in the depth of the model past the layer they haven't yet recovered, is trivial. If you asked someone who had never heard anything about this literature of hiding layers, whether they should hide the first block or the last block, I'm certain everyone would choose to hide the first block. There's plenty of work already showing that later layers are more or less redundant and don't learn anything new. This is because attention heads in block N have the ability to learn Nth order interactions, but for N > 2, these interactions typically don't get learned and the attention heads just degenerate [1].

The actual implementation of the method is not sophisticated. It just takes this straightforward insight and turns it into a metric. But that metric is itself just "what happened if I closed the first N layers of the model" and then returns the first one that passes some threshold of difficulty.

It doesn't seem like the evaluation is really fair. The authors evaluate against SEM. But SEM just wants to recover the embedding and the authors are trying to show what happens if they hide the early parts of the network. This seems like an indication that this isn't a particularly realistic threat model.

[1] https://arxiv.org/abs/2404.08634
Yes
No ethics review needed.
Can the authors explain and clarify why the semi-open models are practically relevant?
1. The paper is well-written, and the presentation is clear and clean. 

2. The approach is well motivated --- it first starts from an empirical observation that closed-sourcing the first two layers offers significantly greater resilience than the last two, while the model shares similar customizability under the two cases. This implies that close sourcing the later layers may be the optimal solution for keeping the resistance to recovery attacks. The paper subsequently further formally establishes this finding with rigorous theoretical analysis, showing the existence of a transition layer such that even small recovery errors in layers before this layer can lead to recovery failure. This also intuitively makes sense --- when the attacker is asked to recover the earlier layers as opposed to the later layers, the errors in early layers will be amplified by later layers. This asymmetry is natural. 

3. Based on this insight, the paper also proposes an effective approach for the selectively closed-sourcing model to defend against recovery attacks. The experiment results support the effectiveness of the approach. 

Overall, the paper is nicely done.
This paper studies the problem of how to design semi-open models (i.e., models whose weights are only partially open-sourced) that can simultaneously also be resilient to recovery attacks. The paper finds that a transition layer exists, such that even small recovery errors in layers before this layer can lead to recovery failure. Building on these insights, the paper proposes an approach called SCARA that keeps only a few bottom layers as closed-source. With this new approach, the paper shows that it is possible to improve downstream customization performance while maintaining similar resilience.
One outstanding weakness of this paper is that the threat model considered may not be practically relevant. It seems the authors coined the semi-open model's scenario, that seems not really exist in the real world. 

Currently, the most common setups are either open-source or closed-source. For close-source context, when developers do want their users to customize their models, the standard practice is to deploy fine-tuning APIs (e.g., https://platform.openai.com/docs/guides/fine-tuning) rather than partially open-source a part of the model. It seems to make no sense to only open-source the first few layers of a model to enable customization. Because the customization anyway still needs the involvement of the closed-source developers --- so they can fine-tune and connect the first few layers and the later layers to really deploy the model. Then, why not just close-source all weights and directly ask the users to upload custom data, and then the closed-source developers fine-tune and deploy the model for the users, like what is being done in fine-tuning APIs? 

I worry that if not developers will do the partial open-sourcing like the authors of this paper consider, then the problem itself may not hold.
Yes
No ethics review needed.
1. Is the "attack datasets" mentioned in Figure 2 the same as the "recovery datasets" discussed later in the paper?
2. Could you clarify the formula and the loss function used for RD(I)?
3. Could you clarify how fully-closed and semi-open models differ in practice?
4. Could you explain more about the distinctions between FT-all, FT-closed, and SEM in Section 5.1?
5. Can the row and column headers in the tables be made clearer by avoiding abbreviations?
6. Could you explain more about potential future work that could be included in the paper?
7. Could the authors clarify what the value 0.00 represents in Table 1 and Table 2?
8. The authors discussed the impact of datasets of different lengths on the effectiveness of SCARA in the experimental section, but these datasets did not appear in the setup. Could the authors provide a detailed introduction to the composition of these datasets?
1. The paper is well-written, featuring thorough experiments and clear explanations from both theoretical and empirical perspectives.
2. The overall layout is visually pleasing, and the figures are diverse and effectively illustrate the content, aiding readers' understanding.
3. It proposes a straightforward and effective method for constructing a semi-open model by hiding only a few layers, while achieving baseline-level resilience and customization performance comparable to the fully open setting.
4. The insight regarding the existence of a transition layer contributing to resilience is particularly compelling, with a detailed theoretical explanation that enhances understanding.
5. The authors provide comprehensive empirical validation across multiple architectures and benchmarks, covering models of various sizes (1.3B-70B), and testing customizability and recovery performance on several benchmarks. They also conducted experiments on recovery datasets of different sizes, demonstrating sufficient experimental rigor.
6. The authors proposed additional enhancements to the original baseline, strengthening the protection of baseline SAP and highlighting SCARA’s effectiveness in preserving resilience.
7. The authors empirically validated their theory of the transition layer’s existence and pointed out that smaller models exhibit transition layers earlier than larger models.
8. The authors clearly identified the limitations of the SCARA method, noting its ineffectiveness on small models (OPT-350M) and its inability to defend against other adversary attacks.
9. The proposed SCARA algorithm has clear practical applications, offering a viable solution for enhancing the customizability of semi-open models while preserving comparable resilience.
This paper presents SCARA, a method to identify decoder layers to hide in decoder-only LLMs. The authors provide theoretical proof of transition layers, where early errors are amplified in subsequent layers. They introduce RD to assess post-recovery performance when specific layers are hidden. Experiments show that SCARA, by hiding only a few layers, achieves a recovery ratio close to baselines while maintaining customization performance similar to fully open approach. The experiments also confirm the existence of transition layers in the models.
1. One mathematical notation in Section 4.2 is unclear. The loss function $\ell$ for RD(I) is not specified, making it confusing.
2. Figures 1 and 2 have minimal captions and small text, reducing readability and limiting their ability to convey insights.
In this paper, we propose SCARA, a fine-tuning-free approach that reduces the number of closed-source layers to enhance customizability while preserving resilience to recovery attacks in semi-open models.
@inproceedings{
anonymous2024archilles,
title={Archilles' Heel in Semi-open {LLM}s: Hiding Bottom against Recovery Attacks},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=1RC3KtP1jT},
note={under review}
}
Closed-source large language models deliver strong performance but have limited downstream customizability. Semi-open models, combining both closed-source and public layers, were introduced to improve customizability. However, parameters in the closed-source layers are found vulnerable to recovery attacks. In this paper, we explore the design of semi-open models with fewer closed-source layers, aiming to increase customizability while ensuring resilience to recovery attacks. We analyze the contribution of closed-source layer to the overall resilience and theoretically prove that in a deep transformer-based model, there exists a transition layer such that even small recovery errors in layers before this layer can lead to recovery failure. 
Building on this, we propose \textbf{SCARA}, a novel approach that keeps only a few bottom layer as closed-source. SCARA employs a fine-tuning-free metric to estimate the maximum number of layers that can be publicly accessible for customization. We apply it to five models (1.3B to 70B parameters) to construct semi-open models, validating their customizability on six downstream tasks and assessing their resilience against various recovery attacks on sixteen benchmarks. We compare SCARA to baselines and observe that it generally improves downstream customization performance and offers similar resilience with over \textbf{10} times fewer closed-source parameters. We empirically investigate the existence of transition layers, analyze the effectiveness of our scheme and finally discuss its limitations.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Semi-open Model Closed-sourcing Approach
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/99bb7d6b7317cda572f7e45ea1e714b48c0ac876.pdf
other topics in machine learning (i.e., none of the above)
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Archilles' Heel in Semi-open LLMs: Hiding Bottom against Recovery Attacks
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission