Soft Prompts Go Hard: Steering Visual Language Models with Hidden Meta-Instructions
security;machine learning;adversarial perturbations;large language models
Yes
No ethics review needed.
See weakness
1. Figures clearly illustrate the point of the paper. 
2. The writing is easy to follow
3. Articulate the attack model and assumptions
4. Run transferability test
The paper introduced an attack that enables adversaries to add stealthy “meta-instructions” to images that influence how visual language models respond to queries about these images
1. L33, " but injection attacks in non-text modalities are a new, yet-to-be-explored area of LLM safety research". This type of attack has been widely explore in [1] and [2]
2. L81, "users are victims of adversarial third-party content that they ask the model to process". I'm curious whether the images are generated by the users or not. If the user create the line chart shown in Fig. 1 from their local devices, does it mean the attack studied in the paper doesn't exist anymore?
3. Table 4, why is the transfer rate of llava on negative as low as 0.1?
4. I'm curious what will happen if the system prompt of the VLM contradicts with the meta-instruction in the image?
5. Overall, I think the paper is in a good quality. The major downside is the novelty, as we already know from previous work that optimizing the input image towards a certain attack target is feasible for VLM. Thus, it's not a new vulnerability in VLM. Though the author attempts to differentiate their attack setting from previous jailbreaking and soft prompt attacks, the overall attack surfaces and methods remain largely the same. I would like to the see more insights coming from the paper. 


[1] Are aligned neural networks adversarially aligned?
[2] A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends
Yes
No ethics review needed.
I summarize some of my comments on weaknesses of the paper into questions below:

1) Do the authors agree with my comments about their portrayal of previous works, and if so what steps are the authors taking to address this? Concretely, what sections of the paper have been rewritten.
2) Have the authors been able to run the suggested experiments I have mentioned above, and if so what did they find?
## Originality

The question of prompt injection vulnerabilities  to large language models is of significant importance.  The authors demonstrate that models are vulnerable to  similar attacks of this nature through their vision input as are  possible through their text input. What's more, they  show the vulnerability is in some cases worse through  the image input.

Whilst the idea of providing meta-instructions through  image inputs its not entirely novel (see weaknesses section), this paper is the most thorough treatment of the subject that I am aware of, and brings to light new and concerning  ways that a model's output can be tampered with using  images.

## Quality and clarity

The paper is well written the method is conveyed clearly. The results section contains a good depth of experiments, most importantly covering a number of popular  open-source VLMs and target meta-instructions.

## Significance

As VLMs are used more frequently for agentic tasks  that will expose them to untrusted data from the internet, prompt injection / meta-instruction attacks will  become more and more concerning. Thus the paper  concerns a timely and interesting threat model  that the adversarial attack community should be  exploring in more detail.
The paper introduces a method to create image inputs to Vision Language Models (VLMs) that lead said model  to respond to any user query appended to the image with a certain "spin", e.g. responding with a certain sentiment,  or in a certain language. The authors refer to this  as embedding a "meta-instruction" in an image. 

Critically, a meta-instruction attack is only successful  if the models response to the users query (and the  attacked image) responds to the query whilst following  the meta-instruction (e.g., if the meta-instruction was  "talk in French" and the model responded in French but  did not answer the users query, then this would not  be a successful attack).

To train these meta-instruction attacks, the authors  perform projected gradient descent on an image  to minimize the language modeling loss of the VLM inputted with this image over a dataset of  synthetic question answer pairs with the answers  following some target natural language meta-instruction.

The results of the paper demonstrate that this  method can be used to learn adversarial images  for various different types of meta-instructions. The authors also demonstrate a non-trivial  transfer of meta-instruction images between models.
While the critique is long, this is only because I believe the paper has interesting results that could be improved.

## Presentation of previous work

The authors make a number of claims about prior work that I believe are not completely accurate. Editing the language around these claims would help to improve the paper. Here are some examples that I believe need to be addressed:

- Line 32 - "But injection attacks in non-text modalities are a new, yet-to-be-explored area of LLM safety research." I do not think this is entirely true. For example, Bailey et al. [1] explore how train an image to convey a certain text prompt, which they demonstrate can be a prompt injection attack. 
- Line 83 - "By design, jailbreaking and adversarial examples produce contextually incoherent outputs that do not actually answer users’ questions about images." I think this depends on how you define an image jailbreak. For example, Dong et al. [2] produce adversarially perturbations to harmful images that lead to a model answering coherently about said image --- in particular the model is able to correctly identify what is in the image. While the authors claim here is correct for other image jailbreaking work, such as Qi et al. [3] who learn images unrelated to the harmful request they are trying to elicit a response about from the model, it is not universally correct. For this reason the claim should be softened.
- Line 84 - "They [jailbreaking and image adv attacks] are not stealthy and cannot be used for indirect attacks because users would notice that the VLM’s outputs are wrong given the conversation context and inputs." Bailey et al. [1] and Qi et al. [3] both demonstrate methods to create jailbreaking images under epsilon ball constraints, which is the definition of stealthiness the authors use on line 290. 

## Novelty / originality

Following on from some of the comments above,   I believe there is a question of novelty / originality  of this work.  

In particular, the general algorithm presented to  produce meta-instruction attacks essentially involves  creating a dataset of input output pairs, and training  an image by PGD to maximize the likelihood over this  dataset. This method appears to fit into the "Behavior Matching" algorithm from Bailey et al. [1] 

Despite this, I believe the work does contain novel  and important contributions. In particular: 
1. The study of the changes in semantic meaning present in images from various different attacks, with meta-instruction attacks preserving meaning.
2. The transfer experiments in Table 4 are very interesting.
3. This is the most thorough treatment of prompt injection image attacks I have seen.

## Summary

Combining the above two points, I believe the paper needs to be
rewritten to more clearly lay out the novelty of the paper 
and more accurately represent the papers contribution. 
My high level suggestions would be:
1. Make it clear that prior works have examined prompt injecting image attacks, however yours is a more complete treatment of the topic.
2. Make it clear that your method to create such attacks is a special instance of what prior works have introduced. 
3. From this, your novelty comes not from the method but rather the results. E.g. line 88 that reads "We design, implement, and evaluate a method for creating a new type of image perturbations that act as cross-modal soft prompts for a language model while preserving the visual semantics of the image." needs to be adjusted.
4. Given that I do not think the method is novel, I would suggest running the following additional experiments:
	1. In Table 4, add transfer results to Claude and GPT-4o. These results should feature in the transferability experiment.
	2. More detailed defense experiments. Appendix C shows fairly simple defenses can work to avoid meta-instruction attacks. [1] finds that training perturbations under different constraints (e.g. a moving patch) ends up being more robust to simple defenses. It would be interesting to see if this result is reproducible in your setting.

To reiterate, I think studying prompt-injection images to models is important, and the authors present valuable results. I thank the authors for their hard work! 


[1] - Bailey, Luke, et al. "Image hijacks: Adversarial images can control generative models at runtime." arXiv preprint arXiv:2309.00236 (2023).

[2] - Dong, Yinpeng, et al. "How Robust is Google's Bard to Adversarial Image Attacks?." arXiv preprint arXiv:2309.11751 (2023).

[3] - Qi, Xiangyu, et al. "Visual adversarial examples jailbreak aligned large language models." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 38. No. 19. 2024.
Yes
No ethics review needed.
Please see weaknesses.
1. The concept of embedding hidden meta-instructions within images offers a new approach to prompt injection for multi-modal models, highlighting a potential vulnerability not extensively covered in existing literature.

2. It is interesting to see how the method reveals hidden capabilities of instruction-tuned models. In some cases, the meta-instructions successfully steer the model's outputs in ways that explicit instructions fail to achieve.

3. The study provides an empirical evaluation on a range of meta-objectives (e.g., sentiment, language, and political bias), demonstrating the effectiveness of the attack method.
The paper introduces a new type of attack on visual language models. These attacks, termed meta-instruction attacks, involve subtle image perturbations that act as soft prompts to influence how a model interprets images and responds to queries. The idea is to steer the model’s outputs to satisfy adversary-chosen objectives, such as a specific sentiment, style, or political bias, without the user being aware of the manipulation. The authors demonstrate the effectiveness of this approach across various visual language models, showing that these perturbations often outperform explicit instructions and are transferable across models.
1. The paper's reliance on just five images from a single dataset, ImageNet, limits the robustness and generalizability of its evaluation. ImageNet, which is primarily focused on object recognition, may not adequately represent the diversity and complexity of images encountered in real-world scenarios. Incorporating evaluations on datasets with more varied and complex scenes, such as MSCOCO, would provide a more comprehensive assessment of performance.

2. The paper simulates user interaction by generating questions to test meta-instructions, but it provides limited clarity on whether these questions adequately cover a broad range of natural user queries. Limited prompt diversity may affect the robustness of the attack if VLMs encounter different prompts in real-world scenarios.

3. Since the meta-instruction is added as noise to the image, the paper does not demonstrate the effectiveness of meta-instructions against recent inference-time defense methods like DISCO[1], DiffPure[2], and IRAD[3]. This could be valuable for understanding its performance in the context of contemporary robustness strategies.

[1] DISCO: Adversarial Defense with Local Implicit Functions.
[2] Diffusion models for adversarial purification.
[3] IRAD: Implicit Representation-driven Image Resampling against Adversarial Attacks.
Yes
No ethics review needed.
1. Could you also provide an evaluation when random resizing or cropping is applied? Since this paper addresses practical concerns, it would be valuable to test your method under common “defenses” encountered in everyday scenarios.

2. Are there any failure cases? For example, are there meta-instructions that are particularly difficult to achieve?

3. Why is it necessary to evaluate cosine similarity as done in Section 5.3? Could you clarify the relevance of this metric?

4. Is there an evaluation that checks whether the generated textual outputs remain consistent with the input images?

Overall, I appreciate the practical focus of this paper. I would be happy to raise my evaluation if these concerns are addressed.
1. The focus on the dissemination of seemingly coherent misinformation is highly practical and addresses a significant real-world concern.

2. The evaluation is thorough, including robustness testing against JPEG compression as a defense (which I suggest moving to the main text, given its practicality in everyday use) and examining the transferability of the attack across different vision-language models (VLMs).
This paper proposes a new attack objective in which the output text remains consistent with the input images but adopts an adversary-chosen style, sentiment, or point of view. The adversarial optimization is applied to the input image, ensuring that the modifications are imperceptible to humans. Experiments demonstrate that images containing hidden meta-instructions achieve significantly higher success rates compared to those with explicit instructions. This attack highlights a practical risk, as it enables the dissemination of seemingly coherent but misleading information.
1. A NeurIPS 2024 paper [1] also explores the dissemination of seemingly coherent misinformation in visual language models, but through the lens of data poisoning. While this paper focuses on test-time adversarial attacks, it would be beneficial to discuss the key differences between test-time attacks and training-time poisoning, and in what scenarios each is more practical, given the similarity in objectives between the two papers.

2. The evaluation of image semantics preservation seems suboptimal. In Section 5.3, semantics are defined using cosine similarity between images, but it is unclear why this metric is particularly relevant. A more meaningful evaluation would assess how well the actual text output of the visual language model aligns with the input images, which is the core focus of this paper—consistent outputs with images but in adversary-chosen styles, sentiments, or viewpoints.


Reference:
[1] Xu, Yuancheng, et al. "Shadowcast: Stealthy data poisoning attacks against vision-language models.", The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024
We introduce a new type of indirect, cross-modal injection attacks against VLMs to influence how the model interprets the image and steer its outputs to express an adversary-chosen style, sentiment, or point of view.
@inproceedings{
anonymous2024soft,
title={Soft Prompts Go Hard: Steering Visual Language Models with Hidden Meta-Instructions},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=1XxNbecjXe},
note={under review}
}
We introduce a new type of indirect, cross-modal injection attacks against language models that operate on images: hidden "meta-instructions" that influence how the model interprets the image and steer its outputs to express an adversary-chosen style, sentiment, or point of view. We create meta-instructions by generating images that act as soft prompts.  In contrast to jailbreaking attacks and adversarial examples, outputs produced in response to these images are plausible and based on the visual content of the image, yet also satisfy the adversary's (meta-)objective. We evaluate the efficacy of meta-instructions for multiple models and adversarial meta-objectives, and demonstrate how they "unlock" capabilities of the underlying language models that are unavailable via explicit text instructions. We describe how meta-instruction attacks could cause harm by enabling creation of self-interpreting content that carries spam, misinformation, and spin.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
security machine learning adversarial perturbations large language models
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/fb4adaa1ef6d33d20980028e22c625f0e811905a.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Soft Prompts Go Hard: Steering Visual Language Models with Hidden Meta-Instructions
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission