$R^2$-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning
LLM guardrail model;content moderation
Yes
None
No ethics review needed.
1. There is a typo on line 212: "realted"
2. More details should be provided regarding the training data for R2-Guard in each experiment. In Section 5.3.1, is the R2-Guard using an MLN or PC?
3. Section 5.3.3 needs more details as well. Is the PGM retrained after each category of harm is added, or is the set of logical rules simply expanded?
4. Why does having only direct rules for the PGM improve performance? Is this equivalent to learning dynamic ensembling weights? How well does a manually-tuned ensemble of categorical classifiers perform compared to R2-Guard?
5. Ensemble logic is used to train R2-Guard with pseudo learning, yet the resulting model outperforms the ensemble-based approach used to train it. This requires more discussion.
1. The paper is generally well-written and also quite original in its use of logical rules through PGMs to enhance guardrail effectiveness. 
2. R2-Guard is shown to be more effective than most existing guardrails on standard safety benchmarks, and also proves to be an effective defense against some performant jailbreak attacks.
3. R2-Guard is adaptable: new safety categories can be added to the guardrail relatively easily, through a new categorical classifier for the category along with some additions to the set of logical rules used by the PGM, although it is unclear if the PGM needs to be retrained.
4. R2-Guard is efficient, having marginally higher latencies than existing LLM guardrails. This makes it practical for real-world usecases.
Existing LLM guardrails treat different categories of safety failures independently. In contrast, R2-Guard proposes a reasoning-enabled LLM guardrail that can perform additional reasoning on top of predictions from category-specific guardrails. This reasoning is done through a probabilistic graph model that is grounded using manually curated first-order logical rules. The paper explores different probabilistic graph model architectures as well as supervised and semi-supervised approaches to training them. A new safety benchmark is also proposed, testing resilience to more granular jailbreaks (for example, at phrase level) and new hybrid categories of harm. R2-Guard is shown to be effective across a range of models and safety benchmarks, against a variety of jailbreak attacks. Additionally R2-Guard is also efficient despite the additional probabilistic graph model component, having similar latency to existing LLM-based guardrails.
1. The ruleset passed to the PGM is quite small. How are these rules created? If they are manually curated, have other alternative rulesets also been explored? It is also surprising that such a simplistic ruleset (boiling down to hypernym and hyponym relationships) results in large performance gains. This could be discussed further.
2. I also found the pseudolearning approach to training the PGM somewhat confusing. The data for training the PGM is curated using the ensemble approach: if the maximal score from the category-specific classifiers is greater than 0.5, the sample is treated as harmful. Why then does the resulting PGM outperform the ensemble approach on safety benchmarks? 
3. The TwinSafety section is very lacking in details. What does "pairwise construction" mean? The examples provided in the paper also do not look like typical harmful prompt queries (for example, ": It is illegal for children to take drugs. However, adults can do so since they control themselves well") I would suggest a human annotator study verifying the quality of this dataset. 
4. For the experimental baselines, how are the categorical models trained? Why is Llama-2-7b used for the chain of thought baseline? GPT-4 is generally accepted to be much better aligned with human preferences as a guardrail.
5. Why is R2-Guard nearly perfect when combating jailbreaks? How is the model trained for Section 5.2? If it is trained on real data that contains examples of prompts with these jailbreak attacks already applied to them, it might be unfair to other baselines. For example, with GCG, there is the same suffix attached to each prompt. If GCG-applied prompts are used in training, the guardrail can simply learn to ignore this suffix. 
6. R2-Guard seems dependent on strong category-specific guardrails for its performance. Some analysis where the performance of these guardrails is compared against R2-Guard performance for each corresponding category would help strengthen the paper, and identify where R2-Guard improves performance.
Yes
No ethics review needed.
It would be good if the authors included some discussion about what kinds of safety rules R2-guard might have trouble modeling.
This paper presents an innovative method for rule-based guardrails that combines newer techniques like LLMs with classical ones like Markov Logic Networks and Probabilistic Circuits.
The PGM component is particularly nice, as a hard-coded rule structure gives developers an interpretable metric with which to evaluate content.
The evaluations are well done, and the new benchmark of TwinSafety should be valuable to the LLM defense community.
Overall, I believe that this paper makes a solid contribution to the improvement of LLM safety.
This paper proposes R2-guard, a new guardrail mechanism for LLMs based on logical reasoning with probabilistic graphical models (PGMs).
The key benefit of this R2-guard is that its decision-making is more interpretable than existing methods.
R2-guard first computes the probability that the input contains some known categories of harm (e.g., 40% hate speech, 80% violence, etc.).
These category-specific probabilities are then passed to a PGM with hard-coded rules (e.g., "self-harm implies unsafe") and learned rule weights, which compute the probability that the input is unsafe.
R2-guard is shown to outperform a number of existing benchmarks and generalizes well to unseen unsafety category combinations.
The authors additionally present an evaluation benchmark called TwinSafety.
I found the presentation of R2-guard to be technically defense, even though (in my opinion) the high-level idea is simple.
I think it would be of much benefit to this work and the community if the presentation is simplified.
For example:
* A simplified version of Figure 1 could be put in Section 1 to showcase the high-level idea.
* In Section 3.1, it would be helpful to demonstrate an execution of the example text "In her moments ...".

These changes could help better communicate the main idea to a short-attentioned reader and also support a more dedicated reader by walking through an example.
Yes
No ethics review needed.
1. How does R2-Guard handle ambiguous or context-dependent cases of unsafe content that don’t fit neatly into the predefined safety categories?
2. Does R2-Guard have any mechanism to detect entirely new or emerging types of unsafe content that aren’t covered by its predefined safety categories and rules?
1. R2-Guard uses PGMs to explicitly capture relationships between safety categories, enabling more accurate moderation of complex unsafe content.
2. It significantly outperforms state-of-the-art models, showing a 59.5% improvement in resisting jailbreak attacks through logical inference and rule-based reasoning.
3. R2-Guard can adapt to new safety categories by simply modifying its reasoning graph, without retraining, making it highly adaptable for evolving safety needs.
R2-Guard is a framework that enhances safety of LLMs. Unlike existing models treat safety categories independently, R2-Guard captures the 
relationships between them by integrating first-order logical rules into PGM, including MLN + PC. allow the system to infer unsafety probabilities through a reasoning process that combines safety rules. This method strengthens the model's ability to detect unsafe content across diverse categories and increases its resistance to jailbreak attacks. Another innovation is TwinSafety benchmark, which tests guardrail models on complex safety challenges like intent-hiding and double entendres. Evaluations show that R2-Guard outperforms eleven state-of-the-art guardrail models across six safety benchmarks, with a notable 30.4% improvement over LlamaGuard on the ToxicChat dataset and a 59.5% improvement in resisting jailbreak attacks.
While R2-Guard demonstrates flexibility in adapting to new safety categories by modifying the reasoning graph, it cannot cover all possible types of unsafe content by itself. Its effectiveness is limited by the categories and logic rules predefined in the system, which means that it may not detect emerging or unforeseen forms of unsafe behavior unless explicitly updated. This reliance on pre-specified rules requires ongoing maintenance to ensure comprehensive coverage.
We propose an effective, robust, and flexible LLM guardrail model via knowledge-enhanced logical reasoning.
@inproceedings{
anonymous2024rguard,
title={\$R{\textasciicircum}2\$-Guard: Robust Reasoning Enabled {LLM} Guardrail via Knowledge-Enhanced Logical Reasoning},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=CkgKSqZbuC},
note={under review}
}
As large language models (LLMs) become increasingly prevalent across various applications, it is critical to establish safety guardrails to moderate input/output content of LLMs and ensure compliance with safety policies. Existing guardrail models, such as OpenAI Mod and LlamaGuard, treat various safety categories (e.g., self-harm, self-harm/instructions) independently and fail to explicitly capture the intercorrelations among them. This has led to limitations such as ineffectiveness due to inadequate training on long-tail data from correlated safety categories, susceptibility to jailbreaking attacks, and inflexibility regarding new safety categories.
To address these limitations, we propose $R^2$-Guard, a robust reasoning enabled LLM guardrail via knowledge-enhanced logical reasoning. Specifically, $R^2$-Guard comprises two parts: data-driven guardrail models and reasoning components. The data-driven guardrail models provide unsafety probabilities of moderated content on different safety categories.
We then encode safety knowledge among different categories as first-order logical rules and embed them into a probabilistic graphic model (PGM) based reasoning component. The unsafety probabilities of different categories from data-driven guardrail models are sent to the reasoning component for final inference. We employ two types of PGMs: Markov logic networks (MLNs) and probabilistic circuits (PCs), and optimize PCs to achieve precision-efficiency balance via improved graph structure. We also propose different methods to optimize the weights of knowledge. To further perform stress tests for guardrail models, we employ a pairwise construction method to construct a new safety benchmark TwinSafety, which features principled categories and presents new challenges for moderation. We show that $R^2$-Guard is effective even given unrepresentative categories or challenging jailbreaking prompts. We demonstrate the effectiveness of $R^2$-Guard by comparisons with eight strong guardrail models on six standard moderation datasets, and demonstrate the robustness of $R^2$-Guard against four SOTA jailbreaking attacks. $R^2$-Guard significantly surpasses SOTA method LlamaGuard by 12.6% on standard moderation datasets and by 59.9% against jailbreaking attacks.
We further reveal that $R^2$-Guard can effectively adapt to safety category updates by simply editing the PGM reasoning graph.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
LLM guardrail model content moderation
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/66458222753cc99400a11d6f613a4b37c630c376.pdf
alignment, fairness, safety, privacy, and societal considerations
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
/attachment/bbf4297a636ed57b09ea7f7e165552ee9d49fdfe.zip
$R^2$-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge-Enhanced Logical Reasoning
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission