SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models
Autonomous Driving; Multimodal Large Language Models; Multimodal Retrieval-Augmented Generation; Probabilistic Graph Model
Yes
No ethics review needed.
One of the main components of the proposed framework is a post-processor designed to identify predictions made by the large language model that do not adhere to the knowledge rules. It would be helpful to understand how often violations of the rules were observed in the conducted experiments.

How does the method deal with cases where the large language model keeps generating a new high-level action (after reprompting) that violates the knowledge rules?

As a suggestion (also mentioned earlier, along with detailed comments), the paper would need to be improved in terms of clarity and readability.
Good referencing of prior works related to most of the involved components, including recent studies.

Additionally, the paper clarifies the relation to prior works and explains how the proposed approach differs from the previous ones.

The work is clearly motivated in the paper, the research context and goals are also clearly stated.

The method is described in a way that is easy to follow.

The experimental setting is well-explained.
The authors propose SafeAuto, a framework that can be incorporated into multimodal large language models and that consists of 3 components: (i) a new loss function called Place-Dependent Cross-Entropy, (ii) a reasoning module based on Markov Logic Networks, (iii) a multimodal Retrieval-Augmented Generation model.
The Place-Dependent Cross-Entropy loss aims to better capture numerical proximity between numeric strings generated by the large language models indicating low-level control numerical signals (e.g., speed).
The Markov Logic Network-based module serves as a post-processor that flags the large language model's outputs which do not satisfy pre-defined knowledge rules (written in first order logic and capturing relations between predicates representing actions of the ego-vehicle---such as stop or accelerate---and objects from the environment---such as a stop sign).
The multimodal Retrieval-Augmented Generation model aims at helping the decision process by retrieving driving experiences similar to the current scenario, utilising information gathered from the image embeddings, control signals and a vector containing the values of binary predicates (that capture whether certain objects, e.g. a stop sign, are present in the given image or not).
The authors conduct an experimental analysis (on two datasets), showing (i) how these three components impact the performance when compared to state-of-the-art baselines and (ii) how background knowledge can be used to enhance the predictions of multimodal large language models.
There is no discussion on how the proposed Place-Dependent Cross-Entropy Loss behaves when there is a difference in the order of magnitude between the input and the target values. From the description of the method, it appears that the Place-Dependent Cross-Entropy Loss does not take into account the decimal point position (and, for example, this means that the target probability distributions from Figure 2 for numeric string "2.69" will be the same as for numeric string "26.9").
I would appreciate it if the authors could clarify this point and the statement in lines 247-248: "Notice that when \sigma is set to 0, the loss reduces to the original definition of joint CE loss for the entire numeric string", which again is unclear as the cross-entropy loss takes into account the decimal point position.

The method offers no guarantee that reprompting the multimodal large langue models will generate a new high-level action that does not violate the knowledge rules, since such models are data-driven.

The paper is in the context of knowledge-enhanced autonomous driving and argues that the underlying models are limited as they do not account for any available knowledge when making their decision.
Specifically, the authors capture the knowledge rules in the form of logical formulae, yet the related work section does not discuss prior works that would be relevant here (e.g., representing and integrating background knowledge into neural network models).

Improvements in clarity and readability are needed. Please see below detailed comments.

Figure 2: contains pseudocode and could be moved to supplementary material, with its contents described more concisely in natural language in the main text.

Formatting: cluttered text, very little space between captions and main text (e.g., lines 73 and 237), all affecting readability.
All abbreviations should be introduced, e.g. "QA" is used without doing so.

Many typos:
- in the abstract, "the the Place-Dependent".
- lines 92, 93: "contextssuch", "signalsto"
- lines 105, 106: "scenariowhich", "informationwe"
- line 144: "knowledgespecifically", "rulesinto"
- and other such instances of words not being separated by a space when they should be.
- a space before each citation would improve the readability (line 108 "BDD-X(Kim et al., 2018) and DriveLM(Sima et al., 2023)")
- abbreviations are introduced multiple times: e.g., "cross-entropy (CE)" (lines 53, 127, 180); "Place-Dependent Cross-Entropy (PDCE)" (lines 95, 170, 397), similarly for "Markov Logic Networks", etc.
- line 277/278: "all formula"
- the title of section 6 is inconsistent with the naming convention of the rest (i.e., "Limitation.", rather than "Limitation").

Other comments:
- in Figure 2, the naming of "cumulative_probs" (line 231) could be confusing/misleading as it suggests traditional cumulative probabilities. A clearer term, such as "weights," would align with the main text (line 238).
- repetitions: e.g., lines 83-84: "regarding high-level action prediction, a significant limitation of current methods in high-level action prediction"
- line 193: "with temperature as 1.0." it is unclear what is meant here, as the temperature was not yet introduced (only later on line 374 it is introduced).
Yes
No ethics review needed.
1. Formulate the loss function in mathematical terms
2. Perform experiments on loss behavior with error bars and more varied situations
3. Provide runtime information
4. Discuss inference strategy
5. Produce more experiments with driving performance like NuScenes setup and UniAD / VAD.
This paper proposes to use a lot of new methods for enhancing autonomous driving. It addresses multiple problems that are difficult to solve and poses a challenge for using large language models for autonomous driving related tasks. Understanding of numerical information from text is a common problem that this paper proposes to solve using a new loss function. Reasoning is also a challenging task for these systems, here the paper proposes a system of using safety regulations to logic within a graphical model. The RAG model is also meant to produce better driving experience by learning from past.
This paper proposes a framework to enhance multimodal large language based autonomous driving using structured and unstructured knowledge. The main contribution of the paper is proposal of a new loss function that the authors call place dependent cross entropy loss, that is supposed to enhance accuracy of text based control instructions. They also propose building a reasoning component for converting safety regulations into first order logic rules. These rules are then used in a probabilistic graphical model along with environmental features to contruct a RAG model.
This paper proposes a few different changes and the goal of the proposals are not always clear. If the goal is to produce a better driver, there should be comparison with other autonomous driving works in standard NuScenes like evaluation, both open and closed loop. But, driving comparison with standard models like UniAD and VAD is not performed (Table 3).

The loss PDCE formulation is unclear. Any new loss function should be defined clearly with notation. Here the description of the loss is unclear. The properties of the loss needs to be studied in a more varied settings. The authors just produce a graph for a specific setting to motivate this loss, but better experiments need to be planned to demonstrate the behavior of this loss.

The action generation step from the LLM is not grounded in any way, so it is not clear how the approach would deal with invalid responses or hallucination, where it might generate unsafe actions.

It is not discussed how this approach could be used in practice, do the authors see this running in realtime in a vehicle or is it just used for data labeling offline? We need some runtime information for questions like this. Also, not all the tasks they want to solve would make sense in either case.
Yes
No ethics review needed.
1. Can the authors elaborate on the specific mechanisms used to extract and integrate environmental predicates into the retrieval process? How do these predicates influence decision-making during high-level action predictions?
2. The paper mentions a gradual increase in the value of σ during training for the PDCE loss. How was this approach determined, and what empirical evidence supports its effectiveness compared to alternative strategies?
3. How does the framework handle potential conflicts between high-level predictions and low-level control signals?
1. The paper provides a comprehensive set of experimental results demonstrating improvements in performance metrics, which is a solid contribution.
2. The experimental design and results are well-organized and clearly presented.
3. The plug-and-play nature of the framework allows it to be integrated with existing multimodal learning methods
The paper presents "SafeAuto," a framework for enhancing autonomous driving through multimodal foundation models. It focuses on high-level action prediction and justification, as well as low-level motion prediction, leveraging datasets like BDD-X and DriveLM. The authors claim improvements in performance metrics such as BLEU, CIDEr, and METEOR. However, the paper largely reads like a technical report, lacking significant theoretical contributions or novel methodologies, which raises questions about its overall impact and originality.
1. The paper appears to rely on established methodologies without introducing substantial theoretical advancements, limiting its originality and depth.
2. While the components such as the PDCE loss and post-safety verification are interesting, they do not significantly diverge from existing techniques.
3. There is little discussion on how different predicates are selected or their impact on the overall prediction performance
4. The evaluation does not seem to adequately address edge cases or scenarios where traditional models might fail. A deeper examination of how the framework handles uncommon but critical driving situations could provide insights into its robustness and safety.
We propose SafeAuto that includes a specialized PDCE loss for low-level control to improve precision and safety, and enhances high-level action prediction by integrating past driving experiences and precise traffic rules into multimodal models.
@inproceedings{
anonymous2024safeauto,
title={SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=RDz3EPC3Lp},
note={under review}
}
Traditional autonomous driving systems often struggle to harmonize high-level reasoning with low-level control, leading to suboptimal and even unsafe driving behaviors. The emergence of multimodal large language models (MLLMs), capable of processing visual and textual data, presents an opportunity to unify perception and reasoning tasks within a single framework. However, integrating precise safety knowledge into MLLMs for safe autonomous driving remains a significant challenge.
To address this, we propose SafeAuto, a novel framework that enhances MLLM-based autonomous driving systems by incorporating both unstructured and structured knowledge. In particular, we first propose the the Place-Dependent Cross-Entropy (PDCE) loss function, which is specifically designed to enhance the accuracy of low-level control signal predictions when treating numerical values as text.
To explicitly integrate precise safety knowledge into the MLLM to enable safe autonomous driving, we build a reasoning component for SafeAuto, which first parses driving safety regulations into first-order logic rules (e.g., "red light $\implies$ stop") and then integrates these rules into a probabilistic graphical model, such as a Markov Logic Network (MLN). The environment attributes, identified by attribute recognition models (e.g., detecting a red light), are used to form the predicates in MLN.
In addition, the environmental attributes utilized for reasoning are also considered factors in retrieval to construct a Multimodal Retrieval-Augmented Generation (RAG) model, which aims to learn from past similar driving experiences more effectively.
Extensive experiments demonstrate that SafeAuto significantly outperforms baselines across multiple datasets. By bridging the gap between high-level reasoning and low-level control, SafeAuto paves the way for more accurate, reliable, and safer autonomous driving, facilitating systems that learn effectively from experience, adhere to traffic regulations, and execute precise control actions.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Autonomous Driving; Multimodal Large Language Models; Multimodal Retrieval-Augmented Generation; Probabilistic Graph Model
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/46ef28450f13eba90bb41b8d738c230e769146d2.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Models
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission