Learning Reliable Rules by Re-generating Deep Features
Interpretable ML;Neuro-symbolic AI;SATNet;Logical Reasoning
Yes
No ethics review needed.
As mentioned in the weaknesses section, the visualization of the learned rules indicates that the target-related MSF pays more attention to its surrounding features, which is a trivial finding. Since CNN features are derived from convolutions with neighboring pixels, the features at each position are inherently closely related to those in the surrounding area. Could you provide some more meaningful rules from different perspectives?

The features in the paper appear to be derived from the last layer of the convolutional module in the model. What would happen if features from other convolutional layers were used? Could different layers with varying depths extract distinct rules?
Experiments on MNIST and Imagenette datasets demonstrate the effectiveness of Vi-SATNet in feature regeneration and classification, showing impressive results even under high masking ratios.

The visualizations of the learned rules offer an intuitive understanding of feature dependencies.
This paper extends SATNet by enabling it to learn logical rules from the complex feature space of real-life images, allowing it to regenerate masked features while maintaining high classification accuracy. 
Experimental results on the features from some pre-trained models show that the learned reasoning rules allow Vi-SATNet to re-generate missing feature vectors accurately.
Finally, the authors present a visualization technique to illustrate the rules learned by Vi-SATNet models.
The visualization of the learned rules is not very clear. Whether in the foreground or background, MSF tends to pay more attention to the position around the target, which is a rather trivial result.
Yes
No ethics review needed.
See above.
1.	Vi-SATNet extends the capabilities of the original SATNet by enabling the learning of logical reasoning rules within the feature spaces of convolutional neural networks (CNNs). This generalization allows it to operate effectively on complex datasets beyond simple logical puzzles.
2.	The study presents a new method for feature regeneration that leverages learned logical rules derived from deep features, thereby enhancing the interpretability and reliability of convolutional neural networks (CNNs).
3.	Vi-SATNet can be seamlessly integrated as a drop-in layer within existing CNN architectures, requiring no retraining or fine-tuning.
This manuscript introduces Visual SATNet (Vi-SATNet), an innovative extension of SATNet aimed at learning logical rules within complex feature spaces, particularly those generated by convolutional neural networks (CNNs). By employing Vi-SATNet for feature regeneration, the study illustrates its capacity to improve interpretability in CNNs during image classification tasks. Experiments conducted on datasets such as MNIST and Imagenette reveal promising classification outcomes across different levels of feature masking.
1.	The manuscript effectively outlines the architecture of Vi-SATNet and establishes an evaluation framework through feature regeneration tasks, employing cosine similarity and Vi-C agreement as measurable metrics for assessing feature quality. However, it could enhance clarity regarding the Vi-SATNet training process, especially concerning hyperparameter selection.
2.	The manuscript describes visualization through minimal significant feature sets (MSFs) and their corresponding receptive fields. However, a more in-depth explanation of how MSFs correlate with specific learned rules would be advantageous. Without clear criteria for evaluating rule quality, the reader may find it challenging to interpret the rules’ significance. 
3.	To enhance the study's rigor, it is recommended to include additional datasets, such as CIFAR-10 or other complex real-world datasets.
4.     Some related works are needed to discuss in the manuscript, such as [1-2].

[1] Wei J, Garrette D, Linzen T, et al. Frequency effects on syntactic rule learning in transformers[J]. arXiv preprint arXiv:2109.07020, 2021.
[2] Zhang W, Mo S, Liu X, et al. Learning robust rule representations for abstract reasoning via internal inferences[J]. Advances in Neural Information Processing Systems, 2022, 35: 33550-33562.
Yes
No ethics review needed.
N/A
1. The experiment results are convincing, showing the effectiveness of the proposed method.
2. The author provides detailed definitions and algorithm processes to demonstrate their ideas and contributions.
In this paper, the author proposed Visual SATNet (Vi-SATNet) which targets learning logic reasoning rules in a general domain. Specifically, the author trains and evaluates Vi-SATNet on deep feature re-generation and the experiment results show the effectiveness of the proposed methods.
1. The author proposes a method to learn logical reasoning rules in a more general domain. However, different from the original SATNet, which targets solving Sudoku, a task has a clear logic process, how can we understand the logical reasoning for the real-life images? The author should provide more explanation and analysis.
2. In Table 1, the author obtained the best results with regeneration with a 30% mask ratio. However, without regeneration, the model's performance will drop with a higher maks ratio.  Why does the model have the best performance with a 30% mask ratio? The author should provide more analysis.
@inproceedings{
anonymous2024learning,
title={Learning Reliable Rules by Re-generating Deep Features},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=ELQ8X02IEp},
note={under review}
}
Improving the interpretability and reliability of deep learning models is essential for advancing machine learning applications, though it remains a significant challenge. One promising approach is the integration of logical reasoning into deep learning systems. Previous works have demonstrated that SATNet, a differentiable MaxSAT solver, can learn interpretable and reliable rules from input-output examples in puzzle domains. In this work, we propose *Visual SATNet* (Vi-SATNet), an extended version of SATNet capable of learning logical reasoning rules in more general and complex domains, such as the feature space of real-life images. We find that, given a pre-trained deep convolutional neural network (CNN) architecture, a Vi-SATNet layer can be integrated and trained efficiently to learn a set of reasoning rules on the deep features, guiding the classifier’s decision. Vi-SATNets are trained to perform feature re-generation tasks for a given image dataset, where the re-generated features maintain high accuracy when used for image classification, proving their quality. In our experiment on the Imagenette dataset with a pre-trained VGG19 model, masking out 10\% to 80\% of the features results in classification accuracy ranging from 98.50\% to 93.92\% with Vi-SATNet re-generation, compared to 97.07\% to 9.83\% without re-generation. Furthermore, we introduce a visualization method to illustrate the rules learned by Vi-SATNets, thereby enhancing the interpretability of the pre-trained CNN model.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Interpretable ML Neuro-symbolic AI SATNet Logical Reasoning
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/78f129e648ba580e23394cdd39d7fb856b65af3f.pdf
interpretability and explainable AI
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Learning Reliable Rules by Re-generating Deep Features
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission