Fast Salient Factor Concentration (FSFC) Recurrent Neural Network for Text Classification
Text Classification;Semantic Information Clustering;Recurrent Neural Network
Thank you to all the reviewers for their time and valuable feedback. The original intent of our method was to replace the basic LSTM or GRU modules in various complex models to accelerate training and inference speed. There are aspects of the paper that need improvement, and after further discussion, we have decided to withdraw the manuscript for further refinement. Once again, we sincerely appreciate the reviewers' efforts.
I have read and agree with the venue's withdrawal policy on behalf of myself and my co-authors.
Yes
No ethics review needed.
See weaknesses.
- Easy to follow and clearly written;
- Lightweight solution for text classification compared to transformer models.
The paper introduces an RNN-based model called Fast Salient Factor Concentration (FSFC), designed to improve training efficiency in text classification by using short-term memory and semantic clustering. While FSFC's concept is intuitive, it has several critical limitations. The approach offers limited innovation, appearing incremental relative to existing RNN and attention-based techniques. Additionally, the research feels outdated, as NLP has largely moved toward transformer-based models. Performance evaluations yield mixed results: FSFC reduces training time but sacrifices accuracy on some datasets. Moreover, baseline comparisons are insufficient, with FSFC only benchmarked against LSTM and GRU models, excluding state-of-the-art transformers like RoBERTa and recent LLMs.
- Limited novelty: The concept of text compression and segmentation is well-explored, with similar techniques in attention mechanisms and memory simplification.
- Outdated approach: It’s advisable for the authors to employ BERT-based models or LLMs, as transformers are now the NLP standard. Although addressing transformer complexity with RNNs is a good direction, applying it to basic tasks like text classification feels outdated.
Yes
No ethics review needed.
1. Does your model reduce the inference time?
This paper proposes a structure that is 5 times faster to train than LSTM.
This paper provides a method to remove the gating mechanisms of LSTM and condense memory.  The proposed method achieves better training speed while retaining accuracy.
1. It is tough to understand Figure 1. I can not find any RNN or recurrent in this figure. What is your whole network? Moreover, this figure doesn't have detailed captions.

2.  Why does long-term memory not matter in classification? Please provide justifications with citations or experiments.

3. This paper does not compare with other improvements on LSTM. Do other improvements over LSTM reduce the training time and achieve better accuracy?

4.  This paper does not provide details about the experiment setup.  Although we do not know what the network structure proposed in this paper looks like, nor do we know how the configuration compares to GRU and LSTM, or the length of the experimental data.

5. E-score is not more effective than just presenting accuracy and time.
Yes
No ethics review needed.
1. What is the motivation to propose an alternative to RNN for text classification? What problems are you trying to solve? Does "the long-term memory mechanisms of traditional RNNs do not fully align with human cognitive learning processes" really matter? How does it impact the performance in text classification tasks? Could you give us some concrete examples of how the misalignment between RNNs and human cognitive processes impacts performance in text classification tasks?

2. What is the contribution of FSFC? The author should discuss how FSFC compares to or complements more recent approaches in NLP. RNN-based methods for text classification are quite outdated now. While I am not against RNNs, achieving minor improvements and reducing training time seems to bring little new knowledge to the current NLP community.

3. How about employing other methods like BERT and LLMs? What is the value of an alternative to RNN compared with pretrained language models? Why do we still need RNN-based methods for text classification?
1. The paper is well-organized, and the writing is fine.

2. The paper evaluates the performance of FSFC on four datasets. The experimental results show the effectiveness and efficiency of FSFC.
The paper proposes Fast Salient Factor Concentration (FSFC) RNN, a new architecture for classification tasks, to enhance the processing of crucial information by dynamically clustering and compressing semantic information. The performance on YelpReviewFull proves that FSFC has a higher accuracy with less training time.
1. The motivation of this paper is not very clear. I'm not sure what problems this paper is trying to solve (Text classification? An alternative to RNN? An alternative to RNN on Text classification?).  It would be beneficial to specify what problems the authors aim to address—are they focusing on text classification, proposing an alternative to RNNs, or something else? Anyway, the contribution seems limited.

2. The idea presented in this paper does not appear particularly interesting. It proposes an alternative to RNNs for text classification, but I believe it lacks significant contributions to the current NLP community. Nowadays, many practitioners favor pre-trained models over RNNs for text classification tasks. Additionally, large language models (LLMs) tend to focus on multi-tasking rather than on single NLP tasks.

3. If the goal is to propose a new RNN, conducting experiments only on text classification is insufficient to verify the method's generalization.

4. Even for text classification tasks, the models compared in the paper are not comprehensive (e.g., ELMO, BERT, LLMs, and so on). The paper lacks comparisons with these strong methods for text classification. As I mentioned in Weakness 2, these models are precisely the kinds of models that are commonly utilized in the field of text classification today. If the paper focuses on text classification, not comparing with these mainstream models would be unfair.
@misc{
xia2024fast,
title={Fast Salient Factor Concentration ({FSFC}) Recurrent Neural Network for Text Classification},
author={Weihao Xia and Huachuan Wang and Qiu Chen and Junlong Ma and James Ting-Ho Lo},
year={2024},
url={https://openreview.net/forum?id=4ymHtDAlBv}
}
Models based on Recurrent Neural Networks (RNNs) have been widely employed for text classification tasks. Traditional RNNs primarily emphasize long-term memory capabilities. However, this approach does not fully align with human cognitive learning processes, particularly in the context of classification tasks. The human brain typically extracts essential information relevant to the classification categories, disregards irrelevant details, and compresses the input to accelerate decision-making. Inspired by this, we propose a novel architecture, the Fast Salient Factor Concentration (FSFC) RNN, specifically designed for classification tasks. FSFC dynamically clusters and compresses semantic information by leveraging the short-term memory capabilities of recurrent neural networks. Experimental results demonstrate that FSFC achieves performance comparable to existing RNNs, while significantly improving training efficiency in classification tasks. Based on the YelpReviewFull dataset, FSFC improves accuracy by 1.37% over Long Short-Term Memory (LSTM), while reducing training time by 86%. Additionally, we propose a new evaluation metric, E-score, which integrates both accuracy and time efficiency to comprehensively assess the overall performance of each network.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
~Weihao_Xia2 ~Huachuan_Wang3 ~Qiu_Chen1 ~Junlong_Ma3 ~James_Ting-Ho_Lo1
Weihao Xia Huachuan Wang Qiu Chen Junlong Ma James Ting-Ho Lo
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Text Classification Semantic Information Clustering Recurrent Neural Network
I certify that there is no acknowledgement section in this submission for double blind review.
xia|fast_salient_factor_concentration_fsfc_recurrent_neural_network_for_text_classification
/pdf/6c754b36fb662482c6e84cee434cb837a190bbed.pdf
learning on time series and dynamical systems
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Fast Salient Factor Concentration (FSFC) Recurrent Neural Network for Text Classification
ICLR 2025 Conference Withdrawn Submission
ICLR.cc/2025/Conference/Withdrawn_Submission