Language Models can Self-Lengthen to Generate Long Texts
LLM;Synthetic Data;Long Output
Yes
No ethics review needed.
please see weaknesses
1. Self-Lengthen is cost-effective and easy to use. It only requires a set of seed instructions for long-text output tasks and an open-source instruction model to automatically enhance the model's ability to generate long-text outputs.

2. This paper proposes a two-stage extension method that ensures the extension does not end normally. This creates space for the model to seamlessly connect preceding and succeeding segments, thereby enhancing its ability to complete extension tasks.

3. The author provided the complete code, ensuring high reproducibility.
This paper proposes an innovative iterative training framework called Self-Lengthen, which consists of two roles: a generator and an extender. The generator produces an initial response, which is then split and extended by the extender. This process generates new, longer responses that are used to repeatedly train both the generator and the extender. As a result, the model is gradually trained to handle increasingly longer responses.
1. The paper adopts a rule-based approach to filter out invalid responses to ensure their quality. How is 'frequent repetition' specifically determined? Is it based on rules or scored by a more advanced LLM? If the extended responses merely describe the same meaning in different styles, is this kind of extension meaningful?

2. During the process of instruction evolution, some instruction data are inherently unsuitable for this type of evolution. For example, if the response to an instruction is a proper noun, extending it might generate some meaningless content. Is such data helpful for training? Or, even if it gets filtered out during the evolution process, it still wastes computational resources. So, when collecting seed data, can we pre-filter these unsuitable instructions based on certain methods to save costs?

3. Hope to get an analysis of the GPU hours required for the Self-Lengthen extension in order to discuss its efficiency issues.
Yes
No ethics review needed.
- Why did you choose to truncate at  ½ and ⅔ in the micro iteration? Is there additional evidence showing this decision is empirically the best? If so, any conjecture why this is the case? 
Does this method affect short-form generation capability? 
- Can you provide more explanation about the distinct score? The responses of the backtranslation method are written by humans. Can you explain why its distinct score is lower than self-lengthen?
- Can you give more explanation for why qwen2 and llama3 performance differ significantly for the backtranslation baseline?
- Can you estimate how significant the gains of the proposed method is?
- This paper proposes a new method to improve language models’ long-form generation performance.
- This topic is relevant to a wide range of applications which are bottlenecked by the response length that language models can reliably output.
The paper proposes a new method Self-Lengthen to tackle long-form generation, which existing large language models (LLMs) often fall short. The proposed method involves iteratively training the Generator and the Extender on progressively long responses generated by the model from the previous iteration. To produce these long responses, prompting and rule-based methods are introduced to guarantee the output length and quality. Experimental results show that the proposed method can extend the response length of qwen2-7b and llama3.1-8b and outperform the existing methods such as instruction back-translation and plan-and-write.
- The proposed method contains a seemingly arbitrary decision of truncating the response to ½ or ⅔ for further extension. It is unclear why these cutoffs were chosen and how they compare to other cutoffs.
- The proposed method utilized surface form heuristics (e.g. length, repetition)  to ensure the quality of extended responses, while the semantic content is not quality assured. It is unclear if training on synthetic self-generated data hurts other LM capabilities, e.g., math/code reasoning and generating factual content.
- It is unclear if the proposed method hurts the short-form generation performance.
- In Section 5.1 “Due to the extraordinarily long length of the responses, we utilized GPT-4o to summarize and analyze them, aiding evaluators in their assessments for improved readability and increased annotation efficiency”. Introducing proprietary models into human evaluation is not preferable as the model-introduced error cannot be reliably detected and can further complicate human evaluation results.
- In section 5.2 “To ensure a fair evaluation, we introduce length constraints by applying the same workflow and prompts to their original queries. “ It is unclear if the proposed method is better because the baselines are corrupted by these additional steps. 
- The total evaluation set is quite small (240 examples) and it is unclear how statistically significant the improvement is (e.g. 85.82 vs 85.52). I do notice llama3.1 with backtranslation performs significantly worse than the counter part of qwen. However, no extensive explanation is provided.
- Overall, I believe the paper can be improved by clarifying some experimental and evaluation details.
Yes
The design of LonGen benchmark is too similar to the benchmark in LongWriter (i.e.,  LongBench-Write), and many tables (e.g., Table 2, 3) and figures (e.g., Fig 5, 6) are similar to those in LongWriter without proper citations. The authors should give a more detailed explanation and comparison.
Yes, Research integrity issues (e.g., plagiarism, dual submission)
1. How do you split the response $y$ into $y[:1/2]$ or $y[:2/3]$? Is this based on the number of tokens or what other criteria?
2. How are the distinct scores in human evaluation calculated?
3. In human evaluation, why select different queries for P&W and Slef-Lengthen? Comparing responses to the same queries seems more reasonable.
4. What are the source and statistics of the training data? The paper only mentions they utilize user logs without further details.
5. What is the total number of iterations of the trained models?
6. Why does Section 5.2 say that "LongWriter datasets either lack appropriate length requirements in their queries or have unsuitable length constraints, resulting in results that fall significantly short of expectations."? As I know, LongWrite dataset has length constraints in the queries. Are there any evaluation results of directly using the LongWriter dataset or their open-sourced models such as LongWriter-llama3.1-8b to support the claim?
1. Compared to the previous method, Self-Lengthen has no need for auxiliary data or powerful proprietary models, and supports outputs with more diverse styles and types.
2. Experiments on benchmarks and human evaluations show that Self-Lengthen outperforms existing methods in long-text generation when applied to top open-source LLMs such as Qwen2 and LLaMA3.
This paper proposes an iterative training framework called Self-Lengthen to enable LLMs to generate long responses, without the need for auxiliary data or proprietary models. The framework consists of two roles: the Generator produces the initial response, which is then split and expanded by the Extender to construct a new training set with longer responses. This process can be done iteratively to make the model handle increasingly longer responses.
1. The design of LonGen benchmark is too similar to the benchmark in LongWriter (i.e.,  LongBench-Write), and many tables (e.g., Table 2, 3) and figures (e.g., Fig 5, 6) are similar to those in LongWriter without proper citations. The authors should give a more detailed explanation and comparison.

2. There are many missing details in the experiments, including the calculation method of distinct scores, the training data statistics, and the supported maximum output length.

3. The length control ability (left of Fig. 1, not well aligned with the desired length)  and maximum reported output length (8k) of the trained models are relatively poor compared to the SoTA methods such as LongWriter.
Yes
No ethics review needed.
1. **Comparison with LongWrite**: The differences between Self-Lengthen and LongWrite are not clearly articulated. For instance, both methods use metrics like length-following and quality. The paper should provide a more explicit comparison to distinguish itself from LongWrite, including discussing any unique advantages or limitations.
2.  Figure 6: The figure appears to have display issues and should be corrected for better readability.
1, The proposed Self-Lengthen framework introduces a unique iterative approach to improve long-text generation by utilizing the intrinsic capabilities of LLMs without relying on additional external datasets or proprietary models. 

2, The method is simple yet practical, focusing on leveraging existing models' capabilities through iterative extension. This makes the method easy to implement and potentially scalable to various domains where long-text generation is required.

3, The authors conduct extensive evaluations using different open-source models and comparison baselines. The experiments are well-designed to demonstrate the improvements in generating longer outputs while maintaining acceptable quality.
This paper introduces Self-Lengthen, an iterative training framework designed to enhance the long-text generation capabilities of large language models (LLMs) without auxiliary data or proprietary models. The framework employs a Generator and an Extender to iteratively extend model outputs, resulting in progressively longer and more refined responses. The experimental results show that Self-Lengthen outperforms existing methods like instruction backtranslation and behavior imitation, particularly in long-text generation tasks. The paper also provides a thorough empirical evaluation, including human assessments and benchmark comparisons.
1. **Motivation**: The motivation for this work is not sufficiently compelling. Regarding the instruction backtranslation method, SlimPajama already provides a large amount of long-text data generated by real-world [1]. For the behavior imitation approach, it is difficult to agree that there is a significant difference between using GPT-4 and open-source models. The LongWrite's agentwrite method can also use open-source models to generate data, which undermines the claimed uniqueness of Self-Lengthen.

2. **Methodology Design**: The Self-Lengthen method, while simple, may not be entirely reasonable. When humans perform long-text tasks, they typically do not opt for multiple iterative expansions. The multiple extension approach seems different to agentwrite from LongWrite, where an outline is first provided, and then expanded. It is unclear why the former approach would yield better performance compared to the latter. Providing a justification or empirical evidence to support this claim would strengthen the argument.

3. **Evaluation Metrics**: For long-text tasks, people often prioritize the quality of generated text over whether the length requirements are strictly met. Regarding the Output Quality Score metric, I have concerns about relying on LLMs for quality evaluation.  
   - **Subjectivity**: Much of creative writing inherently involves strong subjective elements, especially concerning creativity. Using LLMs to evaluate such content may not capture the nuances of human judgment.
   - **Limitations of LLMs**: While LLMs are effective at long-text retrieval tasks, I believe they are still insufficient for fully understanding the content of long documents, particularly regarding holistic comprehension. Relying entirely on LLMs for evaluation may not be appropriate.



[1] [SlimPajama: A 627B Token Cleaned and Deduplicated Version of RedPajama](https://cerebras.ai/blog/slimpajama-a-627b-token-cleaned-and-deduplicated-version-of-redpajama)  
[2] *LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs*
@inproceedings{
anonymous2024language,
title={Language Models can Self-Lengthen to Generate Long Texts},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=gGElk5T8sD},
note={under review}
}
Recent advancements in Large Language Models (LLMs) have significantly enhanced their ability to process long contexts, yet a notable gap remains in generating long, aligned outputs. 
This limitation stems from a training gap where pre-training lacks effective instructions for long-text generation, and post-training data primarily consists of short query-response pairs. 
Current approaches, such as instruction backtranslation and behavior imitation, face challenges including data quality, copyright issues, and constraints on proprietary model usage.
In this paper, we introduce an innovative iterative training framework called Self-Lengthen that leverages only the intrinsic knowledge and skills of LLMs without the need for auxiliary data or proprietary models. 
The framework consists of two roles: the Generator and the Extender. The Generator produces the initial response, which is then split and expanded by the Extender. This process results in a new, longer response, which is used to train both the Generator and the Extender iteratively. Through this process, the models are progressively trained to handle increasingly longer responses. 
Experiments on benchmarks and human evaluations show that Self-Lengthen outperforms existing methods in long-text generation, when applied to top open-source LLMs such as Qwen2 and LLaMA3.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
LLM Synthetic Data Long Output
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/4b340c229f1658aae660ceecf25d917435a4f799.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Language Models can Self-Lengthen to Generate Long Texts
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission