Online Detecting LLM-Generated Texts via Sequential Hypothesis Testing by Betting
Large Language Models (LLMs);Machine-Generated Text;Text Generation Detection;Sequential Hypothesis Testing;Online Optimization
Yes
No ethics review needed.
See weaknesses. In particular, please answer the questions on the use case.
- Based on the authors' text, this is the first (if not the first) work to pose this problem. The algorithm is relatively easy to understand, although the presentation could improve slightly (see weaknesses) the first.

- The authors experiment with several different LLM detectors, showing how their method is easily adaptable to several LLM detection models.

- An extensive discussion of the models used and details of the method are presented in the appendixes. I did not double-check these; I only skimmed them, but they are welcomed.
In this paper, the authors present an online sequential hypothesis test for flagging LLM texts. At each time step, the algorithm updates an accumulated reward. 

The reward increases whenever an algorithm scores higher for an LLM text, indicating that we have evidence to reject the null. When this is not the case, the authors control the reward so that it is never negative via an Online Newton algorithm. This is simple because the score functions are bounded (commonly between 0 and 1).

Proofs that the algorithm works are adapted from the literature on sequential hypothesis tests. I point out that this is now my area of expertise, but they seem reasonable. Overall, being one of the first approaches of this kind, the algorithm naturally achieves good results (the baseline is a quite simple approach from 10 years ago).
Please provide an intuition for Line 11 earlier in the text. This is a crucial step of the algorithm, and I searched for the term "dt" on the pdf to better understand it. Ultimately, it is pretty intuitive since the score function is bounded. Nevertheless, with the explanation being brief and after the algorithm, it took me some time to understand it.

The paper does not discuss in detail why this approach is necessary, which is a significant weakness. Online testing may be employed, but is it the correct approach? For instance, how can I use this when evaluating student essays? I have only one essay per student. Maybe we could accumulate evidence that the account is a bot on a social media page. However, has this not already been done in batches? 

The point above is not a weakness of the method but of the use case. Please clarify why this method is needed. Some examples of online forums are present, but were these attacks sequential? Were they in batches?
Yes
No ethics review needed.
Please see Weaknesses.
S1. The problem of detecting LLM-generated text in an online, streaming context is important.

S2. The problem statement and the rationale behind the method are clear.

S3. The algorithm ensures a controlled false positive rate and has bounded detection time, which is crucial for reliability.

S4. The evaluation is very comprehensive. The paper tests the method with multiple score functions, datasets, and LLM configurations.
The paper addresses the challenge of detecting large language model (LLM)-generated text in an online, streaming context. Traditional methods focus on offline detection, but the authors propose a sequential hypothesis testing algorithm that enables real-time detection with statistical guarantees, such as controlling the false positive rate and expected detection time. This method leverages online optimization and betting strategies to discern between human and LLM-generated content across platforms like social media and news websites.
1. The method relies heavily on existing score functions from offline detection methods, potentially limiting its adaptability to new types of LLM-generated content. Certain score functions perform significantly better than others, indicating that the algorithm's efficacy may vary depending on the selected score function and dataset.

2. The experiments primarily focus on LLM-generated news about the Olympics and a pre-collected set of human-written news articles from XSum. This setup is limited in scope and may not reflect the diversity of content types that an LLM could generate. Would additional real-world validations in social media or low-quality content improve confidence in the algorithm’s effectiveness across various online platforms? (This comment doesn't lower my rating for this article, but rather my expectations for follow-up work)

3. The algorithm requires estimations for parameters like $\epsilon$ and $d$ before the main process begins. This estimation is based on initial samples, which introduces the risk of inaccurate settings, especially if the initial sample size is small. Can further optimization in estimating parameters improve consistency across different datasets and score functions?
Yes
No ethics review needed.
I miss a contextualization of the state of the art of LLM-generated text detection – are those algorithms effective?

It seems that an important characteristic of the problem tackled by the authors is that the paper tries to classify sources instead of individual documents – hence, the signal and confidence is dramatically higher than when you have to judge a single document. Consider highlighting this and take a look at https://arxiv.org/html/2406.07016v1.

We do need both validation and characterization of the dataset used in the experiments.
The paper is well-written paper, easy to understand, the method is defined clearly. The experimental results, considering the absolute numbers,  seem to outperform the employed baselines. The method is intuitive and the weight assignment strategy may help the interpretability of the results. The originality is a bit limited, since it basically adapts sequential hypothesis testing for detecting LLM-generated texts. The significance is still rather limited since they use a dataset that they constructed and it is not published or employed in other works.
The authors study the problem of inferring whether a source of texts is human or an LLM in an online setting, i.e, in which texts are observed as a continuous stream, as in social media and online forums.

The paper employs a sequential hypothesis testing framework, allowing the judgment about the likelihood of a source being a LLM to be made dynamically rather than waiting for the entire dataset. The framework is fed with state-of-the-art offline approaches to detect LLM-generated text, and at each step the "wealth" of the system is updated based on the probability of a text being generated by an LLM. This accumulated wealth serves as evidence to reject the null hypothesis (that the text was written by a human) and declare the source as an LLM. 

The paper discusses the statistical guarantees of the algorithm, including the controlled false positive rate and the average time to correctly detect an LLM. Additionally, experimental results are presented, demonstrating the effectiveness of the proposed method compared to baseline methods.
First of all, I believe that authors should better characterize how online detection differs from the offline detection from an algorithmic perspective. I understand that latency and throughput became main requirements, but it is not clear to me, with respect to content handling how it differs from the offline problem, for which there are a large number of works published. Interestingly enough, there are no computational performance assessments in the paper.

The evaluation seems to me as the main weakness to the manuscript. There seems to be several design and experimental decisions that are quite adhoc. For instance, the authors mention that they used 10 score functions in total. How did these functions chosen? Why? Similar issues arise regarding the dataset, which were constructed by the authors and it is not detailed how. For instance, they mention that they generate fake news considering the first 30 tokens of each real news. Is there a validation that the generated fake news are really fake? To what extent?  I also missed more actual baselines that have been employed on the task or similar ones. 

Although the authors provided a quite detailed description of the experimental setup, it is not clear to what extent it is reproducible. For instance, in section 4.1 the authors mention that they constructed a dataset containing both real and fake news, but both the construction process and the characteristics of the resulting dataset are not provided.
Yes
N/A
No ethics review needed.
1. Why introduce a new dataset? Are the existing datasets inadequate? Is 500 samples insufficient for credibility? Why generate only one new dataset instead of two or three? How can we ensure that the series of input texts y_x come from the same source?
2. The examples provided at the beginning are unconvincing; they primarily address account and regulatory issues. Why not present more compelling examples that better reflect your motivation for the revision?
3. For hyperparameters such as d_* and epsilon, is there an optimal selection method? What are the trade-offs involved?
1. The application of sequential hypothesis testing with betting is innovative and well-suited to the online detection problem, distinguishing this approach from conventional offline methods.
2. The method includes strict statistical guarantees for false positive rates and expected detection times, which is critical for applications requiring high reliability.
3. The paper demonstrates solid technical grounding, and a wealth of theoretical knowledge, and is well-written.
This paper proposes a novel method for the online detection of text generated by large language models (LLMs), employing sequential hypothesis testing based on betting techniques. The approach aims to provide robust statistical guarantees, balancing type-I error control with efficient detection times. By leveraging sequential hypothesis testing, the method ensures accurate identification of LLM-generated text while minimizing false positives.
1. The abstract lacks clarity, particularly in the phrasing "publish content in a streaming fashion." The motivation should emphasize the need for continuous online detection due to the constant emergence of text, which differs from offline detection. This ongoing process allows for enhanced detection performance through hypothesis testing.
2. Based on this motivation, the baseline experimental setup appears problematic. It should compare your method with current offline approaches (e.g., detection without leveraging continuous information) rather than with other hypothesis tests, to better evaluate effectiveness and identify advantages and disadvantages.
3. The scoring model has not been optimized for this task. While experiments on text domain matching were conducted, no solutions were proposed. The experiments are somewhat limited; the focus should be on the modules and parameter settings of sequential hypothesis testing compared to offline methods. Ablation studies should emphasize this point.
Yes
No ethics review needed.
* How does the method handle situations where the source content may contain a mixture of human-written and machine-generated text?
* Could the sequential hypothesis testing technique adapt dynamically to model drift or evolution in LLMs over time?
* The paper explores the detection of generated text in a streaming context, which is timely and might be useful for real-world applications such as monitoring social media or news platforms.
* The authors build on the theory of sequential hypothesis testing by betting, establishing clear statistical guarantees. This foundation provides rigor, including control over type-I errors and expected detection times, which is valuable for applications requiring reliable, real-time decisions.
* The authors test their approach using different scoring models, source models, and baselines, showing a commendable effort in evaluating robustness and sensitivity to various parameters.
This paper proposes to detect LLM-generated texts in an online streaming scenario, addressing the gap in current detection methods that primarily operate offline. The authors introduce an algorithm based on sequential hypothesis testing through a betting mechanism, allowing for real-time, statistically robust detection with controlled false-positive rates and bounded detection times. Experimental results demonstrate that the method can effectively distinguish LLM-generated text sequences from human-written texts, even under practical constraints.
* The scope of the paper’s application could be better defined. For example, is the goal to identify whether the content in a stream is entirely machine-generated or to ascertain the overall nature of the source (e.g., a bot account)? If the latter, the assumption that all content from a source is generated by a single LLM may not hold in practice, where mixed content (mixed LLMs or mixed human and LLM-generated text) is common. How the method would handle mixed content or bot accounts using various LLMs is unclear.
* The experiments rely on a single dataset and use a relatively small amount of data, which weakens the persuasiveness of the results. Furthermore, the generated texts tested in this study are produced by limited models; the evaluation would be more compelling if it included results from more advanced models, such as LLaMA 3, Mistral, or GPT-4.
* The writing can be polished for clarity and completeness. Many of the experimental result tables are placed in the appendix, which could have been included in the main text given there is remaining space.
@inproceedings{
anonymous2024online,
title={Online Detecting {LLM}-Generated Texts via Sequential Hypothesis Testing by Betting},
author={Anonymous},
booktitle={Submitted to The Thirteenth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=WbqBj2aC5k},
note={under review}
}
Developing algorithms to differentiate between machine-generated texts and human-written texts has garnered substantial attention in recent years. Existing methods in this direction typically concern an offline setting where a dataset containing a mix of real and machine-generated texts is given upfront, and the task is to determine whether each sample in the dataset is from a large language model (LLM) or a human. However, in many practical scenarios, sources such as news websites, social media accounts, or on other forums publish content in a streaming fashion. Therefore, in this online scenario, how to quickly and accurately determine whether the source is an LLM with strong statistical guarantees is crucial for these media or platforms to function effectively and prevent the spread of misinformation and other potential misuse of LLMs. To tackle the problem of online detection, we develop an algorithm based on the techniques of sequential hypothesis testing by betting that not only builds upon and complements existing offline detection techniques but also enjoys statistical guarantees, which include a controlled false positive rate and the expected time to correctly identify a source as an LLM. Experiments were conducted to demonstrate the effectiveness of our method.
I certify that there is no URL (e.g., github page) that could be used to find authors’ identity.
I acknowledge that I and all co-authors of this work have read and commit to adhering to the ICLR Code of Ethics.
Large Language Models (LLMs) Machine-Generated Text Text Generation Detection Sequential Hypothesis Testing Online Optimization
I certify that there is no acknowledgement section in this submission for double blind review.
/pdf/ed42528c2618e6de3d972baee603db0776214bc0.pdf
foundation or frontier models, including LLMs
I understand the reciprocal reviewing requirement as described on https://iclr.cc/Conferences/2025/CallForPapers. If none of the authors are registered as a reviewer, it may result in a desk rejection at the discretion of the program chairs. To request an exception, please complete this form at https://forms.gle/Huojr6VjkFxiQsUp6.
I certify that this submission complies with the submission instructions as described on https://iclr.cc/Conferences/2025/AuthorGuide.
Online Detecting LLM-Generated Texts via Sequential Hypothesis Testing by Betting
ICLR 2025 Conference Submission
ICLR.cc/2025/Conference/Submission