<!DOCTYPE html><html lang="en">
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<title>[2406.20094] Scaling Synthetic Data Creation with 1,000,000,000 Personas</title><meta property="og:description" content="We propose a novel persona-driven data synthesis methodology that leverages various perspectives within a large language model (LLM) to create diverse synthetic data. To fully exploit this methodology at scale, we intr…">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="twitter:card" content="summary">
<meta name="twitter:title" content="Scaling Synthetic Data Creation with 1,000,000,000 Personas">
<meta name="twitter:image:src" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta name="twitter:image:alt" content="ar5iv logo">
<meta property="og:title" content="Scaling Synthetic Data Creation with 1,000,000,000 Personas">
<meta property="og:site_name" content="ar5iv">
<meta property="og:image" content="https://ar5iv.labs.arxiv.org/assets/ar5iv_card.png">
<meta property="og:type" content="article">
<meta property="og:url" content="https://ar5iv.labs.arxiv.org/html/2406.20094">

<!--Generated on Fri Jul  5 17:51:54 2024 by LaTeXML (version 0.8.8) http://dlmf.nist.gov/LaTeXML/.-->
<meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no">

<script>
  function detectColorScheme(){
    var theme="light";
    var current_theme = localStorage.getItem("ar5iv_theme");
    if(current_theme){
      if(current_theme == "dark"){
        theme = "dark";
      } }
    else if(!window.matchMedia) { return false; }
    else if(window.matchMedia("(prefers-color-scheme: dark)").matches) {
      theme = "dark"; }
    if (theme=="dark") {
      document.documentElement.setAttribute("data-theme", "dark");
    } else {
      document.documentElement.setAttribute("data-theme", "light"); } }

  detectColorScheme();

  function toggleColorScheme(){
    var current_theme = localStorage.getItem("ar5iv_theme");
    if (current_theme) {
      if (current_theme == "light") {
        localStorage.setItem("ar5iv_theme", "dark"); }
      else {
        localStorage.setItem("ar5iv_theme", "light"); } }
    else {
        localStorage.setItem("ar5iv_theme", "dark"); }
    detectColorScheme(); }
</script>
<link media="all" rel="stylesheet" href="/assets/ar5iv-fonts.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv.0.7.9.min.css"><link media="all" rel="stylesheet" href="/assets/ar5iv-site.0.2.2.css">
</head>
<body>
<div class="ltx_page_main">
<div class="ltx_page_content">
<article class="ltx_document ltx_authors_1line">
<h1 class="ltx_title ltx_title_document">Scaling Synthetic Data Creation with 1,000,000,000 Personas</h1>
<div class="ltx_authors">
<span class="ltx_creator ltx_role_author">
<span class="ltx_personname">Xin Chan, Xiaoyang Wang, Dian Yu, Haitao Mi, Dong Yu 
<br class="ltx_break">Tencent AI Lab Seattle 
<br class="ltx_break"><a target="_blank" href="https://github.com/tencent-ailab/persona-hub" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/tencent-ailab/persona-hub</a>
</span></span>
</div>

<div class="ltx_abstract">
<h6 class="ltx_title ltx_title_abstract">Abstract</h6>
<p id="id1.1" class="ltx_p">We propose a novel persona-driven data synthesis methodology that leverages various perspectives within a large language model (LLM) to create diverse synthetic data. To fully exploit this methodology at scale, we introduce Persona Hub – a collection of 1 billion diverse personas automatically curated from web data. These 1 billion personas (<math id="id1.1.m1.1" class="ltx_Math" alttext="\sim" display="inline"><semantics id="id1.1.m1.1a"><mo id="id1.1.m1.1.1" xref="id1.1.m1.1.1.cmml">∼</mo><annotation-xml encoding="MathML-Content" id="id1.1.m1.1b"><csymbol cd="latexml" id="id1.1.m1.1.1.cmml" xref="id1.1.m1.1.1">similar-to</csymbol></annotation-xml><annotation encoding="application/x-tex" id="id1.1.m1.1c">\sim</annotation></semantics></math>13% of the world’s total population), acting as distributed carriers of world knowledge, can tap into almost every perspective encapsulated within the LLM, thereby facilitating the creation of diverse synthetic data at scale for various scenarios. By showcasing Persona Hub’s use cases in synthesizing high-quality mathematical and logical reasoning problems, instructions (i.e., user prompts), knowledge-rich texts, game NPCs and tools (functions) at scale, we demonstrate persona-driven data synthesis is versatile, scalable, flexible, and easy to use, potentially driving a paradigm shift in synthetic data creation and applications in practice, which may have a profound impact on LLM research and development.</p>
<p id="id2.id1" class="ltx_p"><span id="id2.id1.1" class="ltx_text ltx_font_bold ltx_font_italic" style="color:#FF0000;">DISCLAIMER<span id="id2.id1.1.1" class="ltx_text ltx_font_medium">: <span id="id2.id1.1.1.1" class="ltx_text ltx_font_upright">Persona Hub</span> can facilitate synthetic data creation at a billion-scale to simulate diverse inputs (i.e., use cases) from a wide variety of real-world users. If this data is used as input to query a target LLM to obtain its outputs at scale, there is </span>a high risk<span id="id2.id1.1.2" class="ltx_text ltx_font_medium"> that the LLM’s knowledge, intelligence and capabilities will be dumped and easily replicated, thereby challenging the leading position of the most powerful LLMs (e.g., our approach allows a 7B LLM to achieve 65% on MATH, matching the performance of <span id="id2.id1.1.2.1" class="ltx_text ltx_font_typewriter ltx_font_upright">gpt-4-turbo-preview</span>). This tech report is </span>for research purposes only<span id="id2.id1.1.3" class="ltx_text ltx_font_medium">. It is crucial to avoid misuse and ensure ethical and responsible application. We discuss its broad impact and potential concerns in detail in Section <a href="#S5" title="5 Broad Impact and Ethical Concerns ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>.</span></span></p>
</div>
<figure id="S0.F1" class="ltx_figure"><img src="/html/2406.20094/assets/x1.png" id="S0.F1.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="453" height="235" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 1: </span>Personas can work with a wide range of data synthesis prompts (e.g., create a math problem or a user prompt) to guide an LLM to synthesize data with corresponding perspectives. The 1 billion personas in Persona Hub can facilitate synthetic data creation for various data synthesis scenarios at a billion scale.</figcaption>
</figure>
<section id="S1" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">1 </span>Introduction</h2>

<div id="S1.p1" class="ltx_para ltx_noindent">
<p id="S1.p1.1" class="ltx_p">As synthetic data <cite class="ltx_cite ltx_citemacro_citep">(Bauer et al., <a href="#bib.bib5" title="" class="ltx_ref">2024</a>; Liu et al., <a href="#bib.bib25" title="" class="ltx_ref">2024</a>)</cite>, typically referring to data generated by models or algorithms rather than directly by humans, becomes increasingly valued <cite class="ltx_cite ltx_citemacro_citep">(Li et al., <a href="#bib.bib24" title="" class="ltx_ref">2023b</a>)</cite> for training large language models (LLMs), there is a growing interest in data synthesis using LLMs: by simply specifying a data synthesis prompt, an LLM is expected to produce desirable synthetic data.</p>
</div>
<div id="S1.p2" class="ltx_para ltx_noindent">
<p id="S1.p2.1" class="ltx_p">In practice, however, it is non-trivial to create synthetic data at scale: while we can easily scale up the quantity of synthetic data, it is difficult to ensure its diversity scales up as well. Without considering sampling<span id="footnote1" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">1</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">1</sup><span class="ltx_tag ltx_tag_note">1</span>Sampling is orthogonal to this work. The diversity it introduces when solely used for data synthesis is usually limited.</span></span></span>, an LLM can only produce 1 instance given a data synthesis prompt. Therefore, to create diverse synthetic data at scale (e.g., 1 billion diverse math problems), a large number of diverse prompts are needed.</p>
</div>
<div id="S1.p3" class="ltx_para ltx_noindent">
<p id="S1.p3.1" class="ltx_p">Previous research tends to diversify the data synthesis prompt through the following two paradigms, but unfortunately, neither can practically achieve scalable synthetic data creation:</p>
</div>
<div id="S1.p4" class="ltx_para ltx_noindent">
<ul id="S1.I1" class="ltx_itemize">
<li id="S1.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i1.p1" class="ltx_para">
<p id="S1.I1.i1.p1.1" class="ltx_p"><span id="S1.I1.i1.p1.1.1" class="ltx_text ltx_font_bold">Instance-driven</span>: This approach diversifies the data synthesis prompt by leveraging a seed corpus (i.e., creating new instances based on the instances in the seed corpus). Representative studies include <cite class="ltx_cite ltx_citemacro_citet">Wang et al. (<a href="#bib.bib37" title="" class="ltx_ref">2022</a>)</cite> and <cite class="ltx_cite ltx_citemacro_citet">Yu et al. (<a href="#bib.bib41" title="" class="ltx_ref">2023</a>)</cite>. However, under this paradigm, the diversity of the synthesized data mainly comes from the seed instances, making it difficult to truly extend beyond the seed corpus. Given the limited size of a seed corpus in most practical scenarios, it is challenging for this paradigm to scale up the creation of synthetic data.</p>
</div>
</li>
<li id="S1.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I1.i2.p1" class="ltx_para ltx_noindent">
<p id="S1.I1.i2.p1.1" class="ltx_p"><span id="S1.I1.i2.p1.1.1" class="ltx_text ltx_font_bold">Key-point-driven</span>: This approach diversifies the data synthesis prompt with a curated comprehensive list of key points (or concepts) that can be a topic, a subject, or any knowledge we expect synthetic data to encompass. Representative studies include <cite class="ltx_cite ltx_citemacro_citet">Li et al. (<a href="#bib.bib22" title="" class="ltx_ref">2024b</a>)</cite> and <cite class="ltx_cite ltx_citemacro_citet">Huang et al. (<a href="#bib.bib18" title="" class="ltx_ref">2024</a>)</cite>. However, this methodology also faces difficulties in scaling synthetic data creation: it is practically prohibitive to curate a comprehensive list by enumerating all key points across different levels of granularity, unless limited to a narrow and specific domain (e.g., mathematics).</p>
</div>
</li>
</ul>
</div>
<div id="S1.p5" class="ltx_para ltx_noindent">
<p id="S1.p5.1" class="ltx_p">To practically achieve diverse synthetic data creation at scale, we propose a novel <span id="S1.p5.1.1" class="ltx_text ltx_font_bold">persona-driven</span> data synthesis methodology. This is inspired by the observation that simply adding a persona to a data synthesis prompt can steer the LLM towards the corresponding perspective to create distinctive synthetic data, as shown in Figure <a href="#S0.F1" title="Figure 1 ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>. Since almost any LLM use case can be associated with a specific persona, we can create all-encompassing synthetic data at scale as long as we construct a comprehensive persona collection.</p>
</div>
<figure id="S1.F2" class="ltx_figure"><img src="/html/2406.20094/assets/x2.png" id="S1.F2.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="453" height="155" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 2: </span>From a compression perspective <cite class="ltx_cite ltx_citemacro_citep">(Delétang et al., <a href="#bib.bib13" title="" class="ltx_ref">2023</a>; Ge et al., <a href="#bib.bib16" title="" class="ltx_ref">2024</a>)</cite>, Persona Hub (<math id="S1.F2.3.m1.1" class="ltx_Math" alttext="\sim 10^{10}" display="inline"><semantics id="S1.F2.3.m1.1b"><mrow id="S1.F2.3.m1.1.1" xref="S1.F2.3.m1.1.1.cmml"><mi id="S1.F2.3.m1.1.1.2" xref="S1.F2.3.m1.1.1.2.cmml"></mi><mo id="S1.F2.3.m1.1.1.1" xref="S1.F2.3.m1.1.1.1.cmml">∼</mo><msup id="S1.F2.3.m1.1.1.3" xref="S1.F2.3.m1.1.1.3.cmml"><mn id="S1.F2.3.m1.1.1.3.2" xref="S1.F2.3.m1.1.1.3.2.cmml">10</mn><mn id="S1.F2.3.m1.1.1.3.3" xref="S1.F2.3.m1.1.1.3.3.cmml">10</mn></msup></mrow><annotation-xml encoding="MathML-Content" id="S1.F2.3.m1.1c"><apply id="S1.F2.3.m1.1.1.cmml" xref="S1.F2.3.m1.1.1"><csymbol cd="latexml" id="S1.F2.3.m1.1.1.1.cmml" xref="S1.F2.3.m1.1.1.1">similar-to</csymbol><csymbol cd="latexml" id="S1.F2.3.m1.1.1.2.cmml" xref="S1.F2.3.m1.1.1.2">absent</csymbol><apply id="S1.F2.3.m1.1.1.3.cmml" xref="S1.F2.3.m1.1.1.3"><csymbol cd="ambiguous" id="S1.F2.3.m1.1.1.3.1.cmml" xref="S1.F2.3.m1.1.1.3">superscript</csymbol><cn type="integer" id="S1.F2.3.m1.1.1.3.2.cmml" xref="S1.F2.3.m1.1.1.3.2">10</cn><cn type="integer" id="S1.F2.3.m1.1.1.3.3.cmml" xref="S1.F2.3.m1.1.1.3.3">10</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F2.3.m1.1d">\sim 10^{10}</annotation></semantics></math> tokens) can be seen as the compressed form of world knowledge (public web text for training LLMs, <math id="S1.F2.4.m2.1" class="ltx_Math" alttext="\sim 10^{14}" display="inline"><semantics id="S1.F2.4.m2.1b"><mrow id="S1.F2.4.m2.1.1" xref="S1.F2.4.m2.1.1.cmml"><mi id="S1.F2.4.m2.1.1.2" xref="S1.F2.4.m2.1.1.2.cmml"></mi><mo id="S1.F2.4.m2.1.1.1" xref="S1.F2.4.m2.1.1.1.cmml">∼</mo><msup id="S1.F2.4.m2.1.1.3" xref="S1.F2.4.m2.1.1.3.cmml"><mn id="S1.F2.4.m2.1.1.3.2" xref="S1.F2.4.m2.1.1.3.2.cmml">10</mn><mn id="S1.F2.4.m2.1.1.3.3" xref="S1.F2.4.m2.1.1.3.3.cmml">14</mn></msup></mrow><annotation-xml encoding="MathML-Content" id="S1.F2.4.m2.1c"><apply id="S1.F2.4.m2.1.1.cmml" xref="S1.F2.4.m2.1.1"><csymbol cd="latexml" id="S1.F2.4.m2.1.1.1.cmml" xref="S1.F2.4.m2.1.1.1">similar-to</csymbol><csymbol cd="latexml" id="S1.F2.4.m2.1.1.2.cmml" xref="S1.F2.4.m2.1.1.2">absent</csymbol><apply id="S1.F2.4.m2.1.1.3.cmml" xref="S1.F2.4.m2.1.1.3"><csymbol cd="ambiguous" id="S1.F2.4.m2.1.1.3.1.cmml" xref="S1.F2.4.m2.1.1.3">superscript</csymbol><cn type="integer" id="S1.F2.4.m2.1.1.3.2.cmml" xref="S1.F2.4.m2.1.1.3.2">10</cn><cn type="integer" id="S1.F2.4.m2.1.1.3.3.cmml" xref="S1.F2.4.m2.1.1.3.3">14</cn></apply></apply></annotation-xml><annotation encoding="application/x-tex" id="S1.F2.4.m2.1d">\sim 10^{14}</annotation></semantics></math> tokens) into distributed carriers. On the other hand, the public web text can be seen as the decompressed content created by these personas with their knowledge and experiences.</figcaption>
</figure>
<div id="S1.p6" class="ltx_para ltx_noindent">
<p id="S1.p6.1" class="ltx_p">Fortunately, personas are very easy to scale up. From massive web data, we automatically construct Persona Hub — a persona collection containing 1 billion diverse personas (<math id="S1.p6.1.m1.1" class="ltx_Math" alttext="\sim" display="inline"><semantics id="S1.p6.1.m1.1a"><mo id="S1.p6.1.m1.1.1" xref="S1.p6.1.m1.1.1.cmml">∼</mo><annotation-xml encoding="MathML-Content" id="S1.p6.1.m1.1b"><csymbol cd="latexml" id="S1.p6.1.m1.1.1.cmml" xref="S1.p6.1.m1.1.1">similar-to</csymbol></annotation-xml><annotation encoding="application/x-tex" id="S1.p6.1.m1.1c">\sim</annotation></semantics></math>13% of the world’s total population). As Figure <a href="#S1.F2" title="Figure 2 ‣ 1 Introduction ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> shows, these 1 billion personas can be regarded as distributed carriers of world knowledge, and each individual can be associated with their unique knowledge, experience, interest, personality and profession; thus, they can tap into almost every perspective encapsulated within the LLM to create diverse synthetic data at scale, without being limited by the size of a seed corpus. Moreover, in contrast to key points that typically work with specific data synthesis prompts, personas can be combined with almost any data synthesis prompt, benefiting from an LLM’s strong roleplay ability <cite class="ltx_cite ltx_citemacro_citep">(Shanahan et al., <a href="#bib.bib31" title="" class="ltx_ref">2023</a>; Li et al., <a href="#bib.bib23" title="" class="ltx_ref">2023a</a>; Choi &amp; Li, <a href="#bib.bib11" title="" class="ltx_ref">2024</a>; Wang et al., <a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite>, making them generally applicable to a variety of data synthesis scenarios.</p>
</div>
<div id="S1.p7" class="ltx_para ltx_noindent">
<p id="S1.p7.1" class="ltx_p">We showcase Persona Hub’s use cases in large-scale creation of math and logical reasoning problems, instructions (i.e., user prompts), broad-coverage knowledge-rich texts, game NPCs, and tool (function) development. We demonstrate that persona-driven data synthesis is versatile, scalable, flexible, and easy to use, potentially driving a paradigm shift in synthetic data creation and applications in practice, which may have a profound impact on
LLM research and development.</p>
</div>
<div id="S1.p8" class="ltx_para ltx_noindent">
<p id="S1.p8.1" class="ltx_p">To facilitate research in persona-driven data synthesis, we initially release <span id="S1.p8.1.1" class="ltx_text ltx_font_bold">200,000 personas</span> from Persona Hub and following synthetic data samples we created with various personas, including:</p>
</div>
<div class="ltx_pagination ltx_role_start_2_columns"></div>
<div id="S1.p9" class="ltx_para ltx_noindent">
<ul id="S1.I2" class="ltx_itemize">
<li id="S1.I2.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I2.i1.p1" class="ltx_para">
<p id="S1.I2.i1.p1.1" class="ltx_p"><span id="S1.I2.i1.p1.1.1" class="ltx_text ltx_font_bold">50,000 math problems</span></p>
</div>
</li>
<li id="S1.I2.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I2.i2.p1" class="ltx_para">
<p id="S1.I2.i2.p1.1" class="ltx_p"><span id="S1.I2.i2.p1.1.1" class="ltx_text ltx_font_bold">50,000 instructions</span></p>
</div>
</li>
<li id="S1.I2.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I2.i3.p1" class="ltx_para">
<p id="S1.I2.i3.p1.1" class="ltx_p"><span id="S1.I2.i3.p1.1.1" class="ltx_text ltx_font_bold">10,000 game NPCs</span></p>
</div>
</li>
<li id="S1.I2.i4" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I2.i4.p1" class="ltx_para">
<p id="S1.I2.i4.p1.1" class="ltx_p"><span id="S1.I2.i4.p1.1.1" class="ltx_text ltx_font_bold">50,000 logical reasoning problems</span></p>
</div>
</li>
<li id="S1.I2.i5" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I2.i5.p1" class="ltx_para">
<p id="S1.I2.i5.p1.1" class="ltx_p"><span id="S1.I2.i5.p1.1.1" class="ltx_text ltx_font_bold">10,000 knowledge-rich texts</span></p>
</div>
</li>
<li id="S1.I2.i6" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S1.I2.i6.p1" class="ltx_para ltx_noindent">
<p id="S1.I2.i6.p1.1" class="ltx_p"><span id="S1.I2.i6.p1.1.1" class="ltx_text ltx_font_bold">5,000 tools (functions)</span></p>
</div>
</li>
</ul>
<div class="ltx_pagination ltx_role_end_2_columns"></div>
</div>
<div id="S1.p10" class="ltx_para ltx_noindent">
<p id="S1.p10.1" class="ltx_p">We are open to releasing more data when we can better assess the potential risks and concerns, which will be discussed in detail in Section <a href="#S5" title="5 Broad Impact and Ethical Concerns ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>.</p>
</div>
<div id="S1.p11" class="ltx_para ltx_noindent">
<p id="S1.p11.1" class="ltx_p"><span id="S1.p11.1.1" class="ltx_text ltx_font_bold" style="color:#FF0000;">Note:<span id="S1.p11.1.1.1" class="ltx_text ltx_font_medium"> Our proposed methodology is applicable to almost any popular LLM<span id="footnote2" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">2</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">2</sup><span class="ltx_tag ltx_tag_note"><span id="footnote2.1.1.1" class="ltx_text" style="color:#000000;">2</span></span><span id="footnote2.5" class="ltx_text" style="color:#000000;">We mainly use publicly available LLMs such as GPT-4 </span><cite class="ltx_cite ltx_citemacro_citep"><span id="footnote2.6.1" class="ltx_text" style="color:#000000;">(</span>Achiam et al.<span id="footnote2.7.2.1.1" class="ltx_text" style="color:#000000;">, </span><a href="#bib.bib3" title="" class="ltx_ref">2023</a><span id="footnote2.8.3" class="ltx_text" style="color:#000000;">)</span></cite><span id="footnote2.9" class="ltx_text" style="color:#000000;">, Llama-3 and Qwen </span><cite class="ltx_cite ltx_citemacro_citep"><span id="footnote2.10.1" class="ltx_text" style="color:#000000;">(</span>Team<span id="footnote2.11.2.1.1" class="ltx_text" style="color:#000000;">, </span><a href="#bib.bib33" title="" class="ltx_ref">2024</a>; qwe<span id="footnote2.11.2.1.1" class="ltx_text" style="color:#000000;">, </span><a href="#bib.bib1" title="" class="ltx_ref">2024</a><span id="footnote2.12.3" class="ltx_text" style="color:#000000;">)</span></cite><span id="footnote2.13" class="ltx_text" style="color:#000000;"> in our experiments.</span></span></span></span>. The prompts shown in the figures throughout this paper are not exactly the prompt strings we used in our experiments; instead, they are simplified to fit the space and better illustrate the concepts. Interested readers can easily verify our methodology using the persona samples we have released. It is also worth noting that the main focus of this work is on creating new synthetic data, unlike much previous research that focuses on generating synthetic outputs for specific inputs (e.g., a math problem). Therefore, we use the terms “create” and “synthesize” interchangeably throughout the paper.</span></span></p>
</div>
</section>
<section id="S2" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">2 </span>Persona Hub</h2>

<div id="S2.p1" class="ltx_para ltx_noindent">
<p id="S2.p1.1" class="ltx_p">We propose two scalable approaches to derive diverse personas to construct Persona Hub from massive web data: <span id="S2.p1.1.1" class="ltx_text ltx_font_italic">Text-to-Persona</span> and <span id="S2.p1.1.2" class="ltx_text ltx_font_italic">Persona-to-Persona</span>.</p>
</div>
<figure id="S2.F3" class="ltx_figure"><img src="/html/2406.20094/assets/x3.png" id="S2.F3.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="453" height="167" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 3: </span>The <span id="S2.F3.3.1" class="ltx_text ltx_font_italic">Text-to-Persona</span> approach: it can use any text as input to obtain corresponding personas just by prompting the LLM “Who is likely to [<span id="S2.F3.4.2" class="ltx_text ltx_font_typewriter">read|write|like|dislike|…</span>] the text?”</figcaption>
</figure>
<section id="S2.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.1 </span>Text-to-Persona</h3>

<div id="S2.SS1.p1" class="ltx_para ltx_noindent">
<p id="S2.SS1.p1.1" class="ltx_p">A person with specific professional experiences and cultural backgrounds will have unique interests in reading and writing. Therefore, from a specific text, we can infer a specific persona who is likely to [<span id="S2.SS1.p1.1.1" class="ltx_text ltx_font_typewriter">read|write|like|dislike|…</span>] the text. Given that text data on the web is virtually unlimited and all-encompassing, we can obtain a wide-ranging collection of personas simply by prompting an LLM with these web texts, as shown in Figure <a href="#S2.F3" title="Figure 3 ‣ 2 Persona Hub ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>.</p>
</div>
<div id="S2.SS1.p2" class="ltx_para ltx_noindent">
<p id="S2.SS1.p2.1" class="ltx_p">There are many formats (e.g., plain text or structured text) to represent a persona, which can be controlled within the prompt. The granularity of an output persona description can also be adjusted through the prompt. For example, in the first case, a coarse-grained persona might be “<span id="S2.SS1.p2.1.1" class="ltx_text ltx_font_italic">a computer scientist</span>”, whereas the fine-grained persona is “<span id="S2.SS1.p2.1.2" class="ltx_text ltx_font_italic">a machine learning researcher focused on neural network architectures and attention mechanisms</span>”. In our practice, we ask the LLM (in the prompt) to output persona descriptions as specifically as possible. Besides specifying the granularity of persona descriptions in the prompt, input texts can also influence the granularity of persona descriptions. As shown in Figure <a href="#S2.F4" title="Figure 4 ‣ 2.1 Text-to-Persona ‣ 2 Persona Hub ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>, if an input text (e.g., from a mathematical textbook or an academic paper about superconductivity) contains many detailed elements, the resulting persona description will also be specific and fine-grained. Therefore, by applying the <span id="S2.SS1.p2.1.3" class="ltx_text ltx_font_italic">Text-to-Persona</span> approach to massive web text data, we can obtain billions (or even trillions) of diverse personas, encompassing a wide range of aspects across different granularities.</p>
</div>
<figure id="S2.F4" class="ltx_figure"><img src="/html/2406.20094/assets/x4.png" id="S2.F4.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="453" height="198" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 4: </span>Persona descriptions will be fine-grained if input texts involve many detailed elements.</figcaption>
</figure>
</section>
<section id="S2.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.2 </span>Persona-to-Persona</h3>

<div id="S2.SS2.p1" class="ltx_para ltx_noindent">
<p id="S2.SS2.p1.1" class="ltx_p">As discussed above, <span id="S2.SS2.p1.1.1" class="ltx_text ltx_font_italic">Text-to-Persona</span> is a highly scalable method that can synthesize personas covering almost every aspect. However, it may still miss some personas that have low visibility on the web and thus are less likely to obtain via <span id="S2.SS2.p1.1.2" class="ltx_text ltx_font_italic">Text-to-Persona</span>, such as <span id="S2.SS2.p1.1.3" class="ltx_text ltx_font_italic">a child</span>, <span id="S2.SS2.p1.1.4" class="ltx_text ltx_font_italic">a beggar</span>, or <span id="S2.SS2.p1.1.5" class="ltx_text ltx_font_italic">a behind-the-scenes crew member of a movie</span>. To supplement the personas that <span id="S2.SS2.p1.1.6" class="ltx_text ltx_font_italic">Text-to-Persona</span> might hardly reach, we propose <span id="S2.SS2.p1.1.7" class="ltx_text ltx_font_italic">Persona-to-Persona</span>, which derives personas with interpersonal relationships from those obtained through <span id="S2.SS2.p1.1.8" class="ltx_text ltx_font_italic">Text-to-Persona</span>.</p>
</div>
<figure id="S2.F5" class="ltx_figure"><img src="/html/2406.20094/assets/x5.png" id="S2.F5.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="393" height="148" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 5: </span><span id="S2.F5.2.1" class="ltx_text ltx_font_italic">Persona-to-Persona</span> obtains diverse personas via interpersonal relationships, which can be easily achieved by prompting the LLM “Who is in close relationship with the given persona?”</figcaption>
</figure>
<div id="S2.SS2.p2" class="ltx_para ltx_noindent">
<p id="S2.SS2.p2.1" class="ltx_p">As shown in Figure <a href="#S2.F5" title="Figure 5 ‣ 2.2 Persona-to-Persona ‣ 2 Persona Hub ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5</span></a>, the persona about “<span id="S2.SS2.p2.1.1" class="ltx_text ltx_font_italic">a child</span>” can be derived from the persona of a nurse at a children’s hospital (patient-caregiver relationship). Similarly, “<span id="S2.SS2.p2.1.2" class="ltx_text ltx_font_italic">a beggar</span>” can be derived from the persona of a shelter worker (assistance relationship), and “<span id="S2.SS2.p2.1.3" class="ltx_text ltx_font_italic">a behind-the-scenes movie crew member</span>” can be derived from the persona of the movie’s lead actor (co-worker relationship). According to the six degrees of separation theory <cite class="ltx_cite ltx_citemacro_citep">(Travers &amp; Milgram, <a href="#bib.bib34" title="" class="ltx_ref">1977</a>)</cite>, we perform six iterations of persona relationship expansion for each persona obtained through <span id="S2.SS2.p2.1.4" class="ltx_text ltx_font_italic">Text-to-Persona</span>, thereby enriching our persona collection even further.</p>
</div>
</section>
<section id="S2.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">2.3 </span>Deduplication</h3>

<div id="S2.SS3.p1" class="ltx_para ltx_noindent">
<p id="S2.SS3.p1.1" class="ltx_p">We first run <span id="S2.SS3.p1.1.1" class="ltx_text ltx_font_italic">Text-to-Persona</span> on the RedPajama v2 dataset <cite class="ltx_cite ltx_citemacro_citep">(Computer, <a href="#bib.bib12" title="" class="ltx_ref">2023</a>)</cite> and then perform <span id="S2.SS3.p1.1.2" class="ltx_text ltx_font_italic">Persona-to-Persona</span>, as described in Sections <a href="#S2.SS1" title="2.1 Text-to-Persona ‣ 2 Persona Hub ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.1</span></a> and <a href="#S2.SS2" title="2.2 Persona-to-Persona ‣ 2 Persona Hub ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.2</span></a>. After obtaining billions of personas, it is inevitable that some of the personas will be identical or extremely similar. To ensure the diversity of Persona Hub, we deduplicate these personas in two ways:</p>
</div>
<section id="S2.SS3.SSS0.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">MinHash-based Deduplication</h5>

<div id="S2.SS3.SSS0.Px1.p1" class="ltx_para ltx_noindent">
<p id="S2.SS3.SSS0.Px1.p1.1" class="ltx_p">We use MinHash <cite class="ltx_cite ltx_citemacro_citep">(Broder, <a href="#bib.bib7" title="" class="ltx_ref">1997</a>)</cite> to deduplicate based on the n-gram features of persona descriptions. Since persona descriptions are usually just 1-2 sentences, much shorter than a document, we simply used 1-gram and a signature size of 128 for MinHash deduplication. We deduplicate at the similarity threshold of 0.9.</p>
</div>
</section>
<section id="S2.SS3.SSS0.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Embedding-based Deduplication</h5>

<div id="S2.SS3.SSS0.Px2.p1" class="ltx_para ltx_noindent">
<p id="S2.SS3.SSS0.Px2.p1.1" class="ltx_p">After deduplication based on surface forms (i.e., MinHash with n-gram features), we also adopt embedding-based deduplication. We use a text embedding model (e.g., the <span id="S2.SS3.SSS0.Px2.p1.1.1" class="ltx_text ltx_font_typewriter">text-embedding-3-small</span> model from OpenAI) to compute an embedding for each persona, and then filter out personas with a cosine semantic similarity greater than 0.9.</p>
</div>
<div id="S2.SS3.SSS0.Px2.p2" class="ltx_para ltx_noindent">
<p id="S2.SS3.SSS0.Px2.p2.1" class="ltx_p">Note that although we select 0.9 as the threshold here, we can flexibly adjust it according to specific needs for further deduplication. For instance, when the requirement for the number of instances is not high (e.g., only needing 1 million instances) but the demand for diversity is high, we can further apply a stricter deduplication standard (e.g., discarding personas with a similarity greater than 0.5).</p>
</div>
<div id="S2.SS3.SSS0.Px2.p3" class="ltx_para ltx_noindent">
<p id="S2.SS3.SSS0.Px2.p3.1" class="ltx_p">After deduplication and using simple heuristic methods to filter out low-quality persona descriptions, we have harvested a total of 1,015,863,523 personas, finally forming our Persona Hub.</p>
</div>
</section>
</section>
</section>
<section id="S3" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">3 </span>Persona-driven Synthetic Data Creation</h2>

<div id="S3.p1" class="ltx_para ltx_noindent">
<p id="S3.p1.1" class="ltx_p">Our proposed persona-driven data synthesis approach is straightforward and effective, which involves integrating a persona into the appropriate position in a data synthesis prompt. Simple as it appears, it can significantly influence the LLM to adopt the persona’s perspective to create synthetic data. Driven by the 1 billion personas in Persona Hub, this approach can easily create diverse synthetic data at a billion scale.</p>
</div>
<figure id="S3.F6" class="ltx_figure"><img src="/html/2406.20094/assets/x6.png" id="S3.F6.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="453" height="255" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 6: </span>0-shot, few-shot and persona-enhanced few-shot prompting methods.</figcaption>
</figure>
<div id="S3.p2" class="ltx_para ltx_noindent">
<p id="S3.p2.1" class="ltx_p">Just as we can use either zero-shot or few-shot methods to prompt an LLM, the persona-driven methodology is also flexible and compatible with various forms of prompts to create synthetic data. As shown in Figure <a href="#S3.F6" title="Figure 6 ‣ 3 Persona-driven Synthetic Data Creation ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">6</span></a>, we propose three persona-driven data synthesis prompting methods:</p>
</div>
<div id="S3.p3" class="ltx_para ltx_noindent">
<ul id="S3.I1" class="ltx_itemize">
<li id="S3.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i1.p1" class="ltx_para">
<p id="S3.I1.i1.p1.1" class="ltx_p"><span id="S3.I1.i1.p1.1.1" class="ltx_text ltx_font_bold">Zero-shot prompting</span> does not leverage any existing examples (i.e., demonstrations), thereby fully exploiting the model’s creativity without being constrained by specific examples.</p>
</div>
</li>
<li id="S3.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i2.p1" class="ltx_para">
<p id="S3.I1.i2.p1.1" class="ltx_p"><span id="S3.I1.i2.p1.1.1" class="ltx_text ltx_font_bold">Few-shot prompting</span> can better ensure that the synthesized data meets the requirements by providing some demonstrations.</p>
</div>
</li>
<li id="S3.I1.i3" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S3.I1.i3.p1" class="ltx_para ltx_noindent">
<p id="S3.I1.i3.p1.1" class="ltx_p"><span id="S3.I1.i3.p1.1.1" class="ltx_text ltx_font_bold">Persona-enhanced few-shot prompting</span> is more effective in enhancing the LLM’s persona-driven data synthesis capabilities. However, its drawback is that it requires deriving the corresponding persona for each demonstration in the few-shot prompt beforehand.</p>
</div>
</li>
</ul>
</div>
</section>
<section id="S4" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">4 </span>Use Cases</h2>

<div id="S4.p1" class="ltx_para ltx_noindent">
<p id="S4.p1.1" class="ltx_p">We demonstrate the use cases of Persona Hub in various data synthesis scenarios, including the large-scale creation of math and logical reasoning problems, instructions (i.e., user prompts), knowledge-rich texts, game NPCs, and tool (function) development.</p>
</div>
<div id="S4.p2" class="ltx_para ltx_noindent">
<p id="S4.p2.1" class="ltx_p">As mentioned earlier, the persona-driven approach is general and versatile, making it easily adaptable to different data synthesis scenarios simply by adjusting the data synthesis prompt. Therefore, we will provide a detailed technical discussion only for math problem synthesis (Section <a href="#S4.SS1" title="4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.1</span></a>) and skip the detailed discussion for other use cases.</p>
</div>
<section id="S4.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.1 </span>Math Problems</h3>

<section id="S4.SS1.SSS1" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">4.1.1 </span>Demonstrations</h4>

<div id="S4.SS1.SSS1.p1" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS1.p1.1" class="ltx_p">As the initial example (Figure <a href="#S0.F1" title="Figure 1 ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a>) shows, when prompting an LLM to create a math problem, adding a persona leads the LLM to create math problems related to that persona. The example in Figure <a href="#S4.F7" title="Figure 7 ‣ 4.1.1 Demonstrations ‣ 4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a>(left) further confirms this: when presented with a linguist persona, the LLM will create a math problem in the context of computational linguistics. Moreover, adding a persona does not hinder the flexibility of the prompt – we can still easily specify the focus (Figure <a href="#S4.F7" title="Figure 7 ‣ 4.1.1 Demonstrations ‣ 4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a>(middle)) or difficulty (Figure <a href="#S4.F7" title="Figure 7 ‣ 4.1.1 Demonstrations ‣ 4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a>(right)) of our desired math problem in the prompt.</p>
</div>
<figure id="S4.F7" class="ltx_figure"><img src="/html/2406.20094/assets/x7.png" id="S4.F7.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="453" height="223" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 7: </span>A linguist persona with different math problem creation prompts that specify the focus (e.g., geometry) or the difficulty (e.g., Olympiad-level)</figcaption>
</figure>
<figure id="S4.F8" class="ltx_figure"><img src="/html/2406.20094/assets/x8.png" id="S4.F8.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="453" height="190" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 8: </span>Examples of math problems created with personas of professionals related to the field of mathematics. They tend to be more challenging than those created with general personas because they usually require a deeper and more fine-grained understanding of advanced mathematical knowledge and skills.</figcaption>
</figure>
<div id="S4.SS1.SSS1.p2" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS1.p2.1" class="ltx_p">The examples in Figure <a href="#S4.F7" title="Figure 7 ‣ 4.1.1 Demonstrations ‣ 4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">7</span></a> demonstrate the use of general personas to create math problems. We can certainly employ professionals related to mathematics to create math problems as well. As shown in Figure <a href="#S4.F8" title="Figure 8 ‣ 4.1.1 Demonstrations ‣ 4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">8</span></a>, personas of math professionals<span id="footnote3" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">3</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">3</sup><span class="ltx_tag ltx_tag_note">3</span>We can easily harvest a large number of such personas when running <span id="footnote3.1" class="ltx_text ltx_font_italic">Text-to-Persona</span> (Section <a href="#S2.SS1" title="2.1 Text-to-Persona ‣ 2 Persona Hub ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.1</span></a>) on public web texts, particularly when processing texts in the field of mathematics.</span></span></span> often mention more advanced and granular mathematics knowledge and skills (as discussed earlier in Section <a href="#S2.SS1" title="2.1 Text-to-Persona ‣ 2 Persona Hub ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.1</span></a> and Figure <a href="#S2.F4" title="Figure 4 ‣ 2.1 Text-to-Persona ‣ 2 Persona Hub ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4</span></a>), which in turn allows the created math problems to cover these mathematical concepts, making them more challenging.</p>
</div>
</section>
<section id="S4.SS1.SSS2" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">4.1.2 </span>Evaluation</h4>

<section id="S4.SS1.SSS2.Px1" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Data</h5>

<div id="S4.SS1.SSS2.Px1.p1" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS2.Px1.p1.1" class="ltx_p">We select<span id="footnote4" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">4</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">4</sup><span class="ltx_tag ltx_tag_note">4</span>Technically, if we use the full version of Persona Hub, we can obtain 1 billion math problems synthesized by an LLM. However, due to the cost of GPT-4 APIs, we limit our scaling to 1.09M personas for this experiment.</span></span></span> 1.09 million personas from Persona Hub and employ the 0-shot prompting method using GPT-4 to create math problems with these personas, which does not leverage any instances from benchmarks like MATH <cite class="ltx_cite ltx_citemacro_citep">(Hendrycks et al., <a href="#bib.bib17" title="" class="ltx_ref">2021</a>)</cite> during the creation of math problems. This approach allowed us to synthesize 1.09M math problems. Since this work focuses on creating new synthetic data rather than synthesizing solutions, we simply used <span id="S4.SS1.SSS2.Px1.p1.1.1" class="ltx_text ltx_font_typewriter">gpt-4o</span> (assistant<span id="footnote5" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">5</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">5</sup><span class="ltx_tag ltx_tag_note">5</span>Assistant system message in OpenAI API doc: “You are a helpful assistant.”</span></span></span>) to generate solutions to the created problems. Among these 1.09M math problems, we randomly hold out 20k as a synthetic test set to facilitate evaluation. The remaining 1.07M problems are used for training.</p>
</div>
</section>
<section id="S4.SS1.SSS2.Px2" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Test sets</h5>

<div id="S4.SS1.SSS2.Px2.p1" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS2.Px2.p1.1" class="ltx_p">We use the following two test sets for evaluation:</p>
</div>
<div id="S4.SS1.SSS2.Px2.p2" class="ltx_para ltx_noindent">
<ul id="S4.I1" class="ltx_itemize">
<li id="S4.I1.i1" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S4.I1.i1.p1" class="ltx_para ltx_noindent">
<p id="S4.I1.i1.p1.1" class="ltx_p"><span id="S4.I1.i1.p1.1.1" class="ltx_text ltx_font_bold">Synthetic Test Set</span> (In-distribution): Since the set of the held-out 20K problems is produced in the same way as the 1.07M training instances, it can be considered an in-distribution test set. To ensure the accuracy of the answers in this test set for increasing the reliability of the evaluation, we additionally generate solutions using <span id="S4.I1.i1.p1.1.2" class="ltx_text ltx_font_typewriter">gpt-4o</span> (PoT<span id="footnote6" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">6</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">6</sup><span class="ltx_tag ltx_tag_note">6</span>Program of thought prompting <cite class="ltx_cite ltx_citemacro_citep">(Chen et al., <a href="#bib.bib10" title="" class="ltx_ref">2022</a>)</cite></span></span></span>) and <span id="S4.I1.i1.p1.1.3" class="ltx_text ltx_font_typewriter">gpt-4-turbo</span> (assistant) in addition to the solution generated by <span id="S4.I1.i1.p1.1.4" class="ltx_text ltx_font_typewriter">gpt-4o</span> (assistant). We retain only the test instances where at least two solutions are consistent. The remaining test set consists of 11.6K test instances.</p>
</div>
</li>
<li id="S4.I1.i2" class="ltx_item" style="list-style-type:none;">
<span class="ltx_tag ltx_tag_item">•</span> 
<div id="S4.I1.i2.p1" class="ltx_para ltx_noindent">
<p id="S4.I1.i2.p1.1" class="ltx_p"><span id="S4.I1.i2.p1.1.1" class="ltx_text ltx_font_bold">MATH</span> (Out-of-distribution): The most widely recognized benchmark for testing the mathematical reasoning ability of LLMs. Its test set contains 5,000 competitive-level math problems with reference answers. Since we do not use any instances from the MATH dataset for data synthesis or training, we regard the MATH test set as an out-of-distribution test set.</p>
</div>
</li>
</ul>
</div>
</section>
<section id="S4.SS1.SSS2.Px3" class="ltx_paragraph">
<h5 class="ltx_title ltx_title_paragraph">Equality Checking</h5>

<div id="S4.SS1.SSS2.Px3.p1" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS2.Px3.p1.1" class="ltx_p">We follow the same evaluation protocol as OpenAI<span id="footnote7" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">7</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">7</sup><span class="ltx_tag ltx_tag_note">7</span><a target="_blank" href="https://github.com/openai/simple-evals" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/openai/simple-evals</a></span></span></span> to check answer equality on the MATH benchmark. For the synthetic test set, we use a similar method, except we use Llama-3-70B-Instruct instead of <span id="S4.SS1.SSS2.Px3.p1.1.1" class="ltx_text ltx_font_typewriter">gpt-4-turbo-preview</span> as the equality checker.</p>
</div>
<div id="S4.SS1.SSS2.Px3.p2" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS2.Px3.p2.1" class="ltx_p">We simply fine-tune the latest open-sourced 7B LLM – Qwen2-7B <cite class="ltx_cite ltx_citemacro_citep">(qwe, <a href="#bib.bib1" title="" class="ltx_ref">2024</a>)</cite> with our synthesized 1.07 million math problems and evaluate its greedy decoding outputs on the above two test sets.</p>
</div>
<figure id="S4.T1" class="ltx_table">
<table id="S4.T1.1" class="ltx_tabular ltx_centering ltx_guessed_headers ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S4.T1.1.1.1" class="ltx_tr">
<th id="S4.T1.1.1.1.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_tt"><span id="S4.T1.1.1.1.1.1" class="ltx_text ltx_font_bold">Model</span></th>
<td id="S4.T1.1.1.1.2" class="ltx_td ltx_align_center ltx_border_tt"><span id="S4.T1.1.1.1.2.1" class="ltx_text ltx_font_bold">Model Size</span></td>
<td id="S4.T1.1.1.1.3" class="ltx_td ltx_align_center ltx_border_tt"><span id="S4.T1.1.1.1.3.1" class="ltx_text ltx_font_bold">Accuracy (%)</span></td>
</tr>
<tr id="S4.T1.1.2.2" class="ltx_tr">
<th id="S4.T1.1.2.2.1" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_t" colspan="3"><span id="S4.T1.1.2.2.1.1" class="ltx_text ltx_font_bold">Open-sourced LLMs</span></th>
</tr>
<tr id="S4.T1.1.3.3" class="ltx_tr">
<th id="S4.T1.1.3.3.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t">DeepSeek LLM 67B Chat <cite class="ltx_cite ltx_citemacro_citep">(Bi et al., <a href="#bib.bib6" title="" class="ltx_ref">2024</a>)</cite>
</th>
<td id="S4.T1.1.3.3.2" class="ltx_td ltx_align_center ltx_border_t">67B</td>
<td id="S4.T1.1.3.3.3" class="ltx_td ltx_align_center ltx_border_t">53.2</td>
</tr>
<tr id="S4.T1.1.4.4" class="ltx_tr">
<th id="S4.T1.1.4.4.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Phi-3-Mini-4K-Instruct <cite class="ltx_cite ltx_citemacro_citep">(Abdin et al., <a href="#bib.bib2" title="" class="ltx_ref">2024</a>)</cite>
</th>
<td id="S4.T1.1.4.4.2" class="ltx_td ltx_align_center">3.8B</td>
<td id="S4.T1.1.4.4.3" class="ltx_td ltx_align_center">68.3</td>
</tr>
<tr id="S4.T1.1.5.5" class="ltx_tr">
<th id="S4.T1.1.5.5.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Yi-1.5-34B-Chat <cite class="ltx_cite ltx_citemacro_citep">(Young et al., <a href="#bib.bib40" title="" class="ltx_ref">2024</a>)</cite>
</th>
<td id="S4.T1.1.5.5.2" class="ltx_td ltx_align_center">34B</td>
<td id="S4.T1.1.5.5.3" class="ltx_td ltx_align_center">70.4</td>
</tr>
<tr id="S4.T1.1.6.6" class="ltx_tr">
<th id="S4.T1.1.6.6.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Qwen1.5-72B-Chat <cite class="ltx_cite ltx_citemacro_citep">(Team, <a href="#bib.bib33" title="" class="ltx_ref">2024</a>)</cite>
</th>
<td id="S4.T1.1.6.6.2" class="ltx_td ltx_align_center">72B</td>
<td id="S4.T1.1.6.6.3" class="ltx_td ltx_align_center">60.7</td>
</tr>
<tr id="S4.T1.1.7.7" class="ltx_tr">
<th id="S4.T1.1.7.7.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Qwen1.5-110B-Chat <cite class="ltx_cite ltx_citemacro_citep">(Team, <a href="#bib.bib33" title="" class="ltx_ref">2024</a>)</cite>
</th>
<td id="S4.T1.1.7.7.2" class="ltx_td ltx_align_center">110B</td>
<td id="S4.T1.1.7.7.3" class="ltx_td ltx_align_center">73.0</td>
</tr>
<tr id="S4.T1.1.8.8" class="ltx_tr">
<th id="S4.T1.1.8.8.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Qwen2-7B-Instruct <cite class="ltx_cite ltx_citemacro_citep">(qwe, <a href="#bib.bib1" title="" class="ltx_ref">2024</a>)</cite>
</th>
<td id="S4.T1.1.8.8.2" class="ltx_td ltx_align_center">7B</td>
<td id="S4.T1.1.8.8.3" class="ltx_td ltx_align_center">72.1</td>
</tr>
<tr id="S4.T1.1.9.9" class="ltx_tr">
<th id="S4.T1.1.9.9.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Qwen2-72B-Instruct <cite class="ltx_cite ltx_citemacro_citep">(qwe, <a href="#bib.bib1" title="" class="ltx_ref">2024</a>)</cite>
</th>
<td id="S4.T1.1.9.9.2" class="ltx_td ltx_align_center">72B</td>
<td id="S4.T1.1.9.9.3" class="ltx_td ltx_align_center">77.2</td>
</tr>
<tr id="S4.T1.1.10.10" class="ltx_tr">
<th id="S4.T1.1.10.10.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Llama-3-8B-Instruct</th>
<td id="S4.T1.1.10.10.2" class="ltx_td ltx_align_center">8B</td>
<td id="S4.T1.1.10.10.3" class="ltx_td ltx_align_center">39.8</td>
</tr>
<tr id="S4.T1.1.11.11" class="ltx_tr">
<th id="S4.T1.1.11.11.1" class="ltx_td ltx_align_left ltx_th ltx_th_row">Llama-3-70B-Instruct</th>
<td id="S4.T1.1.11.11.2" class="ltx_td ltx_align_center">70B</td>
<td id="S4.T1.1.11.11.3" class="ltx_td ltx_align_center">63.5</td>
</tr>
<tr id="S4.T1.1.12.12" class="ltx_tr">
<th id="S4.T1.1.12.12.1" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_t" colspan="3"><span id="S4.T1.1.12.12.1.1" class="ltx_text ltx_font_bold">GPT-4</span></th>
</tr>
<tr id="S4.T1.1.13.13" class="ltx_tr">
<th id="S4.T1.1.13.13.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_t"><span id="S4.T1.1.13.13.1.1" class="ltx_text ltx_font_typewriter">gpt-4-turbo-2024-04-09</span></th>
<td id="S4.T1.1.13.13.2" class="ltx_td ltx_align_center ltx_border_t">?</td>
<td id="S4.T1.1.13.13.3" class="ltx_td ltx_align_center ltx_border_t">88.1</td>
</tr>
<tr id="S4.T1.1.14.14" class="ltx_tr">
<th id="S4.T1.1.14.14.1" class="ltx_td ltx_align_left ltx_th ltx_th_row"><span id="S4.T1.1.14.14.1.1" class="ltx_text ltx_font_typewriter">gpt-4o-2024-05-13</span></th>
<td id="S4.T1.1.14.14.2" class="ltx_td ltx_align_center">?</td>
<td id="S4.T1.1.14.14.3" class="ltx_td ltx_align_center">91.2</td>
</tr>
<tr id="S4.T1.1.15.15" class="ltx_tr">
<th id="S4.T1.1.15.15.1" class="ltx_td ltx_align_center ltx_th ltx_th_row ltx_border_t" colspan="3"><span id="S4.T1.1.15.15.1.1" class="ltx_text ltx_font_bold">This work</span></th>
</tr>
<tr id="S4.T1.1.16.16" class="ltx_tr">
<th id="S4.T1.1.16.16.1" class="ltx_td ltx_align_left ltx_th ltx_th_row ltx_border_bb ltx_border_t">Qwen2-7B (fine-tuned <span id="S4.T1.1.16.16.1.1" class="ltx_text ltx_font_italic">w/</span> the 1.07M synthesized instances)</th>
<td id="S4.T1.1.16.16.2" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><span id="S4.T1.1.16.16.2.1" class="ltx_text ltx_font_bold">7B</span></td>
<td id="S4.T1.1.16.16.3" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><span id="S4.T1.1.16.16.3.1" class="ltx_text ltx_font_bold">79.4</span></td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 1: </span>In-distribution evaluation results on the 11.6K synthetic test instances.</figcaption>
</figure>
<figure id="S4.T2" class="ltx_table">
<table id="S4.T2.19" class="ltx_tabular ltx_centering ltx_align_middle">
<tbody class="ltx_tbody">
<tr id="S4.T2.19.20.1" class="ltx_tr">
<td id="S4.T2.19.20.1.1" class="ltx_td ltx_align_left ltx_border_tt"><span id="S4.T2.19.20.1.1.1" class="ltx_text ltx_font_bold">Model</span></td>
<td id="S4.T2.19.20.1.2" class="ltx_td ltx_align_center ltx_border_tt"><span id="S4.T2.19.20.1.2.1" class="ltx_text ltx_font_bold">Model Size</span></td>
<td id="S4.T2.19.20.1.3" class="ltx_td ltx_align_left ltx_border_tt"><span id="S4.T2.19.20.1.3.1" class="ltx_text ltx_font_bold">Accuracy (%)</span></td>
</tr>
<tr id="S4.T2.19.21.2" class="ltx_tr">
<td id="S4.T2.19.21.2.1" class="ltx_td ltx_align_center ltx_border_t" colspan="3"><span id="S4.T2.19.21.2.1.1" class="ltx_text ltx_font_bold">State-of-the-art LLMs</span></td>
</tr>
<tr id="S4.T2.19.22.3" class="ltx_tr">
<td id="S4.T2.19.22.3.1" class="ltx_td ltx_align_left ltx_border_t"><span id="S4.T2.19.22.3.1.1" class="ltx_text ltx_font_typewriter">gpt-4o-2024-05-13</span></td>
<td id="S4.T2.19.22.3.2" class="ltx_td ltx_align_center ltx_border_t">?</td>
<td id="S4.T2.19.22.3.3" class="ltx_td ltx_align_left ltx_border_t"><span id="S4.T2.19.22.3.3.1" class="ltx_text ltx_font_bold">76.6</span></td>
</tr>
<tr id="S4.T2.1.1" class="ltx_tr">
<td id="S4.T2.1.1.2" class="ltx_td ltx_align_left"><span id="S4.T2.1.1.2.1" class="ltx_text ltx_font_typewriter">gpt-4-turbo-2024-04-09</span></td>
<td id="S4.T2.1.1.3" class="ltx_td ltx_align_center">?</td>
<td id="S4.T2.1.1.1" class="ltx_td ltx_align_left"><math id="S4.T2.1.1.1.m1.1" class="ltx_Math" alttext="73.4" display="inline"><semantics id="S4.T2.1.1.1.m1.1a"><mn id="S4.T2.1.1.1.m1.1.1" xref="S4.T2.1.1.1.m1.1.1.cmml">73.4</mn><annotation-xml encoding="MathML-Content" id="S4.T2.1.1.1.m1.1b"><cn type="float" id="S4.T2.1.1.1.m1.1.1.cmml" xref="S4.T2.1.1.1.m1.1.1">73.4</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.1.1.1.m1.1c">73.4</annotation></semantics></math></td>
</tr>
<tr id="S4.T2.2.2" class="ltx_tr">
<td id="S4.T2.2.2.2" class="ltx_td ltx_align_left"><span id="S4.T2.2.2.2.1" class="ltx_text ltx_font_typewriter">gpt-4-turbo-0125-preview</span></td>
<td id="S4.T2.2.2.3" class="ltx_td ltx_align_center">?</td>
<td id="S4.T2.2.2.1" class="ltx_td ltx_align_left"><math id="S4.T2.2.2.1.m1.1" class="ltx_Math" alttext="64.5" display="inline"><semantics id="S4.T2.2.2.1.m1.1a"><mn id="S4.T2.2.2.1.m1.1.1" xref="S4.T2.2.2.1.m1.1.1.cmml">64.5</mn><annotation-xml encoding="MathML-Content" id="S4.T2.2.2.1.m1.1b"><cn type="float" id="S4.T2.2.2.1.m1.1.1.cmml" xref="S4.T2.2.2.1.m1.1.1">64.5</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.2.2.1.m1.1c">64.5</annotation></semantics></math></td>
</tr>
<tr id="S4.T2.3.3" class="ltx_tr">
<td id="S4.T2.3.3.2" class="ltx_td ltx_align_left"><span id="S4.T2.3.3.2.1" class="ltx_text ltx_font_typewriter">gpt-4-turbo-1106-preview</span></td>
<td id="S4.T2.3.3.3" class="ltx_td ltx_align_center">?</td>
<td id="S4.T2.3.3.1" class="ltx_td ltx_align_left"><math id="S4.T2.3.3.1.m1.1" class="ltx_Math" alttext="64.3" display="inline"><semantics id="S4.T2.3.3.1.m1.1a"><mn id="S4.T2.3.3.1.m1.1.1" xref="S4.T2.3.3.1.m1.1.1.cmml">64.3</mn><annotation-xml encoding="MathML-Content" id="S4.T2.3.3.1.m1.1b"><cn type="float" id="S4.T2.3.3.1.m1.1.1.cmml" xref="S4.T2.3.3.1.m1.1.1">64.3</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.3.3.1.m1.1c">64.3</annotation></semantics></math></td>
</tr>
<tr id="S4.T2.5.5" class="ltx_tr">
<td id="S4.T2.5.5.3" class="ltx_td ltx_align_left"><span id="S4.T2.5.5.3.1" class="ltx_text ltx_font_typewriter">gpt-4</span></td>
<td id="S4.T2.5.5.4" class="ltx_td ltx_align_center">?</td>
<td id="S4.T2.5.5.2" class="ltx_td ltx_align_left">
<math id="S4.T2.4.4.1.m1.1" class="ltx_Math" alttext="52.6" display="inline"><semantics id="S4.T2.4.4.1.m1.1a"><mn id="S4.T2.4.4.1.m1.1.1" xref="S4.T2.4.4.1.m1.1.1.cmml">52.6</mn><annotation-xml encoding="MathML-Content" id="S4.T2.4.4.1.m1.1b"><cn type="float" id="S4.T2.4.4.1.m1.1.1.cmml" xref="S4.T2.4.4.1.m1.1.1">52.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.4.4.1.m1.1c">52.6</annotation></semantics></math><span id="S4.T2.5.5.2.1" class="ltx_text ltx_inline-block" style="width:0.0pt;"><sup id="S4.T2.5.5.2.1.1" class="ltx_sup"><span id="S4.T2.5.5.2.1.1.1" class="ltx_text ltx_font_italic">∗</span></sup></span>
</td>
</tr>
<tr id="S4.T2.7.7" class="ltx_tr">
<td id="S4.T2.7.7.3" class="ltx_td ltx_align_left">Claude 3.5 Sonnet</td>
<td id="S4.T2.7.7.4" class="ltx_td ltx_align_center">?</td>
<td id="S4.T2.7.7.2" class="ltx_td ltx_align_left">
<math id="S4.T2.6.6.1.m1.1" class="ltx_Math" alttext="71.1" display="inline"><semantics id="S4.T2.6.6.1.m1.1a"><mn id="S4.T2.6.6.1.m1.1.1" xref="S4.T2.6.6.1.m1.1.1.cmml">71.1</mn><annotation-xml encoding="MathML-Content" id="S4.T2.6.6.1.m1.1b"><cn type="float" id="S4.T2.6.6.1.m1.1.1.cmml" xref="S4.T2.6.6.1.m1.1.1">71.1</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.6.6.1.m1.1c">71.1</annotation></semantics></math><span id="S4.T2.7.7.2.1" class="ltx_text ltx_inline-block" style="width:0.0pt;"><sup id="S4.T2.7.7.2.1.1" class="ltx_sup"><span id="S4.T2.7.7.2.1.1.1" class="ltx_text ltx_font_italic">∗</span></sup></span>
</td>
</tr>
<tr id="S4.T2.8.8" class="ltx_tr">
<td id="S4.T2.8.8.2" class="ltx_td ltx_align_left">Claude 3 Opus</td>
<td id="S4.T2.8.8.3" class="ltx_td ltx_align_center">?</td>
<td id="S4.T2.8.8.1" class="ltx_td ltx_align_left"><math id="S4.T2.8.8.1.m1.1" class="ltx_Math" alttext="63.8" display="inline"><semantics id="S4.T2.8.8.1.m1.1a"><mn id="S4.T2.8.8.1.m1.1.1" xref="S4.T2.8.8.1.m1.1.1.cmml">63.8</mn><annotation-xml encoding="MathML-Content" id="S4.T2.8.8.1.m1.1b"><cn type="float" id="S4.T2.8.8.1.m1.1.1.cmml" xref="S4.T2.8.8.1.m1.1.1">63.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.8.8.1.m1.1c">63.8</annotation></semantics></math></td>
</tr>
<tr id="S4.T2.10.10" class="ltx_tr">
<td id="S4.T2.10.10.3" class="ltx_td ltx_align_left">Gemini Pro 1.5 (May 2024)</td>
<td id="S4.T2.10.10.4" class="ltx_td ltx_align_center">?</td>
<td id="S4.T2.10.10.2" class="ltx_td ltx_align_left">
<math id="S4.T2.9.9.1.m1.1" class="ltx_Math" alttext="67.7" display="inline"><semantics id="S4.T2.9.9.1.m1.1a"><mn id="S4.T2.9.9.1.m1.1.1" xref="S4.T2.9.9.1.m1.1.1.cmml">67.7</mn><annotation-xml encoding="MathML-Content" id="S4.T2.9.9.1.m1.1b"><cn type="float" id="S4.T2.9.9.1.m1.1.1.cmml" xref="S4.T2.9.9.1.m1.1.1">67.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.9.9.1.m1.1c">67.7</annotation></semantics></math><span id="S4.T2.10.10.2.1" class="ltx_text ltx_inline-block" style="width:0.0pt;"><sup id="S4.T2.10.10.2.1.1" class="ltx_sup"><span id="S4.T2.10.10.2.1.1.1" class="ltx_text ltx_font_italic">∗</span></sup></span>
</td>
</tr>
<tr id="S4.T2.12.12" class="ltx_tr">
<td id="S4.T2.12.12.3" class="ltx_td ltx_align_left">Gemini Ultra</td>
<td id="S4.T2.12.12.4" class="ltx_td ltx_align_center">?</td>
<td id="S4.T2.12.12.2" class="ltx_td ltx_align_left">
<math id="S4.T2.11.11.1.m1.1" class="ltx_Math" alttext="53.2" display="inline"><semantics id="S4.T2.11.11.1.m1.1a"><mn id="S4.T2.11.11.1.m1.1.1" xref="S4.T2.11.11.1.m1.1.1.cmml">53.2</mn><annotation-xml encoding="MathML-Content" id="S4.T2.11.11.1.m1.1b"><cn type="float" id="S4.T2.11.11.1.m1.1.1.cmml" xref="S4.T2.11.11.1.m1.1.1">53.2</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.11.11.1.m1.1c">53.2</annotation></semantics></math><span id="S4.T2.12.12.2.1" class="ltx_text ltx_inline-block" style="width:0.0pt;"><sup id="S4.T2.12.12.2.1.1" class="ltx_sup"><span id="S4.T2.12.12.2.1.1.1" class="ltx_text ltx_font_italic">∗</span></sup></span>
</td>
</tr>
<tr id="S4.T2.14.14" class="ltx_tr">
<td id="S4.T2.14.14.3" class="ltx_td ltx_align_left">DeepSeek-Coder-V2-Instruct <cite class="ltx_cite ltx_citemacro_citep">(Zhu et al., <a href="#bib.bib44" title="" class="ltx_ref">2024</a>)</cite>
</td>
<td id="S4.T2.14.14.4" class="ltx_td ltx_align_center">236B/21B</td>
<td id="S4.T2.14.14.2" class="ltx_td ltx_align_left">
<math id="S4.T2.13.13.1.m1.1" class="ltx_Math" alttext="75.7" display="inline"><semantics id="S4.T2.13.13.1.m1.1a"><mn id="S4.T2.13.13.1.m1.1.1" xref="S4.T2.13.13.1.m1.1.1.cmml">75.7</mn><annotation-xml encoding="MathML-Content" id="S4.T2.13.13.1.m1.1b"><cn type="float" id="S4.T2.13.13.1.m1.1.1.cmml" xref="S4.T2.13.13.1.m1.1.1">75.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.13.13.1.m1.1c">75.7</annotation></semantics></math><span id="S4.T2.14.14.2.1" class="ltx_text ltx_inline-block" style="width:0.0pt;"><sup id="S4.T2.14.14.2.1.1" class="ltx_sup"><span id="S4.T2.14.14.2.1.1.1" class="ltx_text ltx_font_italic">∗</span></sup></span>
</td>
</tr>
<tr id="S4.T2.15.15" class="ltx_tr">
<td id="S4.T2.15.15.2" class="ltx_td ltx_align_left">Llama-3-70B-Instruct</td>
<td id="S4.T2.15.15.3" class="ltx_td ltx_align_center">70B</td>
<td id="S4.T2.15.15.1" class="ltx_td ltx_align_left"><math id="S4.T2.15.15.1.m1.1" class="ltx_Math" alttext="52.8" display="inline"><semantics id="S4.T2.15.15.1.m1.1a"><mn id="S4.T2.15.15.1.m1.1.1" xref="S4.T2.15.15.1.m1.1.1.cmml">52.8</mn><annotation-xml encoding="MathML-Content" id="S4.T2.15.15.1.m1.1b"><cn type="float" id="S4.T2.15.15.1.m1.1.1.cmml" xref="S4.T2.15.15.1.m1.1.1">52.8</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.15.15.1.m1.1c">52.8</annotation></semantics></math></td>
</tr>
<tr id="S4.T2.17.17" class="ltx_tr">
<td id="S4.T2.17.17.3" class="ltx_td ltx_align_left">Qwen2-72B-Instruct</td>
<td id="S4.T2.17.17.4" class="ltx_td ltx_align_center">72B</td>
<td id="S4.T2.17.17.2" class="ltx_td ltx_align_left">
<math id="S4.T2.16.16.1.m1.1" class="ltx_Math" alttext="59.7" display="inline"><semantics id="S4.T2.16.16.1.m1.1a"><mn id="S4.T2.16.16.1.m1.1.1" xref="S4.T2.16.16.1.m1.1.1.cmml">59.7</mn><annotation-xml encoding="MathML-Content" id="S4.T2.16.16.1.m1.1b"><cn type="float" id="S4.T2.16.16.1.m1.1.1.cmml" xref="S4.T2.16.16.1.m1.1.1">59.7</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.16.16.1.m1.1c">59.7</annotation></semantics></math><span id="S4.T2.17.17.2.1" class="ltx_text ltx_inline-block" style="width:0.0pt;"><sup id="S4.T2.17.17.2.1.1" class="ltx_sup"><span id="S4.T2.17.17.2.1.1.1" class="ltx_text ltx_font_italic">∗</span></sup></span>
</td>
</tr>
<tr id="S4.T2.19.19" class="ltx_tr">
<td id="S4.T2.19.19.3" class="ltx_td ltx_align_left">Qwen2-7B-Instruct</td>
<td id="S4.T2.19.19.4" class="ltx_td ltx_align_center">7B</td>
<td id="S4.T2.19.19.2" class="ltx_td ltx_align_left">
<math id="S4.T2.18.18.1.m1.1" class="ltx_Math" alttext="49.6" display="inline"><semantics id="S4.T2.18.18.1.m1.1a"><mn id="S4.T2.18.18.1.m1.1.1" xref="S4.T2.18.18.1.m1.1.1.cmml">49.6</mn><annotation-xml encoding="MathML-Content" id="S4.T2.18.18.1.m1.1b"><cn type="float" id="S4.T2.18.18.1.m1.1.1.cmml" xref="S4.T2.18.18.1.m1.1.1">49.6</cn></annotation-xml><annotation encoding="application/x-tex" id="S4.T2.18.18.1.m1.1c">49.6</annotation></semantics></math><span id="S4.T2.19.19.2.1" class="ltx_text ltx_inline-block" style="width:0.0pt;"><sup id="S4.T2.19.19.2.1.1" class="ltx_sup"><span id="S4.T2.19.19.2.1.1.1" class="ltx_text ltx_font_italic">∗</span></sup></span>
</td>
</tr>
<tr id="S4.T2.19.23.4" class="ltx_tr">
<td id="S4.T2.19.23.4.1" class="ltx_td ltx_align_center ltx_border_t" colspan="3"><span id="S4.T2.19.23.4.1.1" class="ltx_text ltx_font_bold">This work</span></td>
</tr>
<tr id="S4.T2.19.24.5" class="ltx_tr">
<td id="S4.T2.19.24.5.1" class="ltx_td ltx_align_left ltx_border_bb ltx_border_t">Qwen2-7B (fine-tuned <span id="S4.T2.19.24.5.1.1" class="ltx_text ltx_font_italic">w/</span> the 1.07M synthesized instances)</td>
<td id="S4.T2.19.24.5.2" class="ltx_td ltx_align_center ltx_border_bb ltx_border_t"><span id="S4.T2.19.24.5.2.1" class="ltx_text ltx_font_bold">7B</span></td>
<td id="S4.T2.19.24.5.3" class="ltx_td ltx_align_left ltx_border_bb ltx_border_t"><span id="S4.T2.19.24.5.3.1" class="ltx_text ltx_font_bold">64.9</span></td>
</tr>
</tbody>
</table>
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_table">Table 2: </span>Out-of-distribution evaluation on MATH. Results marked with an asterisk (*) may not use the OpenAI’s evaluation method. The model fine-tuned with our synthesized 1.07M math problems achieves 64.9% on MATH, matching the performance of <span id="S4.T2.21.1" class="ltx_text ltx_font_typewriter">gpt-4-turbo-preview</span> at only a 7B scale.</figcaption>
</figure>
<figure id="S4.F9" class="ltx_figure ltx_align_floatright"><img src="/html/2406.20094/assets/x9.png" id="S4.F9.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="253" height="201" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 9: </span>Accuracy on MATH with scaling the synthetic instances used for training Qwen2-7B</figcaption>
</figure>
<div id="S4.SS1.SSS2.Px3.p3" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS2.Px3.p3.1" class="ltx_p">Table <a href="#S4.T1" title="Table 1 ‣ Equality Checking ‣ 4.1.2 Evaluation ‣ 4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> presents the in-distribution (ID) evaluation results on the 11.6K synthetic test instances. Among the tested open-source LLMs, Qwen2-72B-Instruct achieves the best result, and the ranking of the other models is generally consistent with their reported performance on other mathematical benchmarks. Our model, with the help of the 1.07M synthetic math problems, achieves nearly 80% accuracy, surpassing all the open-source LLMs. However, considering that the answers in the synthetic test are not absolutely reliable and that our model might be the only one using ID training data, this ID evaluation results should be taken as a reference only.</p>
</div>
<div id="S4.SS1.SSS2.Px3.p4" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS2.Px3.p4.1" class="ltx_p">We present the evaluation results on MATH in Table <a href="#S4.T2" title="Table 2 ‣ Equality Checking ‣ 4.1.2 Evaluation ‣ 4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>. The 7B model fine-tuned with the synthetic training data achieved an impressive 64.9% accuracy on MATH simply using greedy decoding, outperformed only by <span id="S4.SS1.SSS2.Px3.p4.1.1" class="ltx_text ltx_font_typewriter">gpt-4o</span>, <span id="S4.SS1.SSS2.Px3.p4.1.2" class="ltx_text ltx_font_typewriter">gpt-4-turbo-2024-04-09</span>, Claude 3.5 Sonnet, Gemini Pro 1.5 (May 2024) and DeepSeek-Coder-V2-Instruct.</p>
</div>
<div id="S4.SS1.SSS2.Px3.p5" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS2.Px3.p5.1" class="ltx_p">Figure <a href="#S4.F9" title="Figure 9 ‣ Equality Checking ‣ 4.1.2 Evaluation ‣ 4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">9</span></a> presents the performance of the model on MATH when trained with synthetic math problems at different scales. Its performance trend generally aligns with the scaling law <cite class="ltx_cite ltx_citemacro_citep">(Kaplan et al., <a href="#bib.bib20" title="" class="ltx_ref">2020</a>)</cite>. Unlike previous research <cite class="ltx_cite ltx_citemacro_citep">(Yu et al., <a href="#bib.bib41" title="" class="ltx_ref">2023</a>; Wang et al., <a href="#bib.bib36" title="" class="ltx_ref">2023</a>; Li et al., <a href="#bib.bib21" title="" class="ltx_ref">2024a</a>)</cite> that performs scaling on in-distribution data (e.g., heavily relying on MATH train data to augment in-distribution data), we did not use any instances from MATH during data synthesis or training. Therefore, in this out-of-distribution (OOD) evaluation setting, achieving performance on MATH that surpasses <span id="S4.SS1.SSS2.Px3.p5.1.1" class="ltx_text ltx_font_typewriter">gpt-4-turbo-preview (1106/0125)</span> is indeed impressive and promising for a 7B model.</p>
</div>
<figure id="S4.F10" class="ltx_figure"><img src="/html/2406.20094/assets/x10.png" id="S4.F10.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="460" height="138" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 10: </span>Similarities of math problems created by personas with different similarities: <span id="S4.F10.6.1" class="ltx_text ltx_font_bold">(a)</span> Similarity of math problems when no specific focus is given; <span id="S4.F10.7.2" class="ltx_text ltx_font_bold">(b)</span> Similarity of math problems when the prompt specifies they must be related to finance and probability; <span id="S4.F10.8.3" class="ltx_text ltx_font_bold">(c)</span> Similarity of math problems synthesized by <span id="S4.F10.9.4" class="ltx_text ltx_font_typewriter">gpt-4o</span> and <span id="S4.F10.10.5" class="ltx_text ltx_font_typewriter">gpt-35-turbo</span> with persona similarity of 0.9.</figcaption>
</figure>
<div id="S4.SS1.SSS2.Px3.p6" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS2.Px3.p6.1" class="ltx_p">We examine the quality of our synthesized math problems: we sample 200 challenging problems (involving high school and university-level math knowledge points in China), and have two math experts evaluate their validity. Only 7 out of 200 problems are marked as invalid (e.g., due to insufficient or conflicting conditions), yielding a reliable validity rate of 96.5%.</p>
</div>
<div id="S4.SS1.SSS2.Px3.p7" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS2.Px3.p7.1" class="ltx_p">Moreover, we specifically examine the impact of differences in personas within the prompts on the synthesized math problems. We first sample 100 pairs of personas with semantic similarities<span id="footnote8" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">8</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">8</sup><span class="ltx_tag ltx_tag_note">8</span>We use OpenAI’s <span id="footnote8.1" class="ltx_text ltx_font_typewriter">text-embedding-3-small</span> (dim=512) to obtain the semantic representation and compute cosine similarity in this experiment. Here, a persona similarity of 0.4 means that a pair of personas has a semantic similarity within the range of 0.39 to 0.41.</span></span></span> of 0.4, 0.6, and 0.8, respectively. For each pair of personas, we use them to create a pair of math problems using greedy decoding (i.e., temperature=0). Then, we compute the semantic similarity of these math problem pairs and show the results in Figure <a href="#S4.F10" title="Figure 10 ‣ Equality Checking ‣ 4.1.2 Evaluation ‣ 4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">10</span></a>.</p>
</div>
<div id="S4.SS1.SSS2.Px3.p8" class="ltx_para ltx_noindent">
<p id="S4.SS1.SSS2.Px3.p8.1" class="ltx_p">We can clearly observe that the semantic similarity between synthesized math problems tends to be correlated with but lower than the similarity between their corresponding personas. When we add more specific constraints to the prompts (e.g., math problems about finance and probability), the similarity between the synthesized math problems tends to become higher (Figure <a href="#S4.F10" title="Figure 10 ‣ Equality Checking ‣ 4.1.2 Evaluation ‣ 4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">10</span></a>(b)). In Figure <a href="#S4.F10" title="Figure 10 ‣ Equality Checking ‣ 4.1.2 Evaluation ‣ 4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">10</span></a>(c), we also test the similarity of math problems created by <span id="S4.SS1.SSS2.Px3.p8.1.1" class="ltx_text ltx_font_typewriter">gpt-4o</span> and <span id="S4.SS1.SSS2.Px3.p8.1.2" class="ltx_text ltx_font_typewriter">gpt-35-turbo</span> using highly similar personas (similarity=0.9). The results indicate that the semantic similarity of the math problems created by <span id="S4.SS1.SSS2.Px3.p8.1.3" class="ltx_text ltx_font_typewriter">gpt-4o</span> and <span id="S4.SS1.SSS2.Px3.p8.1.4" class="ltx_text ltx_font_typewriter">gpt-35-turbo</span> seems not significantly different: most synthesized math problems’ similarity falls within the range of 0.6 to 0.75, which is much lower than the similarity of the personas (0.9). Given these observations, we believe that using the personas in Persona Hub can ensure the diversity of synthesized data – even at a billion scale.</p>
</div>
</section>
</section>
</section>
<section id="S4.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.2 </span>Logical Reasoning Problems</h3>

<figure id="S4.F11" class="ltx_figure"><img src="/html/2406.20094/assets/x11.png" id="S4.F11.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="393" height="247" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 11: </span>Logical reasoning problems created by our proposed persona-driven methodology</figcaption>
</figure>
<figure id="S4.F12" class="ltx_figure"><img src="/html/2406.20094/assets/x12.png" id="S4.F12.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="453" height="200" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 12: </span>Ruozhiba-style logical reasoning problems created with various personas. Note that Logical Prompt 3 in this figure is a simplified prompt. In practice, we need to specifically define a Ruozhiba-style logical reasoning problem in this prompt in order to obtain desired synthetic data.</figcaption>
</figure>
<div id="S4.SS2.p1" class="ltx_para ltx_noindent">
<p id="S4.SS2.p1.1" class="ltx_p">Similar to math problems, logical reasoning problems can also be easily synthesized. We present examples of typical logical reasoning problems synthesized using our proposed persona-driven methodology in Figure <a href="#S4.F11" title="Figure 11 ‣ 4.2 Logical Reasoning Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">11</span></a>.</p>
</div>
<figure id="S4.F13" class="ltx_figure"><img src="/html/2406.20094/assets/x13.png" id="S4.F13.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="363" height="246" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 13: </span>Two typical prompts used for creating instructions (i.e., user prompts).</figcaption>
</figure>
<div id="S4.SS2.p2" class="ltx_para ltx_noindent">
<p id="S4.SS2.p2.1" class="ltx_p">Moreover, we also show several Ruozhiba-style<span id="footnote9" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">9</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">9</sup><span class="ltx_tag ltx_tag_note">9</span><a target="_blank" href="https://tieba.baidu.com/f?ie=utf-8&amp;kw=%E5%BC%B1%E6%99%BA%E5%90%A7" title="" class="ltx_ref ltx_href">Ruozhiba</a> is a subforum on <a target="_blank" href="https://tieba.baidu.com" title="" class="ltx_ref ltx_href">Baidu Tieba</a>, featuring numerous intricate and challenging questions posted by Chinese netizens. These questions combine elements such as puns, polysemy, causal inversion, and homophones, embedding logical traps that rigorously test the ability to understand complex Chinese language constructs. Recent research <cite class="ltx_cite ltx_citemacro_citep">(Bai et al., <a href="#bib.bib4" title="" class="ltx_ref">2024</a>)</cite> has demonstrated that these data significantly benefit the improvement of LLMs’ logical reasoning abilities.</span></span></span> logical reasoning problems created with personas in Figure <a href="#S4.F12" title="Figure 12 ‣ 4.2 Logical Reasoning Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">12</span></a>. All the examples demonstrate that as long as we can clearly describe the requirements for the logical reasoning problem to be created, we can use a large variety of personas to steer the LLM to generate diverse logical reasoning problems that not only meet the requirements but are also highly relevant to the personas, even for whimsical Ruozhiba-style problems.</p>
</div>
<div id="S4.SS2.p3" class="ltx_para ltx_noindent">
<p id="S4.SS2.p3.1" class="ltx_p">For more examples, please refer to the 50,000 synthetic reasoning problems we have released.</p>
</div>
</section>
<section id="S4.SS3" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.3 </span>Instructions</h3>

<div id="S4.SS3.p1" class="ltx_para ltx_noindent">
<p id="S4.SS3.p1.1" class="ltx_p">The end users of LLMs are ultimately humans. We can use Persona Hub to simulate a variety of users to understand their typical requests for LLM assistance, resulting in diverse instructions (i.e., user prompts).</p>
</div>
<div id="S4.SS3.p2" class="ltx_para ltx_noindent">
<p id="S4.SS3.p2.1" class="ltx_p">Figure <a href="#S4.F13" title="Figure 13 ‣ 4.2 Logical Reasoning Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">13</span></a> shows two typical persona-driven prompts for synthesizing instructions, corresponding to the zero-shot prompting and persona-enhanced few-shot prompting methods described in Section <a href="#S3" title="3 Persona-driven Synthetic Data Creation ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">3</span></a>. The zero-shot method does not rely on any existing instruction dataset and allows the LLM to generate various instructions based on different personas. In contrast, the persona-enhanced few-shot method requires existing instruction datasets (e.g., we use WildChat <cite class="ltx_cite ltx_citemacro_citep">(Zhao et al., <a href="#bib.bib43" title="" class="ltx_ref">2024</a>)</cite> in our experiments) to sample some instructions as demonstrations and involves inferring the associated personas of these instructions through the <span id="S4.SS3.p2.1.1" class="ltx_text ltx_font_italic">Text-to-Persona</span> method described in Section <a href="#S2.SS1" title="2.1 Text-to-Persona ‣ 2 Persona Hub ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2.1</span></a>. While this approach is more complex, it results in synthesized instructions that more closely resembles instructions from real users.</p>
</div>
<div id="S4.SS3.p3" class="ltx_para ltx_noindent">
<p id="S4.SS3.p3.1" class="ltx_p">With diverse instructions created using Persona Hub, which typically represent the first turn of a user-LLM conversation, we can easily generate their subsequent conversational turns using an LLM, resulting in a large number of simulated user-LLM conversations, which will be valuable for enhancing the LLM’s instruction-following and conversational abilities. Furthermore, we can even adopt a similar approach by selecting two personas from Persona Hub and having LLMs role-play both, thereby simulating conversations <cite class="ltx_cite ltx_citemacro_citep">(Jandaghi et al., <a href="#bib.bib19" title="" class="ltx_ref">2023</a>)</cite> between two real people.</p>
</div>
<div id="S4.SS3.p4" class="ltx_para ltx_noindent">
<p id="S4.SS3.p4.1" class="ltx_p">As Figure <a href="#S0.F1" title="Figure 1 ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">1</span></a> has already shown some instructions created using this methodology, we skip showing examples here. Readers interested in more examples can refer to the released 50,000 instructions synthesized through 0-shot and persona-enhanced 2-shot prompting.</p>
</div>
</section>
<section id="S4.SS4" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.4 </span>Knowledge-rich Texts</h3>

<figure id="S4.F14" class="ltx_figure"><img src="" id="S4.F14.g1" class="ltx_graphics ltx_centering ltx_missing ltx_missing_image" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 14: </span>Examples of knowledge-rich plain text synthesis with personas</figcaption>
</figure>
<div id="S4.SS4.p1" class="ltx_para ltx_noindent">
<p id="S4.SS4.p1.1" class="ltx_p">In addition to synthesizing instructions that can enhance the instruction tuning of LLMs, the persona-driven methodology can be easily adapted to create knowledge-rich plain text that benefits pre-training and post-training of LLMs. As illustrated in Figure <a href="#S4.F14" title="Figure 14 ‣ 4.4 Knowledge-rich Texts ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">14</span></a>, we can prompt an LLM to write a Quora<span id="footnote10" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">10</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">10</sup><span class="ltx_tag ltx_tag_note">10</span><a target="_blank" href="https://www.quora.com/" title="" class="ltx_ref ltx_href">Quora</a> is a popular question-and-answer website where users can ask questions and provide answers on a wide range of topics. Articles (i.e., posts) on Quora are often written by knowledgeable individuals, including experts in various fields, ensuring high-quality, well-researched, and informative content.</span></span></span> article<span id="footnote11" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">11</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">11</sup><span class="ltx_tag ltx_tag_note">11</span>The methods for synthesizing knowledge-rich plain text are not limited to having the LLM write Quora articles. For instance, we can also prompt the LLM to synthesize (educational) reading material that a persona may be interested in, thereby obtaining a large amount of knowledge-rich text.</span></span></span> using a persona sampled from Persona Hub. This approach elicits the LLM’s corresponding knowledge and perspective, resulting in highly informative and knowledge-rich content. By scaling this process with 1 billion personas in Persona Hub, we can easily obtain a vast array of knowledge-rich texts that cover almost any topic across various levels of granularity.</p>
</div>
</section>
<section id="S4.SS5" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.5 </span>Game NPCs</h3>

<div id="S4.SS5.p1" class="ltx_para ltx_noindent">
<p id="S4.SS5.p1.1" class="ltx_p">A straightforward and practical application of Persona Hub is creating diverse NPCs (Non-Player Characters) at scale for games. As long as we can provide a game’s background and world-building information to the LLM, we can prompt the LLM to project personas from Persona Hub (which are typically real-world personas) into characters within the game’s world. In this way, we can significantly reduce the effort required for brainstorming NPCs during the game design process.</p>
</div>
<figure id="S4.F15" class="ltx_figure"><img src="/html/2406.20094/assets/x15.png" id="S4.F15.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="453" height="257" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 15: </span>NPC creation for the game “World of Warcraft” using personas in Persona Hub</figcaption>
</figure>
<div id="S4.SS5.p2" class="ltx_para ltx_noindent">
<p id="S4.SS5.p2.1" class="ltx_p">Figure <a href="#S4.F15" title="Figure 15 ‣ 4.5 Game NPCs ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">15</span></a> and <a href="#S4.F16" title="Figure 16 ‣ 4.5 Game NPCs ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">16</span></a> show concrete examples where we use personas in Persona Hub to create game NPCs for the game “World of Warcraft<span id="footnote12" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">12</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">12</sup><span class="ltx_tag ltx_tag_note">12</span><a target="_blank" href="https://worldofwarcraft.blizzard.com/en-us/" title="" class="ltx_ref ltx_href">World of Warcraft</a>, developed by Blizzard Entertainment, is a highly influential MMORPG with millions of active players worldwide, spanning over 100 countries since its release in 2004.</span></span></span>” and “Moonlight Blade<span id="footnote13" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">13</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">13</sup><span class="ltx_tag ltx_tag_note">13</span><a target="_blank" href="https://wuxia.qq.com/main.shtml" title="" class="ltx_ref ltx_href">Moonlight Blade (天涯明月刀)</a> is a 3D martial arts-themed MMORPG developed by Tencent, officially launched in China on July 1, 2016.</span></span></span>”.</p>
</div>
<figure id="S4.F16" class="ltx_figure"><img src="/html/2406.20094/assets/x16.png" id="S4.F16.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="454" height="243" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 16: </span>NPC creation for the game “Moonlight Blade (天涯明月刀)” using Persona Hub</figcaption>
</figure>
</section>
<section id="S4.SS6" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">4.6 </span>Tool (Function) Development</h3>

<figure id="S4.F17" class="ltx_figure"><img src="/html/2406.20094/assets/x17.png" id="S4.F17.g1" class="ltx_graphics ltx_centering ltx_img_landscape" width="453" height="194" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 17: </span>Examples of tool (function) creation with Persona Hub</figcaption>
</figure>
<div id="S4.SS6.p1" class="ltx_para ltx_noindent">
<p id="S4.SS6.p1.1" class="ltx_p">As Section <a href="#S4.SS3" title="4.3 Instructions ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.3</span></a> demonstrates, Persona Hub can be used to simulate a wide variety of real users to anticipate their possible requests (i.e., instructions) to an LLM. Similarly, we can use Persona Hub to predict the tools <cite class="ltx_cite ltx_citemacro_citep">(Cai et al., <a href="#bib.bib8" title="" class="ltx_ref">2023</a>; Schick et al., <a href="#bib.bib30" title="" class="ltx_ref">2024</a>)</cite> that users might need, so that we can pre-build these tools (functions) beforehand. When a real user makes a similar request, the LLM can directly call these pre-built tools to return results without having to build tools from scratch each time. This paradigm, introduced by Persona Hub, is a completely new solution that allows LLMs to better serve users. We believe it will have great potential in the future as LLMs become more democratized and multifunctional.</p>
</div>
<div id="S4.SS6.p2" class="ltx_para ltx_noindent">
<p id="S4.SS6.p2.1" class="ltx_p">Figure <a href="#S4.F17" title="Figure 17 ‣ 4.6 Tool (Function) Development ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">17</span></a> shows examples of tools created with various personas. These tools provide functionalities that the personas may need (e.g., a cab driver needs to check traffic conditions) but cannot be accessed by an LLM, greatly expanding the range of services provided by the LLM. Note that although the tools in Figure <a href="#S4.F17" title="Figure 17 ‣ 4.6 Tool (Function) Development ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">17</span></a> are just interface definitions, these definitions can be easily converted into code implementations, as shown in Figure <a href="#S4.F18" title="Figure 18 ‣ 4.6 Tool (Function) Development ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">18</span></a>.</p>
</div>
<figure id="S4.F18" class="ltx_figure"><img src="/html/2406.20094/assets/x18.png" id="S4.F18.g1" class="ltx_graphics ltx_centering ltx_img_square" width="363" height="346" alt="Refer to caption">
<figcaption class="ltx_caption ltx_centering"><span class="ltx_tag ltx_tag_figure">Figure 18: </span>The interface definitions (e.g., the species identification interface) in Figure <a href="#S4.F17" title="Figure 17 ‣ 4.6 Tool (Function) Development ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">17</span></a> can be easily converted into code implementations by calling an LLM to implement them. The resulting pre-built tools can then be directly utilized by the LLM in the future, eliminating the need to build them from scratch each time.</figcaption>
</figure>
</section>
</section>
<section id="S5" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">5 </span>Broad Impact and Ethical Concerns</h2>

<section id="S5.SS1" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.1 </span>Broad Impact</h3>

<section id="S5.SS1.SSS1" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">5.1.1 </span>Paradigm Shift in Data Creation by Humans and LLMs</h4>

<div id="S5.SS1.SSS1.p1" class="ltx_para ltx_noindent">
<p id="S5.SS1.SSS1.p1.1" class="ltx_p">Traditionally, it has been widely accepted that while LLMs excel at processing data (e.g., rewriting, annotation, or generating outputs/solutions to specific inputs), they are not particularly adept at creating new data. Consequently, the task of data creation has still largely been the domain of humans, and the collaboration paradigm between humans and LLMs has always been humans creating data and LLMs processing it <cite class="ltx_cite ltx_citemacro_citep">(Maini et al., <a href="#bib.bib27" title="" class="ltx_ref">2024</a>)</cite>. However, the introduction of our proposed persona-driven methodology potentially revolutionizes this paradigm. With Persona Hub, LLMs are no longer confined to processing existing data; they can now create various types of new data from a multitude of perspectives, much like the diverse population of the world.</p>
</div>
<div id="S5.SS1.SSS1.p2" class="ltx_para ltx_noindent">
<p id="S5.SS1.SSS1.p2.1" class="ltx_p">While the current capabilities of LLMs may not yet fully replace humans in fulfilling the mission of data creation—whether in terms of data quality or breadth—the ongoing advancements in LLM capabilities suggest a future where LLMs will increasingly excel in data creation. As LLMs continue to improve, both the quality and breadth of the data they can create will also likely enhance, leading us to a point where LLMs may fully take on the role of data creation. When this day arrives, we will no longer be constrained <cite class="ltx_cite ltx_citemacro_citep">(Villalobos et al., <a href="#bib.bib35" title="" class="ltx_ref">2024</a>)</cite> by the limited high-quality human-produced real-world data<span id="footnote14" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">14</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">14</sup><span class="ltx_tag ltx_tag_note">14</span><a target="_blank" href="https://lilianweng.github.io/posts/2024-02-05-human-data-quality/" title="" class="ltx_ref ltx_href">https://lilianweng.github.io/posts/2024-02-05-human-data-quality/</a></span></span></span>. Persona Hub ensures the diversity and coverage of synthetic data, significantly mitigating concerns about the negative impacts <cite class="ltx_cite ltx_citemacro_citep">(Shumailov et al., <a href="#bib.bib32" title="" class="ltx_ref">2023</a>; Dohmatob et al., <a href="#bib.bib14" title="" class="ltx_ref">2024</a>)</cite> of synthetic data on model training. This may effectively eliminate the data bottleneck, thereby pushing the scaling law to its limit.</p>
</div>
</section>
<section id="S5.SS1.SSS2" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">5.1.2 </span>Reality Simulation</h4>

<div id="S5.SS1.SSS2.p1" class="ltx_para ltx_noindent">
<p id="S5.SS1.SSS2.p1.1" class="ltx_p">In Section <a href="#S4.SS3" title="4.3 Instructions ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.3</span></a> and <a href="#S4.SS6" title="4.6 Tool (Function) Development ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.6</span></a>, we have demonstrated that Persona Hub can represent a vast array of real-world individuals with its 1 billion personas. By employing these personas to simulate and infer the potential needs and behaviors of real users, we can not only allow LLMs to autonomously prepare for upcoming use cases (queries), but also pave the way for LLMs to effectively mimic the real world, thereby creating many new opportunities. For instance, companies can use this method to predict how different types of users might react to a new product launch; governments can foresee the public’s response to new legislation, considering various population group; in online services that require user profiling and behavior modeling, Persona Hub can facilitate the simulation of diverse user behaviors, significantly alleviating the cold start challenge.</p>
</div>
<div id="S5.SS1.SSS2.p2" class="ltx_para ltx_noindent">
<p id="S5.SS1.SSS2.p2.1" class="ltx_p">Recent research on LLM roleplay, agent collaboration <cite class="ltx_cite ltx_citemacro_citep">(Liu et al., <a href="#bib.bib26" title="" class="ltx_ref">2023</a>; Wang et al., <a href="#bib.bib38" title="" class="ltx_ref">2024</a>)</cite>, strategic reasoning <cite class="ltx_cite ltx_citemacro_citep">(Gandhi et al., <a href="#bib.bib15" title="" class="ltx_ref">2023</a>; Zhang et al., <a href="#bib.bib42" title="" class="ltx_ref">2024</a>)</cite>, and related areas can, of course, be facilitated by the vast and diverse personas in Persona Hub. More ambitiously, the 1 billion personas can even sustain a well-organized society within a virtual world, such as sandbox environments <cite class="ltx_cite ltx_citemacro_citep">(Park et al., <a href="#bib.bib29" title="" class="ltx_ref">2023</a>)</cite>, online games, parallel worlds, or the metaverse, using the method discussed in Section <a href="#S4.SS5" title="4.5 Game NPCs ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.5</span></a> to simulate operations with powerful LLMs. This virtual society can serve as a testing ground for new policies, radical initiatives, and social dynamics, providing valuable insights before real-world implementation. By creating a controlled environment where diverse personas interact, we can observe emergent behaviors, test hypotheses, and refine strategies in a risk-free setting. This can not only help deepen our understanding of complex systems but also speed up innovation by facilitating rapid iteration and experimentation.</p>
</div>
</section>
<section id="S5.SS1.SSS3" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">5.1.3 </span>Full Memory Access of LLMs</h4>

<div id="S5.SS1.SSS3.p1" class="ltx_para ltx_noindent">
<p id="S5.SS1.SSS3.p1.1" class="ltx_p">When we interact with an LLM in a specific scenario, we can only elicit a fraction of its memory and capabilities, and we are unable to fully access the vast knowledge encapsulated within the LLM. However, Persona Hub potentially offers access to the full memory of an LLM because the 1 billion personas in Persona Hub can tap into almost every perspective and piece of information encoded within the LLM.</p>
</div>
<div id="S5.SS1.SSS3.p2" class="ltx_para ltx_noindent">
<p id="S5.SS1.SSS3.p2.1" class="ltx_p">By leveraging these 1 billion personas, we can create diverse queries and obtain solutions from a target LLM, thereby transforming the LLM’s comprehensive memory (parameters) into synthetic data in textual form. If we consider an LLM as a parameterized compression of world knowledge, then Persona Hub can be viewed as a distributed carrier-based compression<span id="footnote15" class="ltx_note ltx_role_footnote"><sup class="ltx_note_mark">15</sup><span class="ltx_note_outer"><span class="ltx_note_content"><sup class="ltx_note_mark">15</sup><span class="ltx_tag ltx_tag_note">15</span>As Figure <a href="#S1.F2" title="Figure 2 ‣ 1 Introduction ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a> illustrates, Persona Hub has on the order of <math id="footnote15.m1.1" class="ltx_Math" alttext="10^{10}" display="inline"><semantics id="footnote15.m1.1b"><msup id="footnote15.m1.1.1" xref="footnote15.m1.1.1.cmml"><mn id="footnote15.m1.1.1.2" xref="footnote15.m1.1.1.2.cmml">10</mn><mn id="footnote15.m1.1.1.3" xref="footnote15.m1.1.1.3.cmml">10</mn></msup><annotation-xml encoding="MathML-Content" id="footnote15.m1.1c"><apply id="footnote15.m1.1.1.cmml" xref="footnote15.m1.1.1"><csymbol cd="ambiguous" id="footnote15.m1.1.1.1.cmml" xref="footnote15.m1.1.1">superscript</csymbol><cn type="integer" id="footnote15.m1.1.1.2.cmml" xref="footnote15.m1.1.1.2">10</cn><cn type="integer" id="footnote15.m1.1.1.3.cmml" xref="footnote15.m1.1.1.3">10</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote15.m1.1d">10^{10}</annotation></semantics></math> tokens, equivalent to a <math id="footnote15.m2.2" class="ltx_math_unparsed" alttext="10,000\times" display="inline"><semantics id="footnote15.m2.2b"><mrow id="footnote15.m2.2c"><mn id="footnote15.m2.1.1">10</mn><mo id="footnote15.m2.2.3">,</mo><mn id="footnote15.m2.2.2">000</mn><mo lspace="0.222em" id="footnote15.m2.2.4">×</mo></mrow><annotation encoding="application/x-tex" id="footnote15.m2.2d">10,000\times</annotation></semantics></math> compression of the public web text (<math id="footnote15.m3.1" class="ltx_Math" alttext="10^{14}" display="inline"><semantics id="footnote15.m3.1b"><msup id="footnote15.m3.1.1" xref="footnote15.m3.1.1.cmml"><mn id="footnote15.m3.1.1.2" xref="footnote15.m3.1.1.2.cmml">10</mn><mn id="footnote15.m3.1.1.3" xref="footnote15.m3.1.1.3.cmml">14</mn></msup><annotation-xml encoding="MathML-Content" id="footnote15.m3.1c"><apply id="footnote15.m3.1.1.cmml" xref="footnote15.m3.1.1"><csymbol cd="ambiguous" id="footnote15.m3.1.1.1.cmml" xref="footnote15.m3.1.1">superscript</csymbol><cn type="integer" id="footnote15.m3.1.1.2.cmml" xref="footnote15.m3.1.1.2">10</cn><cn type="integer" id="footnote15.m3.1.1.3.cmml" xref="footnote15.m3.1.1.3">14</cn></apply></annotation-xml><annotation encoding="application/x-tex" id="footnote15.m3.1d">10^{14}</annotation></semantics></math> tokens).</span></span></span> of world knowledge, as demonstrated in Figure <a href="#S1.F2" title="Figure 2 ‣ 1 Introduction ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">2</span></a>. This distributed carrier-based compression provides us with an opportunity to decompress the LLM’s parameters back into world knowledge and information it has ever learned (e.g., using the method discussed in Section <a href="#S4.SS4" title="4.4 Knowledge-rich Texts ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.4</span></a>).</p>
</div>
<div id="S5.SS1.SSS3.p3" class="ltx_para ltx_noindent">
<p id="S5.SS1.SSS3.p3.1" class="ltx_p">However, considering that the current Persona Hub is still in a very preliminary stage and that today’s LLMs are not yet capable of losslessly converting their memory into synthetic data due to inevitable hallucination <cite class="ltx_cite ltx_citemacro_citep">(Xu et al., <a href="#bib.bib39" title="" class="ltx_ref">2024</a>)</cite>, the breadth and quality of the synthetic data generated through this methodology are still limited. Nevertheless, as Persona Hub continues to improve and scale, and as LLMs become more powerful (with less hallucination), we can look forward to a day when it will be possible to nearly losslessly extract the full memory of an LLM into plain text.</p>
</div>
</section>
</section>
<section id="S5.SS2" class="ltx_subsection">
<h3 class="ltx_title ltx_title_subsection">
<span class="ltx_tag ltx_tag_subsection">5.2 </span>Ethical Concerns</h3>

<section id="S5.SS2.SSS1" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">5.2.1 </span>Training Data Security and Threats to Current LLM Dominance</h4>

<div id="S5.SS2.SSS1.p1" class="ltx_para ltx_noindent">
<p id="S5.SS2.SSS1.p1.1" class="ltx_p">As discussed in Section <a href="#S5.SS1.SSS3" title="5.1.3 Full Memory Access of LLMs ‣ 5.1 Broad Impact ‣ 5 Broad Impact and Ethical Concerns ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">5.1.3</span></a>, Persona Hub offers an opportunity to access the full memory of a target LLM. However, this also introduces a significant issue: the security of the training data. All data synthesized through the target LLM essentially represents a form of its seen training data. Therefore, the process of extensively extracting a target LLM’s memory is essentially dumping its training data, even though this process is generally lossy.</p>
</div>
<div id="S5.SS2.SSS1.p2" class="ltx_para ltx_noindent">
<p id="S5.SS2.SSS1.p2.1" class="ltx_p">Moreover, if we employ the method described in Section <a href="#S4.SS3" title="4.3 Instructions ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.3</span></a> to synthesize instructions (i.e., user prompts) that nearly covers all use cases to query a target LLM to obtain its outputs at scale, there is a high risk that the target LLM’s knowledge, intelligence, and capabilities could be extracted and replicated. This poses a challenge to the leading position of the most powerful LLMs, as we have already validated in Section <a href="#S4.SS1" title="4.1 Math Problems ‣ 4 Use Cases ‣ Scaling Synthetic Data Creation with 1,000,000,000 Personas" class="ltx_ref"><span class="ltx_text ltx_ref_tag">4.1</span></a> through mathematical reasoning.</p>
</div>
<div id="S5.SS2.SSS1.p3" class="ltx_para ltx_noindent">
<p id="S5.SS2.SSS1.p3.1" class="ltx_p">Given that current LLMs generally share similar architectures and their performance advantage primarily lies in their data, this work is likely to impact the current practices and may potentially serve as a turning point, accelerating the shift in the competitive landscape of LLMs from one that heavily depends on data advantage to one that focuses on more advanced technologies.</p>
</div>
</section>
<section id="S5.SS2.SSS2" class="ltx_subsubsection">
<h4 class="ltx_title ltx_title_subsubsection">
<span class="ltx_tag ltx_tag_subsubsection">5.2.2 </span>Miscellaneous</h4>

<div id="S5.SS2.SSS2.p1" class="ltx_para ltx_noindent">
<p id="S5.SS2.SSS2.p1.1" class="ltx_p">Synthetic data presents a general concern of misinformation and fake news, which has been frequently discussed in previous research <cite class="ltx_cite ltx_citemacro_citep">(Pan et al., <a href="#bib.bib28" title="" class="ltx_ref">2023</a>)</cite>. Persona Hub potentially amplifies this issue, as diverse personas bring diverse writing styles, making machine-generated texts harder to distinguish from human-generated content <cite class="ltx_cite ltx_citemacro_citep">(Chakraborty et al., <a href="#bib.bib9" title="" class="ltx_ref">2023</a>)</cite>. This increased difficulty in detection may worsen issues related to data contamination, where synthetic data is mixed with real data, potentially skewing research results and public information.</p>
</div>
</section>
</section>
</section>
<section id="S6" class="ltx_section">
<h2 class="ltx_title ltx_title_section">
<span class="ltx_tag ltx_tag_section">6 </span>Conclusion and Future Work</h2>

<div id="S6.p1" class="ltx_para ltx_noindent">
<p id="S6.p1.1" class="ltx_p">We propose a novel persona-driven data synthesis methodology and present Persona Hub, a collection of 1 billion diverse personas automatically curated from web data. We show that this methodology can facilitate the scaling of synthetic data creation across various scenarios, demonstrating its potential to revolutionize creation and applications of synthetic data, and its prospects as a general data synthesis engine for both research and practice.</p>
</div>
<div id="S6.p2" class="ltx_para ltx_noindent">
<p id="S6.p2.1" class="ltx_p">As the first version of Persona Hub, although it already contains 1 billion personas, the descriptions of these personas are focused only on major aspects and lack fine-grained details (e.g., preferences for colors and numbers; specific family backgrounds, historical contexts, and life experiences). We plan to refine the personas in subsequent versions of Persona Hub, aiming for their descriptions to be as detailed as those found in Wikipedia articles about individuals. These more detailed persona descriptions will make each persona more unique, thereby scaling up Persona Hub and fostering more opportunities for synthetic data creation, while also empowering practical applications such as personalized conversations (e.g., <a target="_blank" href="https://character.ai/" title="" class="ltx_ref ltx_href">character.ai</a>).</p>
</div>
<div id="S6.p3" class="ltx_para ltx_noindent">
<p id="S6.p3.1" class="ltx_p">Also, while this work only explores data synthesis with text-based LLMs, the methodology should also be applicable to multimodal LLMs. Therefore, we will explore multi-modal synthetic data creation as a future direction. Moreover, given that specific personas can elicit corresponding perspectives from LLMs, we are curious about the possibilities of using some super personas to guide LLMs to explore beyond the scope of existing knowledge. This may provide a new approach to tapping into the super intelligence of LLMs, which will be studied in the future.</p>
</div>
</section>
<section id="bib" class="ltx_bibliography">
<h2 class="ltx_title ltx_title_bibliography">References</h2>

<ul class="ltx_biblist">
<li id="bib.bib1" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">qwe (2024)</span>
<span class="ltx_bibblock">
Qwen2 technical report.

</span>
<span class="ltx_bibblock">2024.

</span>
</li>
<li id="bib.bib2" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Abdin et al. (2024)</span>
<span class="ltx_bibblock">
Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al.

</span>
<span class="ltx_bibblock">Phi-3 technical report: A highly capable language model locally on your phone.

</span>
<span class="ltx_bibblock"><em id="bib.bib2.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2404.14219</em>, 2024.

</span>
</li>
<li id="bib.bib3" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Achiam et al. (2023)</span>
<span class="ltx_bibblock">
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al.

</span>
<span class="ltx_bibblock">Gpt-4 technical report.

</span>
<span class="ltx_bibblock"><em id="bib.bib3.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2303.08774</em>, 2023.

</span>
</li>
<li id="bib.bib4" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bai et al. (2024)</span>
<span class="ltx_bibblock">
Yuelin Bai, Xinrun Du, Yiming Liang, Yonggang Jin, Ziqiang Liu, Junting Zhou, Tianyu Zheng, Xincheng Zhang, Nuo Ma, Zekun Wang, et al.

</span>
<span class="ltx_bibblock">Coig-cqia: Quality is all you need for chinese instruction fine-tuning.

</span>
<span class="ltx_bibblock"><em id="bib.bib4.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2403.18058</em>, 2024.

</span>
</li>
<li id="bib.bib5" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bauer et al. (2024)</span>
<span class="ltx_bibblock">
André Bauer, Simon Trapp, Michael Stenger, Robert Leppich, Samuel Kounev, Mark Leznik, Kyle Chard, and Ian Foster.

</span>
<span class="ltx_bibblock">Comprehensive exploration of synthetic data generation: A survey.

</span>
<span class="ltx_bibblock"><em id="bib.bib5.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2401.02524</em>, 2024.

</span>
</li>
<li id="bib.bib6" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Bi et al. (2024)</span>
<span class="ltx_bibblock">
Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al.

</span>
<span class="ltx_bibblock">Deepseek llm: Scaling open-source language models with longtermism.

</span>
<span class="ltx_bibblock"><em id="bib.bib6.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2401.02954</em>, 2024.

</span>
</li>
<li id="bib.bib7" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Broder (1997)</span>
<span class="ltx_bibblock">
Andrei Z Broder.

</span>
<span class="ltx_bibblock">On the resemblance and containment of documents.

</span>
<span class="ltx_bibblock">In <em id="bib.bib7.1.1" class="ltx_emph ltx_font_italic">Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No. 97TB100171)</em>, pp.  21–29. IEEE, 1997.

</span>
</li>
<li id="bib.bib8" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Cai et al. (2023)</span>
<span class="ltx_bibblock">
Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou.

</span>
<span class="ltx_bibblock">Large language models as tool makers.

</span>
<span class="ltx_bibblock"><em id="bib.bib8.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2305.17126</em>, 2023.

</span>
</li>
<li id="bib.bib9" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chakraborty et al. (2023)</span>
<span class="ltx_bibblock">
Souradip Chakraborty, Amrit Singh Bedi, Sicheng Zhu, Bang An, Dinesh Manocha, and Furong Huang.

</span>
<span class="ltx_bibblock">On the possibilities of ai-generated text detection.

</span>
<span class="ltx_bibblock"><em id="bib.bib9.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2304.04736</em>, 2023.

</span>
</li>
<li id="bib.bib10" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Chen et al. (2022)</span>
<span class="ltx_bibblock">
Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen.

</span>
<span class="ltx_bibblock">Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks.

</span>
<span class="ltx_bibblock"><em id="bib.bib10.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2211.12588</em>, 2022.

</span>
</li>
<li id="bib.bib11" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Choi &amp; Li (2024)</span>
<span class="ltx_bibblock">
Hyeong Kyu Choi and Yixuan Li.

</span>
<span class="ltx_bibblock">Picle: Eliciting diverse behaviors from large language models with persona in-context learning.

</span>
<span class="ltx_bibblock">In <em id="bib.bib11.1.1" class="ltx_emph ltx_font_italic">Forty-first International Conference on Machine Learning</em>, 2024.

</span>
</li>
<li id="bib.bib12" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Computer (2023)</span>
<span class="ltx_bibblock">
Together Computer.

</span>
<span class="ltx_bibblock">Redpajama: an open dataset for training large language models, 2023.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://github.com/togethercomputer/RedPajama-Data" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://github.com/togethercomputer/RedPajama-Data</a>.

</span>
</li>
<li id="bib.bib13" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Delétang et al. (2023)</span>
<span class="ltx_bibblock">
Grégoire Delétang, Anian Ruoss, Paul-Ambroise Duquenne, Elliot Catt, Tim Genewein, Christopher Mattern, Jordi Grau-Moya, Li Kevin Wenliang, Matthew Aitchison, Laurent Orseau, et al.

</span>
<span class="ltx_bibblock">Language modeling is compression.

</span>
<span class="ltx_bibblock"><em id="bib.bib13.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2309.10668</em>, 2023.

</span>
</li>
<li id="bib.bib14" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Dohmatob et al. (2024)</span>
<span class="ltx_bibblock">
Elvis Dohmatob, Yunzhen Feng, Pu Yang, Francois Charton, and Julia Kempe.

</span>
<span class="ltx_bibblock">A tale of tails: Model collapse as a change of scaling laws.

</span>
<span class="ltx_bibblock"><em id="bib.bib14.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2402.07043</em>, 2024.

</span>
</li>
<li id="bib.bib15" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Gandhi et al. (2023)</span>
<span class="ltx_bibblock">
Kanishk Gandhi, Dorsa Sadigh, and Noah D Goodman.

</span>
<span class="ltx_bibblock">Strategic reasoning with language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib15.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2305.19165</em>, 2023.

</span>
</li>
<li id="bib.bib16" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Ge et al. (2024)</span>
<span class="ltx_bibblock">
Tao Ge, Hu Jing, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei.

</span>
<span class="ltx_bibblock">In-context autoencoder for context compression in a large language model.

</span>
<span class="ltx_bibblock">In <em id="bib.bib16.1.1" class="ltx_emph ltx_font_italic">The Twelfth International Conference on Learning Representations</em>, 2024.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=uREj4ZuGJE" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=uREj4ZuGJE</a>.

</span>
</li>
<li id="bib.bib17" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Hendrycks et al. (2021)</span>
<span class="ltx_bibblock">
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt.

</span>
<span class="ltx_bibblock">Measuring mathematical problem solving with the math dataset.

</span>
<span class="ltx_bibblock"><em id="bib.bib17.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2103.03874</em>, 2021.

</span>
</li>
<li id="bib.bib18" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Huang et al. (2024)</span>
<span class="ltx_bibblock">
Yiming Huang, Xiao Liu, Yeyun Gong, Zhibin Gou, Yelong Shen, Nan Duan, and Weizhu Chen.

</span>
<span class="ltx_bibblock">Key-point-driven data synthesis with its enhancement on mathematical reasoning.

</span>
<span class="ltx_bibblock"><em id="bib.bib18.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2403.02333</em>, 2024.

</span>
</li>
<li id="bib.bib19" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Jandaghi et al. (2023)</span>
<span class="ltx_bibblock">
Pegah Jandaghi, XiangHai Sheng, Xinyi Bai, Jay Pujara, and Hakim Sidahmed.

</span>
<span class="ltx_bibblock">Faithful persona-based conversational dataset generation with large language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib19.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2312.10007</em>, 2023.

</span>
</li>
<li id="bib.bib20" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Kaplan et al. (2020)</span>
<span class="ltx_bibblock">
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei.

</span>
<span class="ltx_bibblock">Scaling laws for neural language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib20.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2001.08361</em>, 2020.

</span>
</li>
<li id="bib.bib21" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2024a)</span>
<span class="ltx_bibblock">
Chen Li, Weiqi Wang, Jingcheng Hu, Yixuan Wei, Nanning Zheng, Han Hu, Zheng Zhang, and Houwen Peng.

</span>
<span class="ltx_bibblock">Common 7b language models already possess strong math capabilities.

</span>
<span class="ltx_bibblock"><em id="bib.bib21.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2403.04706</em>, 2024a.

</span>
</li>
<li id="bib.bib22" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2024b)</span>
<span class="ltx_bibblock">
Haoran Li, Qingxiu Dong, Zhengyang Tang, Chaojun Wang, Xingxing Zhang, Haoyang Huang, Shaohan Huang, Xiaolong Huang, Zeqiang Huang, Dongdong Zhang, et al.

</span>
<span class="ltx_bibblock">Synthetic data (almost) from scratch: Generalized instruction tuning for language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib22.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2402.13064</em>, 2024b.

</span>
</li>
<li id="bib.bib23" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2023a)</span>
<span class="ltx_bibblock">
Junyi Li, Ninareh Mehrabi, Charith Peris, Palash Goyal, Kai-Wei Chang, Aram Galstyan, Richard Zemel, and Rahul Gupta.

</span>
<span class="ltx_bibblock">On the steerability of large language models toward data-driven personas.

</span>
<span class="ltx_bibblock"><em id="bib.bib23.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2311.04978</em>, 2023a.

</span>
</li>
<li id="bib.bib24" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Li et al. (2023b)</span>
<span class="ltx_bibblock">
Yuanzhi Li, Sébastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee.

</span>
<span class="ltx_bibblock">Textbooks are all you need ii: phi-1.5 technical report.

</span>
<span class="ltx_bibblock"><em id="bib.bib24.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2309.05463</em>, 2023b.

</span>
</li>
<li id="bib.bib25" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Liu et al. (2024)</span>
<span class="ltx_bibblock">
Ruibo Liu, Jerry Wei, Fangyu Liu, Chenglei Si, Yanzhe Zhang, Jinmeng Rao, Steven Zheng, Daiyi Peng, Diyi Yang, Denny Zhou, et al.

</span>
<span class="ltx_bibblock">Best practices and lessons learned on synthetic data for language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib25.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2404.07503</em>, 2024.

</span>
</li>
<li id="bib.bib26" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Liu et al. (2023)</span>
<span class="ltx_bibblock">
Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang.

</span>
<span class="ltx_bibblock">Dynamic llm-agent network: An llm-agent collaboration framework with agent team optimization.

</span>
<span class="ltx_bibblock"><em id="bib.bib26.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2310.02170</em>, 2023.

</span>
</li>
<li id="bib.bib27" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Maini et al. (2024)</span>
<span class="ltx_bibblock">
Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, and Navdeep Jaitly.

</span>
<span class="ltx_bibblock">Rephrasing the web: A recipe for compute and data-efficient language modeling.

</span>
<span class="ltx_bibblock"><em id="bib.bib27.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2401.16380</em>, 2024.

</span>
</li>
<li id="bib.bib28" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Pan et al. (2023)</span>
<span class="ltx_bibblock">
Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min-Yen Kan, and William Yang Wang.

</span>
<span class="ltx_bibblock">On the risk of misinformation pollution with large language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib28.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2305.13661</em>, 2023.

</span>
</li>
<li id="bib.bib29" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Park et al. (2023)</span>
<span class="ltx_bibblock">
Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein.

</span>
<span class="ltx_bibblock">Generative agents: Interactive simulacra of human behavior.

</span>
<span class="ltx_bibblock">In <em id="bib.bib29.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology</em>, pp.  1–22, 2023.

</span>
</li>
<li id="bib.bib30" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Schick et al. (2024)</span>
<span class="ltx_bibblock">
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom.

</span>
<span class="ltx_bibblock">Toolformer: Language models can teach themselves to use tools.

</span>
<span class="ltx_bibblock"><em id="bib.bib30.1.1" class="ltx_emph ltx_font_italic">Advances in Neural Information Processing Systems</em>, 36, 2024.

</span>
</li>
<li id="bib.bib31" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Shanahan et al. (2023)</span>
<span class="ltx_bibblock">
Murray Shanahan, Kyle McDonell, and Laria Reynolds.

</span>
<span class="ltx_bibblock">Role play with large language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib31.1.1" class="ltx_emph ltx_font_italic">Nature</em>, 623(7987):493–498, 2023.

</span>
</li>
<li id="bib.bib32" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Shumailov et al. (2023)</span>
<span class="ltx_bibblock">
Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson.

</span>
<span class="ltx_bibblock">The curse of recursion: Training on generated data makes models forget.

</span>
<span class="ltx_bibblock"><em id="bib.bib32.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2305.17493</em>, 2023.

</span>
</li>
<li id="bib.bib33" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Team (2024)</span>
<span class="ltx_bibblock">
Qwen Team.

</span>
<span class="ltx_bibblock">Introducing qwen1.5, February 2024.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://qwenlm.github.io/blog/qwen1.5/" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://qwenlm.github.io/blog/qwen1.5/</a>.

</span>
</li>
<li id="bib.bib34" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Travers &amp; Milgram (1977)</span>
<span class="ltx_bibblock">
Jeffrey Travers and Stanley Milgram.

</span>
<span class="ltx_bibblock">An experimental study of the small world problem.

</span>
<span class="ltx_bibblock">In <em id="bib.bib34.1.1" class="ltx_emph ltx_font_italic">Social networks</em>, pp.  179–197. Elsevier, 1977.

</span>
</li>
<li id="bib.bib35" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Villalobos et al. (2024)</span>
<span class="ltx_bibblock">
Pablo Villalobos, Anson Ho, Jaime Sevilla, Tamay Besiroglu, Lennart Heim, and Marius Hobbhahn.

</span>
<span class="ltx_bibblock">Position: Will we run out of data? limits of llm scaling based on human-generated data.

</span>
<span class="ltx_bibblock">In <em id="bib.bib35.1.1" class="ltx_emph ltx_font_italic">Forty-first International Conference on Machine Learning</em>, 2024.

</span>
</li>
<li id="bib.bib36" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wang et al. (2023)</span>
<span class="ltx_bibblock">
Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li.

</span>
<span class="ltx_bibblock">Mathcoder: Seamless code integration in llms for enhanced mathematical reasoning.

</span>
<span class="ltx_bibblock"><em id="bib.bib36.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2310.03731</em>, 2023.

</span>
</li>
<li id="bib.bib37" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wang et al. (2022)</span>
<span class="ltx_bibblock">
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi.

</span>
<span class="ltx_bibblock">Self-instruct: Aligning language models with self-generated instructions.

</span>
<span class="ltx_bibblock"><em id="bib.bib37.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2212.10560</em>, 2022.

</span>
</li>
<li id="bib.bib38" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Wang et al. (2024)</span>
<span class="ltx_bibblock">
Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji.

</span>
<span class="ltx_bibblock">Unleashing the emergent cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration.

</span>
<span class="ltx_bibblock">In <em id="bib.bib38.1.1" class="ltx_emph ltx_font_italic">Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)</em>, pp.  257–279, 2024.

</span>
</li>
<li id="bib.bib39" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Xu et al. (2024)</span>
<span class="ltx_bibblock">
Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli.

</span>
<span class="ltx_bibblock">Hallucination is inevitable: An innate limitation of large language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib39.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2401.11817</em>, 2024.

</span>
</li>
<li id="bib.bib40" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Young et al. (2024)</span>
<span class="ltx_bibblock">
Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, et al.

</span>
<span class="ltx_bibblock">Yi: Open foundation models by 01. ai.

</span>
<span class="ltx_bibblock"><em id="bib.bib40.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2403.04652</em>, 2024.

</span>
</li>
<li id="bib.bib41" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Yu et al. (2023)</span>
<span class="ltx_bibblock">
Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu.

</span>
<span class="ltx_bibblock">Metamath: Bootstrap your own mathematical questions for large language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib41.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2309.12284</em>, 2023.

</span>
</li>
<li id="bib.bib42" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhang et al. (2024)</span>
<span class="ltx_bibblock">
Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Adrian de Wynter, Yan Xia, Wenshan Wu, Ting Song, Man Lan, and Furu Wei.

</span>
<span class="ltx_bibblock">Llm as a mastermind: A survey of strategic reasoning with large language models.

</span>
<span class="ltx_bibblock"><em id="bib.bib42.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2404.01230</em>, 2024.

</span>
</li>
<li id="bib.bib43" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhao et al. (2024)</span>
<span class="ltx_bibblock">
Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng.

</span>
<span class="ltx_bibblock">Wildchat: 1m chatGPT interaction logs in the wild.

</span>
<span class="ltx_bibblock">In <em id="bib.bib43.1.1" class="ltx_emph ltx_font_italic">The Twelfth International Conference on Learning Representations</em>, 2024.

</span>
<span class="ltx_bibblock">URL <a target="_blank" href="https://openreview.net/forum?id=Bl8u7ZRlbM" title="" class="ltx_ref ltx_url ltx_font_typewriter">https://openreview.net/forum?id=Bl8u7ZRlbM</a>.

</span>
</li>
<li id="bib.bib44" class="ltx_bibitem">
<span class="ltx_tag ltx_role_refnum ltx_tag_bibitem">Zhu et al. (2024)</span>
<span class="ltx_bibblock">
Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al.

</span>
<span class="ltx_bibblock">Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence.

</span>
<span class="ltx_bibblock"><em id="bib.bib44.1.1" class="ltx_emph ltx_font_italic">arXiv preprint arXiv:2406.11931</em>, 2024.

</span>
</li>
</ul>
</section>
</article>
</div>
<div class="ar5iv-footer"><a href="/html/2406.20093" class="ar5iv-nav-button ar5iv-nav-button-prev">◄</a>
    <a class="ar5iv-home-button" href="/"><img height="40" alt="ar5iv homepage" src="/assets/ar5iv.png"></a>
    <a href="/feeling_lucky" class="ar5iv-text-button">Feeling<br>lucky?</a>
    <a href="/log/2406.20094" class="ar5iv-text-button ar5iv-severity-error">Conversion<br>report</a>
    <a class="ar5iv-text-button" target="_blank" href="https://github.com/dginev/ar5iv/issues/new?template=improve-article--arxiv-id-.md&title=Improve+article+2406.20094">Report<br>an issue</a>
    <a href="https://arxiv.org/abs/2406.20094" class="ar5iv-text-button arxiv-ui-theme">View&nbsp;original<br>on&nbsp;arXiv</a><a href="/html/2406.20095" class="ar5iv-nav-button ar5iv-nav-button-next">►</a>
</div><footer class="ltx_page_footer">
<a class="ar5iv-toggle-color-scheme" href="javascript:toggleColorScheme()" title="Toggle ar5iv color scheme"><span class="color-scheme-icon"></span></a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/license" target="_blank">Copyright</a>
<a class="ar5iv-footer-button" href="https://arxiv.org/help/policies/privacy_policy" target="_blank">Privacy Policy</a>

<div class="ltx_page_logo">Generated  on Fri Jul  5 17:51:54 2024 by <a target="_blank" href="http://dlmf.nist.gov/LaTeXML/" class="ltx_LaTeXML_logo"><span style="letter-spacing:-0.2em; margin-right:0.1em;">L<span class="ltx_font_smallcaps" style="position:relative; bottom:2.2pt;">a</span>T<span class="ltx_font_smallcaps" style="font-size:120%;position:relative; bottom:-0.2ex;">e</span></span><span style="font-size:90%; position:relative; bottom:-0.2ex;">XML</span><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAsAAAAOCAYAAAD5YeaVAAAAAXNSR0IArs4c6QAAAAZiS0dEAP8A/wD/oL2nkwAAAAlwSFlzAAALEwAACxMBAJqcGAAAAAd0SU1FB9wKExQZLWTEaOUAAAAddEVYdENvbW1lbnQAQ3JlYXRlZCB3aXRoIFRoZSBHSU1Q72QlbgAAAdpJREFUKM9tkL+L2nAARz9fPZNCKFapUn8kyI0e4iRHSR1Kb8ng0lJw6FYHFwv2LwhOpcWxTjeUunYqOmqd6hEoRDhtDWdA8ApRYsSUCDHNt5ul13vz4w0vWCgUnnEc975arX6ORqN3VqtVZbfbTQC4uEHANM3jSqXymFI6yWazP2KxWAXAL9zCUa1Wy2tXVxheKA9YNoR8Pt+aTqe4FVVVvz05O6MBhqUIBGk8Hn8HAOVy+T+XLJfLS4ZhTiRJgqIoVBRFIoric47jPnmeB1mW/9rr9ZpSSn3Lsmir1fJZlqWlUonKsvwWwD8ymc/nXwVBeLjf7xEKhdBut9Hr9WgmkyGEkJwsy5eHG5vN5g0AKIoCAEgkEkin0wQAfN9/cXPdheu6P33fBwB4ngcAcByHJpPJl+fn54mD3Gg0NrquXxeLRQAAwzAYj8cwTZPwPH9/sVg8PXweDAauqqr2cDjEer1GJBLBZDJBs9mE4zjwfZ85lAGg2+06hmGgXq+j3+/DsixYlgVN03a9Xu8jgCNCyIegIAgx13Vfd7vdu+FweG8YRkjXdWy329+dTgeSJD3ieZ7RNO0VAXAPwDEAO5VKndi2fWrb9jWl9Esul6PZbDY9Go1OZ7PZ9z/lyuD3OozU2wAAAABJRU5ErkJggg==" alt="Mascot Sammy"></a>
</div></footer>
</div>

    <script>
      var canMathML = typeof(MathMLElement) == "function";
      if (!canMathML) {
        var body = document.querySelector("body");
        body.firstElementChild.setAttribute('style', 'opacity: 0;');
        var loading = document.createElement("div");
        loading.setAttribute("id", "mathjax-loading-spinner");
        var message = document.createElement("div");
        message.setAttribute("id", "mathjax-loading-message");
        message.innerText = "Typesetting Equations...";
        body.prepend(loading);
        body.prepend(message);

        var el = document.createElement("script");
        el.src = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js";
        document.querySelector("head").appendChild(el);

        window.MathJax = {
          startup: {
            pageReady: () => {
              return MathJax.startup.defaultPageReady().then(() => {
                body.removeChild(loading);
                body.removeChild(message);
                body.firstElementChild.removeAttribute('style');
              }); } } };
      }
    </script>
    <script>
    // Auxiliary function, building the preview feature when
    // an inline citation is clicked
    function clicked_cite(e) {
      e.preventDefault();
      let cite = this.closest('.ltx_cite');
      let next = cite.nextSibling;
      if (next && next.nodeType == Node.ELEMENT_NODE && next.getAttribute('class') == "ar5iv-bibitem-preview") {
        next.remove();
        return; }
      // Before adding a preview modal,
      // cleanup older previews, in case they're still open
      document.querySelectorAll('span.ar5iv-bibitem-preview').forEach(function(node) {
        node.remove();
      })

      // Create the preview
      preview = document.createElement('span');
      preview.setAttribute('class','ar5iv-bibitem-preview');
      let target = document.getElementById(this.getAttribute('href').slice(1));
      target.childNodes.forEach(function (child) {
        preview.append(child.cloneNode(true));
      });
      let close_x = document.createElement('button');
      close_x.setAttribute("aria-label","Close modal for bibliography item preview");
      close_x.textContent = "×";
      close_x.setAttribute('class', 'ar5iv-button-close-preview');
      close_x.setAttribute('onclick','this.parentNode.remove()');
      preview.append(close_x);
      preview.querySelectorAll('.ltx_tag_bibitem').forEach(function(node) {
        node.remove();
      });
      cite.parentNode.insertBefore(preview, cite.nextSibling);
      return;
    }
    // Global Document initialization:
    // - assign the preview feature to all inline citation links
    document.querySelectorAll(".ltx_cite .ltx_ref").forEach(function (link) {
      link.addEventListener("click", clicked_cite);
    });
    </script>
    </body>
</html>

2406.20094
We propose a novel persona-driven data synthesis methodology that leverages various perspectives within a large language model (LLM)  to create diverse synthetic data. To fully exploit this methodology at scale, we introduce Persona Hub – a collection of 1 billion diverse personas automatically curated from web data. These 1 billion personas (13% of the world’s total population) , acting as distributed carriers of world knowledge, can tap into almost every perspective encapsulated within the LLM, thereby facilitating the creation of diverse synthetic data at scale for various scenarios. By showcasing Persona Hub’s use cases in synthesizing high-quality mathematical and logical reasoning problems, instructions (i.e., user prompts) , knowledge-rich texts, game NPCs and tools (functions)  at scale, we demonstrate persona-driven data synthesis is versatile, scalable, flexible, and easy to use, potentially driving a paradigm shift in synthetic data creation and applications in practice, which may have a profound impact on LLM research and development. DISCLAIMER:Persona Hubcan facilitate synthetic data creation at a billion-scale to simulate diverse inputs (i.e., use cases)  from a wide variety of real-world users. If this data is used as input to query a target LLM to obtain its outputs at scale, there isa high riskthat the LLM’s knowledge, intelligence and capabilities will be dumped and easily replicated, thereby challenging the leading position of the most powerful LLMs (e.g., our approach allows a 7B LLM to achieve 65% on MATH, matching the performance ofgpt-4-turbo-preview) . This tech report isfor research purposes only. It is crucial to avoid misuse and ensure ethical and responsible application. We discuss its broad impact and potential concerns in detail in Section5.
abstract
Scaling Synthetic Data Creation with 1,000,000,000 Personas
Abstract
2406.20094
1IntroductionAs synthetic data, typically referring to data generated by models or algorithms rather than directly by humans, becomes increasingly valuedfor training large language models (LLMs) , there is a growing interest in data synthesis using LLMs: by simply specifying a data synthesis prompt, an LLM is expected to produce desirable synthetic data.In practice, however, it is non-trivial to create synthetic data at scale: while we can easily scale up the quantity of synthetic data, it is difficult to ensure its diversity scales up as well. Without considering sampling111Sampling is orthogonal to this work. The diversity it introduces when solely used for data synthesis is usually limited., an LLM can only produce 1 instance given a data synthesis prompt. Therefore, to create diverse synthetic data at scale (e.g., 1 billion diverse math problems) , a large number of diverse prompts are needed.Previous research tends to diversify the data synthesis prompt through the following two paradigms, but unfortunately, neither can practically achieve scalable synthetic data creation:•Instance-driven: This approach diversifies the data synthesis prompt by leveraging a seed corpus (i.e., creating new instances based on the instances in the seed corpus) . Representative studies includeand. However, under this paradigm, the diversity of the synthesized data mainly comes from the seed instances, making it difficult to truly extend beyond the seed corpus. Given the limited size of a seed corpus in most practical scenarios, it is challenging for this paradigm to scale up the creation of synthetic data.•Key-point-driven: This approach diversifies the data synthesis prompt with a curated comprehensive list of key points (or concepts)  that can be a topic, a subject, or any knowledge we expect synthetic data to encompass. Representative studies includeand. However, this methodology also faces difficulties in scaling synthetic data creation: it is practically prohibitive to curate a comprehensive list by enumerating all key points across different levels of granularity, unless limited to a narrow and specific domain (e.g., mathematics) .To practically achieve diverse synthetic data creation at scale, we propose a novelpersona-drivendata synthesis methodology. This is inspired by the observation that simply adding a persona to a data synthesis prompt can steer the LLM towards the corresponding perspective to create distinctive synthetic data, as shown in Figure1. Since almost any LLM use case can be associated with a specific persona, we can create all-encompassing synthetic data at scale as long as we construct a comprehensive persona collection.Figure 2:From a compression perspective, Persona Hub (tokens)  can be seen as the compressed form of world knowledge (public web text for training LLMs,tokens)  into distributed carriers. On the other hand, the public web text can be seen as the decompressed content created by these personas with their knowledge and experiences.Fortunately, personas are very easy to scale up. From massive web data, we automatically construct Persona Hub — a persona collection containing 1 billion diverse personas (13% of the world’s total population) . As Figure2shows, these 1 billion personas can be regarded as distributed carriers of world knowledge, and each individual can be associated with their unique knowledge, experience, interest, personality and profession; thus, they can tap into almost every perspective encapsulated within the LLM to create diverse synthetic data at scale, without being limited by the size of a seed corpus. Moreover, in contrast to key points that typically work with specific data synthesis prompts, personas can be combined with almost any data synthesis prompt, benefiting from an LLM’s strong roleplay ability, making them generally applicable to a variety of data synthesis scenarios.We showcase Persona Hub’s use cases in large-scale creation of math and logical reasoning problems, instructions (i.e., user prompts) , broad-coverage knowledge-rich texts, game NPCs, and tool (function)  development. We demonstrate that persona-driven data synthesis is versatile, scalable, flexible, and easy to use, potentially driving a paradigm shift in synthetic data creation and applications in practice, which may have a profound impact on
LLM research and development.To facilitate research in persona-driven data synthesis, we initially release200,000 personasfrom Persona Hub and following synthetic data samples we created with various personas, including:•50,000 math problems•50,000 instructions•10,000 game NPCs•50,000 logical reasoning problems•10,000 knowledge-rich texts•5,000 tools (functions) We are open to releasing more data when we can better assess the potential risks and concerns, which will be discussed in detail in Section5.Note:Our proposed methodology is applicable to almost any popular LLM222We mainly use publicly available LLMs such as GPT-4, Llama-3 and Qwenin our experiments.. The prompts shown in the figures throughout this paper are not exactly the prompt strings we used in our experiments; instead, they are simplified to fit the space and better illustrate the concepts. Interested readers can easily verify our methodology using the persona samples we have released. It is also worth noting that the main focus of this work is on creating new synthetic data, unlike much previous research that focuses on generating synthetic outputs for specific inputs (e.g., a math problem) . Therefore, we use the terms “create” and “synthesize” interchangeably throughout the paper.
introduction
Scaling Synthetic Data Creation with 1,000,000,000 Personas
1Introduction
2406.20094
2Persona HubWe propose two scalable approaches to derive diverse personas to construct Persona Hub from massive web data:Text-to-PersonaandPersona-to-Persona.Figure 3:TheText-to-Personaapproach: it can use any text as input to obtain corresponding personas just by prompting the LLM “Who is likely to [read|write|like|dislike|…] the text?”2.1Text-to-PersonaA person with specific professional experiences and cultural backgrounds will have unique interests in reading and writing. Therefore, from a specific text, we can infer a specific persona who is likely to [read|write|like|dislike|…] the text. Given that text data on the web is virtually unlimited and all-encompassing, we can obtain a wide-ranging collection of personas simply by prompting an LLM with these web texts, as shown in Figure3.There are many formats (e.g., plain text or structured text)  to represent a persona, which can be controlled within the prompt. The granularity of an output persona description can also be adjusted through the prompt. For example, in the first case, a coarse-grained persona might be “a computer scientist”, whereas the fine-grained persona is “a machine learning researcher focused on neural network architectures and attention mechanisms”. In our practice, we ask the LLM (in the prompt)  to output persona descriptions as specifically as possible. Besides specifying the granularity of persona descriptions in the prompt, input texts can also influence the granularity of persona descriptions. As shown in Figure4, if an input text (e.g., from a mathematical textbook or an academic paper about superconductivity)  contains many detailed elements, the resulting persona description will also be specific and fine-grained. Therefore, by applying theText-to-Personaapproach to massive web text data, we can obtain billions (or even trillions)  of diverse personas, encompassing a wide range of aspects across different granularities.Figure 4:Persona descriptions will be fine-grained if input texts involve many detailed elements.2.2Persona-to-PersonaAs discussed above,Text-to-Personais a highly scalable method that can synthesize personas covering almost every aspect. However, it may still miss some personas that have low visibility on the web and thus are less likely to obtain viaText-to-Persona, such asa child,a beggar, ora behind-the-scenes crew member of a movie. To supplement the personas thatText-to-Personamight hardly reach, we proposePersona-to-Persona, which derives personas with interpersonal relationships from those obtained throughText-to-Persona.Figure 5:Persona-to-Personaobtains diverse personas via interpersonal relationships, which can be easily achieved by prompting the LLM “Who is in close relationship with the given persona?”As shown in Figure5, the persona about “a child” can be derived from the persona of a nurse at a children’s hospital (patient-caregiver relationship) . Similarly, “a beggar” can be derived from the persona of a shelter worker (assistance relationship) , and “a behind-the-scenes movie crew member” can be derived from the persona of the movie’s lead actor (co-worker relationship) . According to the six degrees of separation theory, we perform six iterations of persona relationship expansion for each persona obtained throughText-to-Persona, thereby enriching our persona collection even further.2.3DeduplicationWe first runText-to-Personaon the RedPajama v2 datasetand then performPersona-to-Persona, as described in Sections2.1and2.2. After obtaining billions of personas, it is inevitable that some of the personas will be identical or extremely similar. To ensure the diversity of Persona Hub, we deduplicate these personas in two ways:MinHash-based DeduplicationWe use MinHashto deduplicate based on the n-gram features of persona descriptions. Since persona descriptions are usually just 1-2 sentences, much shorter than a document, we simply used 1-gram and a signature size of 128 for MinHash deduplication. We deduplicate at the similarity threshold of 0.9.Embedding-based DeduplicationAfter deduplication based on surface forms (i.e., MinHash with n-gram features) , we also adopt embedding-based deduplication. We use a text embedding model (e.g., thetext-embedding-3-smallmodel from OpenAI)  to compute an embedding for each persona, and then filter out personas with a cosine semantic similarity greater than 0.9.Note that although we select 0.9 as the threshold here, we can flexibly adjust it according to specific needs for further deduplication. For instance, when the requirement for the number of instances is not high (e.g., only needing 1 million instances)  but the demand for diversity is high, we can further apply a stricter deduplication standard (e.g., discarding personas with a similarity greater than 0.5) .After deduplication and using simple heuristic methods to filter out low-quality persona descriptions, we have harvested a total of 1,015,863,523 personas, finally forming our Persona Hub.
body
Scaling Synthetic Data Creation with 1,000,000,000 Personas
2Persona Hub
2406.20094
3Persona-driven Synthetic Data CreationOur proposed persona-driven data synthesis approach is straightforward and effective, which involves integrating a persona into the appropriate position in a data synthesis prompt. Simple as it appears, it can significantly influence the LLM to adopt the persona’s perspective to create synthetic data. Driven by the 1 billion personas in Persona Hub, this approach can easily create diverse synthetic data at a billion scale.Figure 6:0-shot, few-shot and persona-enhanced few-shot prompting methods.Just as we can use either zero-shot or few-shot methods to prompt an LLM, the persona-driven methodology is also flexible and compatible with various forms of prompts to create synthetic data. As shown in Figure6, we propose three persona-driven data synthesis prompting methods:•Zero-shot promptingdoes not leverage any existing examples (i.e., demonstrations) , thereby fully exploiting the model’s creativity without being constrained by specific examples.•Few-shot promptingcan better ensure that the synthesized data meets the requirements by providing some demonstrations.•Persona-enhanced few-shot promptingis more effective in enhancing the LLM’s persona-driven data synthesis capabilities. However, its drawback is that it requires deriving the corresponding persona for each demonstration in the few-shot prompt beforehand.
body
Scaling Synthetic Data Creation with 1,000,000,000 Personas
3Persona-driven Synthetic Data Creation
2406.20094
4Use CasesWe demonstrate the use cases of Persona Hub in various data synthesis scenarios, including the large-scale creation of math and logical reasoning problems, instructions (i.e., user prompts) , knowledge-rich texts, game NPCs, and tool (function)  development.As mentioned earlier, the persona-driven approach is general and versatile, making it easily adaptable to different data synthesis scenarios simply by adjusting the data synthesis prompt. Therefore, we will provide a detailed technical discussion only for math problem synthesis (Section4.1)  and skip the detailed discussion for other use cases.4.1Math Problems4.1.1DemonstrationsAs the initial example (Figure1)  shows, when prompting an LLM to create a math problem, adding a persona leads the LLM to create math problems related to that persona. The example in Figure7(left)  further confirms this: when presented with a linguist persona, the LLM will create a math problem in the context of computational linguistics. Moreover, adding a persona does not hinder the flexibility of the prompt – we can still easily specify the focus (Figure7(middle) )  or difficulty (Figure7(right) )  of our desired math problem in the prompt.Figure 7:A linguist persona with different math problem creation prompts that specify the focus (e.g., geometry)  or the difficulty (e.g., Olympiad-level) Figure 8:Examples of math problems created with personas of professionals related to the field of mathematics. They tend to be more challenging than those created with general personas because they usually require a deeper and more fine-grained understanding of advanced mathematical knowledge and skills.The examples in Figure7demonstrate the use of general personas to create math problems. We can certainly employ professionals related to mathematics to create math problems as well. As shown in Figure8, personas of math professionals333We can easily harvest a large number of such personas when runningText-to-Persona(Section2.1)  on public web texts, particularly when processing texts in the field of mathematics.often mention more advanced and granular mathematics knowledge and skills (as discussed earlier in Section2.1and Figure4) , which in turn allows the created math problems to cover these mathematical concepts, making them more challenging.4.1.2EvaluationDataWe select444Technically, if we use the full version of Persona Hub, we can obtain 1 billion math problems synthesized by an LLM. However, due to the cost of GPT-4 APIs, we limit our scaling to 1.09M personas for this experiment.1.09 million personas from Persona Hub and employ the 0-shot prompting method using GPT-4 to create math problems with these personas, which does not leverage any instances from benchmarks like MATHduring the creation of math problems. This approach allowed us to synthesize 1.09M math problems. Since this work focuses on creating new synthetic data rather than synthesizing solutions, we simply usedgpt-4o(assistant555Assistant system message in OpenAI API doc: “You are a helpful assistant.”)  to generate solutions to the created problems. Among these 1.09M math problems, we randomly hold out 20k as a synthetic test set to facilitate evaluation. The remaining 1.07M problems are used for training.Test setsWe use the following two test sets for evaluation:•Synthetic Test Set(In-distribution) : Since the set of the held-out 20K problems is produced in the same way as the 1.07M training instances, it can be considered an in-distribution test set. To ensure the accuracy of the answers in this test set for increasing the reliability of the evaluation, we additionally generate solutions usinggpt-4o(PoT666Program of thought prompting)  andgpt-4-turbo(assistant)  in addition to the solution generated bygpt-4o(assistant) . We retain only the test instances where at least two solutions are consistent. The remaining test set consists of 11.6K test instances.•MATH(Out-of-distribution) : The most widely recognized benchmark for testing the mathematical reasoning ability of LLMs. Its test set contains 5,000 competitive-level math problems with reference answers. Since we do not use any instances from the MATH dataset for data synthesis or training, we regard the MATH test set as an out-of-distribution test set.Equality CheckingWe follow the same evaluation protocol as OpenAI777https://github.com/openai/simple-evalsto check answer equality on the MATH benchmark. For the synthetic test set, we use a similar method, except we use Llama-3-70B-Instruct instead ofgpt-4-turbo-previewas the equality checker.We simply fine-tune the latest open-sourced 7B LLM – Qwen2-7Bwith our synthesized 1.07 million math problems and evaluate its greedy decoding outputs on the above two test sets.ModelModel SizeAccuracy (%) Open-sourced LLMsDeepSeek LLM 67B Chat67B53.2Phi-3-Mini-4K-Instruct3.8B68.3Yi-1.5-34B-Chat34B70.4Qwen1.5-72B-Chat72B60.7Qwen1.5-110B-Chat110B73.0Qwen2-7B-Instruct7B72.1Qwen2-72B-Instruct72B77.2Llama-3-8B-Instruct8B39.8Llama-3-70B-Instruct70B63.5GPT-4gpt-4-turbo-2024-04-09?88.1gpt-4o-2024-05-13?91.2This workQwen2-7B (fine-tunedw/the 1.07M synthesized instances) 7B79.4Table 1:In-distribution evaluation results on the 11.6K synthetic test instances.ModelModel SizeAccuracy (%) State-of-the-art LLMsgpt-4o-2024-05-13?76.6gpt-4-turbo-2024-04-09?gpt-4-turbo-0125-preview?gpt-4-turbo-1106-preview?gpt-4?∗Claude 3.5 Sonnet?∗Claude 3 Opus?Gemini Pro 1.5 (May 2024) ?∗Gemini Ultra?∗DeepSeek-Coder-V2-Instruct236B/21B∗Llama-3-70B-Instruct70BQwen2-72B-Instruct72B∗Qwen2-7B-Instruct7B∗This workQwen2-7B (fine-tunedw/the 1.07M synthesized instances) 7B64.9Table 2:Out-of-distribution evaluation on MATH. Results marked with an asterisk (*)  may not use the OpenAI’s evaluation method. The model fine-tuned with our synthesized 1.07M math problems achieves 64.9% on MATH, matching the performance ofgpt-4-turbo-previewat only a 7B scale.Figure 9:Accuracy on MATH with scaling the synthetic instances used for training Qwen2-7BTable1presents the in-distribution (ID)  evaluation results on the 11.6K synthetic test instances. Among the tested open-source LLMs, Qwen2-72B-Instruct achieves the best result, and the ranking of the other models is generally consistent with their reported performance on other mathematical benchmarks. Our model, with the help of the 1.07M synthetic math problems, achieves nearly 80% accuracy, surpassing all the open-source LLMs. However, considering that the answers in the synthetic test are not absolutely reliable and that our model might be the only one using ID training data, this ID evaluation results should be taken as a reference only.We present the evaluation results on MATH in Table2. The 7B model fine-tuned with the synthetic training data achieved an impressive 64.9% accuracy on MATH simply using greedy decoding, outperformed only bygpt-4o,gpt-4-turbo-2024-04-09, Claude 3.5 Sonnet, Gemini Pro 1.5 (May 2024)  and DeepSeek-Coder-V2-Instruct.Figure9presents the performance of the model on MATH when trained with synthetic math problems at different scales. Its performance trend generally aligns with the scaling law. Unlike previous researchthat performs scaling on in-distribution data (e.g., heavily relying on MATH train data to augment in-distribution data) , we did not use any instances from MATH during data synthesis or training. Therefore, in this out-of-distribution (OOD)  evaluation setting, achieving performance on MATH that surpassesgpt-4-turbo-preview (1106/0125) is indeed impressive and promising for a 7B model.Figure 10:Similarities of math problems created by personas with different similarities:(a) Similarity of math problems when no specific focus is given;(b) Similarity of math problems when the prompt specifies they must be related to finance and probability;(c) Similarity of math problems synthesized bygpt-4oandgpt-35-turbowith persona similarity of 0.9.We examine the quality of our synthesized math problems: we sample 200 challenging problems (involving high school and university-level math knowledge points in China) , and have two math experts evaluate their validity. Only 7 out of 200 problems are marked as invalid (e.g., due to insufficient or conflicting conditions) , yielding a reliable validity rate of 96.5%.Moreover, we specifically examine the impact of differences in personas within the prompts on the synthesized math problems. We first sample 100 pairs of personas with semantic similarities888We use OpenAI’stext-embedding-3-small(dim=512)  to obtain the semantic representation and compute cosine similarity in this experiment. Here, a persona similarity of 0.4 means that a pair of personas has a semantic similarity within the range of 0.39 to 0.41.of 0.4, 0.6, and 0.8, respectively. For each pair of personas, we use them to create a pair of math problems using greedy decoding (i.e., temperature=0) . Then, we compute the semantic similarity of these math problem pairs and show the results in Figure10.We can clearly observe that the semantic similarity between synthesized math problems tends to be correlated with but lower than the similarity between their corresponding personas. When we add more specific constraints to the prompts (e.g., math problems about finance and probability) , the similarity between the synthesized math problems tends to become higher (Figure10(b) ) . In Figure10(c) , we also test the similarity of math problems created bygpt-4oandgpt-35-turbousing highly similar personas (similarity=0.9) . The results indicate that the semantic similarity of the math problems created bygpt-4oandgpt-35-turboseems not significantly different: most synthesized math problems’ similarity falls within the range of 0.6 to 0.75, which is much lower than the similarity of the personas (0.9) . Given these observations, we believe that using the personas in Persona Hub can ensure the diversity of synthesized data – even at a billion scale.4.2Logical Reasoning ProblemsFigure 11:Logical reasoning problems created by our proposed persona-driven methodologyFigure 12:Ruozhiba-style logical reasoning problems created with various personas. Note that Logical Prompt 3 in this figure is a simplified prompt. In practice, we need to specifically define a Ruozhiba-style logical reasoning problem in this prompt in order to obtain desired synthetic data.Similar to math problems, logical reasoning problems can also be easily synthesized. We present examples of typical logical reasoning problems synthesized using our proposed persona-driven methodology in Figure11.Figure 13:Two typical prompts used for creating instructions (i.e., user prompts) .Moreover, we also show several Ruozhiba-style999Ruozhibais a subforum onBaidu Tieba, featuring numerous intricate and challenging questions posted by Chinese netizens. These questions combine elements such as puns, polysemy, causal inversion, and homophones, embedding logical traps that rigorously test the ability to understand complex Chinese language constructs. Recent researchhas demonstrated that these data significantly benefit the improvement of LLMs’ logical reasoning abilities.logical reasoning problems created with personas in Figure12. All the examples demonstrate that as long as we can clearly describe the requirements for the logical reasoning problem to be created, we can use a large variety of personas to steer the LLM to generate diverse logical reasoning problems that not only meet the requirements but are also highly relevant to the personas, even for whimsical Ruozhiba-style problems.For more examples, please refer to the 50,000 synthetic reasoning problems we have released.4.3InstructionsThe end users of LLMs are ultimately humans. We can use Persona Hub to simulate a variety of users to understand their typical requests for LLM assistance, resulting in diverse instructions (i.e., user prompts) .Figure13shows two typical persona-driven prompts for synthesizing instructions, corresponding to the zero-shot prompting and persona-enhanced few-shot prompting methods described in Section3. The zero-shot method does not rely on any existing instruction dataset and allows the LLM to generate various instructions based on different personas. In contrast, the persona-enhanced few-shot method requires existing instruction datasets (e.g., we use WildChatin our experiments)  to sample some instructions as demonstrations and involves inferring the associated personas of these instructions through theText-to-Personamethod described in Section2.1. While this approach is more complex, it results in synthesized instructions that more closely resembles instructions from real users.With diverse instructions created using Persona Hub, which typically represent the first turn of a user-LLM conversation, we can easily generate their subsequent conversational turns using an LLM, resulting in a large number of simulated user-LLM conversations, which will be valuable for enhancing the LLM’s instruction-following and conversational abilities. Furthermore, we can even adopt a similar approach by selecting two personas from Persona Hub and having LLMs role-play both, thereby simulating conversationsbetween two real people.As Figure1has already shown some instructions created using this methodology, we skip showing examples here. Readers interested in more examples can refer to the released 50,000 instructions synthesized through 0-shot and persona-enhanced 2-shot prompting.4.4Knowledge-rich TextsFigure 14:Examples of knowledge-rich plain text synthesis with personasIn addition to synthesizing instructions that can enhance the instruction tuning of LLMs, the persona-driven methodology can be easily adapted to create knowledge-rich plain text that benefits pre-training and post-training of LLMs. As illustrated in Figure14, we can prompt an LLM to write a Quora101010Quorais a popular question-and-answer website where users can ask questions and provide answers on a wide range of topics. Articles (i.e., posts)  on Quora are often written by knowledgeable individuals, including experts in various fields, ensuring high-quality, well-researched, and informative content.article111111The methods for synthesizing knowledge-rich plain text are not limited to having the LLM write Quora articles. For instance, we can also prompt the LLM to synthesize (educational)  reading material that a persona may be interested in, thereby obtaining a large amount of knowledge-rich text.using a persona sampled from Persona Hub. This approach elicits the LLM’s corresponding knowledge and perspective, resulting in highly informative and knowledge-rich content. By scaling this process with 1 billion personas in Persona Hub, we can easily obtain a vast array of knowledge-rich texts that cover almost any topic across various levels of granularity.4.5Game NPCsA straightforward and practical application of Persona Hub is creating diverse NPCs (Non-Player Characters)  at scale for games. As long as we can provide a game’s background and world-building information to the LLM, we can prompt the LLM to project personas from Persona Hub (which are typically real-world personas)  into characters within the game’s world. In this way, we can significantly reduce the effort required for brainstorming NPCs during the game design process.Figure 15:NPC creation for the game “World of Warcraft” using personas in Persona HubFigure15and16show concrete examples where we use personas in Persona Hub to create game NPCs for the game “World of Warcraft121212World of Warcraft, developed by Blizzard Entertainment, is a highly influential MMORPG with millions of active players worldwide, spanning over 100 countries since its release in 2004.” and “Moonlight Blade131313Moonlight Blade (天涯明月刀) is a 3D martial arts-themed MMORPG developed by Tencent, officially launched in China on July 1, 2016.”.Figure 16:NPC creation for the game “Moonlight Blade (天涯明月刀) ” using Persona Hub4.6Tool (Function)  DevelopmentFigure 17:Examples of tool (function)  creation with Persona HubAs Section4.3demonstrates, Persona Hub can be used to simulate a wide variety of real users to anticipate their possible requests (i.e., instructions)  to an LLM. Similarly, we can use Persona Hub to predict the toolsthat users might need, so that we can pre-build these tools (functions)  beforehand. When a real user makes a similar request, the LLM can directly call these pre-built tools to return results without having to build tools from scratch each time. This paradigm, introduced by Persona Hub, is a completely new solution that allows LLMs to better serve users. We believe it will have great potential in the future as LLMs become more democratized and multifunctional.Figure17shows examples of tools created with various personas. These tools provide functionalities that the personas may need (e.g., a cab driver needs to check traffic conditions)  but cannot be accessed by an LLM, greatly expanding the range of services provided by the LLM. Note that although the tools in Figure17are just interface definitions, these definitions can be easily converted into code implementations, as shown in Figure18.Figure 18:The interface definitions (e.g., the species identification interface)  in Figure17can be easily converted into code implementations by calling an LLM to implement them. The resulting pre-built tools can then be directly utilized by the LLM in the future, eliminating the need to build them from scratch each time.
body
Scaling Synthetic Data Creation with 1,000,000,000 Personas
4Use Cases
2406.20094
5Broad Impact and Ethical Concerns5.1Broad Impact5.1.1Paradigm Shift in Data Creation by Humans and LLMsTraditionally, it has been widely accepted that while LLMs excel at processing data (e.g., rewriting, annotation, or generating outputs/solutions to specific inputs) , they are not particularly adept at creating new data. Consequently, the task of data creation has still largely been the domain of humans, and the collaboration paradigm between humans and LLMs has always been humans creating data and LLMs processing it. However, the introduction of our proposed persona-driven methodology potentially revolutionizes this paradigm. With Persona Hub, LLMs are no longer confined to processing existing data; they can now create various types of new data from a multitude of perspectives, much like the diverse population of the world.While the current capabilities of LLMs may not yet fully replace humans in fulfilling the mission of data creation—whether in terms of data quality or breadth—the ongoing advancements in LLM capabilities suggest a future where LLMs will increasingly excel in data creation. As LLMs continue to improve, both the quality and breadth of the data they can create will also likely enhance, leading us to a point where LLMs may fully take on the role of data creation. When this day arrives, we will no longer be constrainedby the limited high-quality human-produced real-world data141414https://lilianweng.github.io/posts/2024-02-05-human-data-quality/. Persona Hub ensures the diversity and coverage of synthetic data, significantly mitigating concerns about the negative impactsof synthetic data on model training. This may effectively eliminate the data bottleneck, thereby pushing the scaling law to its limit.5.1.2Reality SimulationIn Section4.3and4.6, we have demonstrated that Persona Hub can represent a vast array of real-world individuals with its 1 billion personas. By employing these personas to simulate and infer the potential needs and behaviors of real users, we can not only allow LLMs to autonomously prepare for upcoming use cases (queries) , but also pave the way for LLMs to effectively mimic the real world, thereby creating many new opportunities. For instance, companies can use this method to predict how different types of users might react to a new product launch; governments can foresee the public’s response to new legislation, considering various population group; in online services that require user profiling and behavior modeling, Persona Hub can facilitate the simulation of diverse user behaviors, significantly alleviating the cold start challenge.Recent research on LLM roleplay, agent collaboration, strategic reasoning, and related areas can, of course, be facilitated by the vast and diverse personas in Persona Hub. More ambitiously, the 1 billion personas can even sustain a well-organized society within a virtual world, such as sandbox environments, online games, parallel worlds, or the metaverse, using the method discussed in Section4.5to simulate operations with powerful LLMs. This virtual society can serve as a testing ground for new policies, radical initiatives, and social dynamics, providing valuable insights before real-world implementation. By creating a controlled environment where diverse personas interact, we can observe emergent behaviors, test hypotheses, and refine strategies in a risk-free setting. This can not only help deepen our understanding of complex systems but also speed up innovation by facilitating rapid iteration and experimentation.5.1.3Full Memory Access of LLMsWhen we interact with an LLM in a specific scenario, we can only elicit a fraction of its memory and capabilities, and we are unable to fully access the vast knowledge encapsulated within the LLM. However, Persona Hub potentially offers access to the full memory of an LLM because the 1 billion personas in Persona Hub can tap into almost every perspective and piece of information encoded within the LLM.By leveraging these 1 billion personas, we can create diverse queries and obtain solutions from a target LLM, thereby transforming the LLM’s comprehensive memory (parameters)  into synthetic data in textual form. If we consider an LLM as a parameterized compression of world knowledge, then Persona Hub can be viewed as a distributed carrier-based compression151515As Figure2illustrates, Persona Hub has on the order oftokens, equivalent to acompression of the public web text (tokens) .of world knowledge, as demonstrated in Figure2. This distributed carrier-based compression provides us with an opportunity to decompress the LLM’s parameters back into world knowledge and information it has ever learned (e.g., using the method discussed in Section4.4) .However, considering that the current Persona Hub is still in a very preliminary stage and that today’s LLMs are not yet capable of losslessly converting their memory into synthetic data due to inevitable hallucination, the breadth and quality of the synthetic data generated through this methodology are still limited. Nevertheless, as Persona Hub continues to improve and scale, and as LLMs become more powerful (with less hallucination) , we can look forward to a day when it will be possible to nearly losslessly extract the full memory of an LLM into plain text.5.2Ethical Concerns5.2.1Training Data Security and Threats to Current LLM DominanceAs discussed in Section5.1.3, Persona Hub offers an opportunity to access the full memory of a target LLM. However, this also introduces a significant issue: the security of the training data. All data synthesized through the target LLM essentially represents a form of its seen training data. Therefore, the process of extensively extracting a target LLM’s memory is essentially dumping its training data, even though this process is generally lossy.Moreover, if we employ the method described in Section4.3to synthesize instructions (i.e., user prompts)  that nearly covers all use cases to query a target LLM to obtain its outputs at scale, there is a high risk that the target LLM’s knowledge, intelligence, and capabilities could be extracted and replicated. This poses a challenge to the leading position of the most powerful LLMs, as we have already validated in Section4.1through mathematical reasoning.Given that current LLMs generally share similar architectures and their performance advantage primarily lies in their data, this work is likely to impact the current practices and may potentially serve as a turning point, accelerating the shift in the competitive landscape of LLMs from one that heavily depends on data advantage to one that focuses on more advanced technologies.5.2.2MiscellaneousSynthetic data presents a general concern of misinformation and fake news, which has been frequently discussed in previous research. Persona Hub potentially amplifies this issue, as diverse personas bring diverse writing styles, making machine-generated texts harder to distinguish from human-generated content. This increased difficulty in detection may worsen issues related to data contamination, where synthetic data is mixed with real data, potentially skewing research results and public information.
body
Scaling Synthetic Data Creation with 1,000,000,000 Personas
5Broad Impact and Ethical Concerns
2406.20094
6Conclusion and Future WorkWe propose a novel persona-driven data synthesis methodology and present Persona Hub, a collection of 1 billion diverse personas automatically curated from web data. We show that this methodology can facilitate the scaling of synthetic data creation across various scenarios, demonstrating its potential to revolutionize creation and applications of synthetic data, and its prospects as a general data synthesis engine for both research and practice.As the first version of Persona Hub, although it already contains 1 billion personas, the descriptions of these personas are focused only on major aspects and lack fine-grained details (e.g., preferences for colors and numbers; specific family backgrounds, historical contexts, and life experiences) . We plan to refine the personas in subsequent versions of Persona Hub, aiming for their descriptions to be as detailed as those found in Wikipedia articles about individuals. These more detailed persona descriptions will make each persona more unique, thereby scaling up Persona Hub and fostering more opportunities for synthetic data creation, while also empowering practical applications such as personalized conversations (e.g.,character.ai) .Also, while this work only explores data synthesis with text-based LLMs, the methodology should also be applicable to multimodal LLMs. Therefore, we will explore multi-modal synthetic data creation as a future direction. Moreover, given that specific personas can elicit corresponding perspectives from LLMs, we are curious about the possibilities of using some super personas to guide LLMs to explore beyond the scope of existing knowledge. This may provide a new approach to tapping into the super intelligence of LLMs, which will be studied in the future.
conclusion
Scaling Synthetic Data Creation with 1,000,000,000 Personas
6Conclusion and Future Work