Improving Diversity of Commonsense Generation by Large Language Models via In-Context Learning
Tianhui Zhang Bei Peng Danushka Bollegala

Atmospheric Retrievals of the Phase-resolved Spectra of Irradiated Brown Dwarfs WD-0137B and EPIC-2122B
    [
    April 29, 2024
=======================================================================================================




GCR requires a model to reason about a situation using commonsense knowledge, while generating coherent sentences. 
Although the quality of the generated sentences is crucial, the diversity of the generation is equally important because it reflects the model's ability to use a range of commonsense knowledge facts. 
LLM have shown proficiency in enhancing the generation quality across various tasks through ICL using given examples without the need for any fine-tuning.  
However, the diversity aspect in LLM outputs has not been systematically studied before. 
To address this, we propose a simple method that diversifies the LLM generations, while preserving their quality. 
Experimental results on three benchmark GCR datasets show that our method achieves an ideal balance between the quality and diversity. 

Moreover, the sentences generated by our proposed method can be used as training data to improve diversity in existing commonsense generators.







§ INTRODUCTION



Commonsense reasoning is the ability to make logical deductions about concepts  encountered in daily life, and is considered as a critical property of intelligent agents <cit.>.
Concepts are mental representations of classes and are expressed using words in a language <cit.>.
Given the inputs, the GCR task requires a model to generate a high quality sentence that is grammatical and adheres to commonsense, evaluated by its similarity to a set of human-written reference sentences covering the same set of concepts <cit.>.



Often there exists multiple relationships between a given set of concepts, leading to alternative reasoning paths that take diverse view points.
For example, given the four concepts dog, frisbee, throw and catch, different sentences can be generated as shown in <ref>.
Although all sentences shown in <ref> are grammatical, the bottom set expresses diverse view points (e.g. from the dog's as well as the man's) compared to the set at the top.
Apart from the generation quality, diversity is also an important factor in text generation because the low-diversity texts tend to be dull, repetitive or biased towards a particular view point <cit.>. 
Diversity is an important consideration in many NLG applications, such as story generation <cit.>, paraphrase generation <cit.>, and GCR <cit.>. 
In GCR tasks, the input text often provides insufficient information to support diverse reasoning and generate multiple plausible outputs. Therefore, the diversity present in GCR task enables the exploration of different perspectives or all possible outcomes for a real-world situation.
Existing methods promote diversity through special decoding strategies, such as nucleus sampling <cit.>, or encoding interventions such as random noise injection <cit.> or MoE approaches <cit.>.

We propose Prop, a computationally-efficient and accurate method to improve the diversity in GCR, where the sentences are generated from a pre-trained LLM, and strikes a fine-balance between the output diversity and quality.  
Prop uses an ICL approach to increase the diversity of the sentences generated by an LLM, while maintaining the quality of the generation.
Prop is a two-step process where it first lets an LLM to freely generate high-quality sentences that are grammatical, commonsense bearing and cover all the given input concepts.
Next, Prop uses a user-specified diversity metric to evaluate the diversity of the generated sentences.
If the diversity is low, Prop provides feedback to the LLM, instructing it to generate more diverse sentences considering the already generated sentences.

Given that Prop is using LLMs to generate diverse sentences via ICL and without updating the parameters of the LLMs, an interesting and open question is whether an LLM can accurately judge the diversity of a given set of sentences, covering a common set of concepts.
To answer this question, we conduct an experiment where we instruct  to judge the diversity of the set of input sentences according to a five-scale grading system, and convert the predicted grades into binary judgements (i.e. diverse vs. non-diverse).
We compare the LLM-assigned grades against those by a group of human annotators, and find a 
moderate-level (Cohen's Kappa of 0.409) agreement between human vs. LLM judgements, demonstrating that LLMs can indeed be instructed to obtain diversity judgements for GCR tasks.

We evaluate Prop on three GCR tasks/datasets: CommonGen <cit.>, ComVE <cit.>, and DimonGen <cit.>. 
We find that our proposed Prop balances diversity and quality appropriately, improving their harmonic mean by at least 6% over that of a  baseline. 
Moreover, the sentences generated by Prop can be used as training data to improve diversity in a Seq2Seq model <cit.>, producing results that are comparable to the models that are trained on knowledge graphs or human-written text corpora <cit.>. 



























§ RELATED WORK





  
Diverse Text Generation.
 A variety of methods have been proposed to enhance the diversity of NLG. 
 Sampling-based decoding is an effective method to increase the generation diversity. 
 <cit.> proposed nucleus sampling to generate diverse content at the generation stage. 
 Truncated sampling <cit.> prunes and then samples the tokens based on the probability distribution. 
 Furthermore,  <cit.> proposed an MoE approach to diversify translation outputs. 
 Moreover, incorporating external corpora in the MoE further promotes diversity, such as by using a knowledge graph <cit.> or by a collection of retrieved sentences <cit.>. 
 Although LLMs have reported superior performance in numerous NLP tasks <cit.>, to the best of our knowledge, diversifying their generations in commonsense reasoning with ICL has not been explored in prior work on GCR.


















  
In-Context Learning.

Recent studies demonstrate that LLMs can exhibit robust few-shot performance on a variety of downstream tasks through ICL <cit.>. 
ICL is a technique for instructing an LLM using one or more examples for a particular text generation task. 
The generated text is conditioned on both the input as well as the instruction prompt.
<cit.> show that in ICL, label words in the demonstration examples function as anchors, which aggregate semantic information to their word representations in the shallow (closer to the input) layers, while providing that information to the final predictions performed by the deeper (closer to the output) layers.
In contrast to fine-tuning-based methods, ICL is computationally lightweight because it does not update the parameters of the LLM.
Therefore, ICL is an attractive method when integrating task-specific knowledge to an LLM by simply changing the prompt and the few-shot examples <cit.>.










 


§ IN-CONTEXT DIVERSIFICATION







We consider the problem of generating a set of diverse sentences that express commonsense reasoning, either by covering a set of given concepts (in CommonGen and DimonGen) or by providing an explanation for a given counterfactual statement (in ComVE).

Formally, given a sequence (a set of concepts or a statement) = {x_1, …, x_m}, the goal of GCR is to generate a set of  grammatically correct and commonsense bearing sentences 
= {y_1,  …, y_n}, where y_i is the i-th output generated by the model with probability p(y_i|). 
Moreover, we require that the generated sentences {y_1,  …, y_n} to be  lexically as well as semantically diverse.


 



 §.§ Sentence Generation







To explain our proposed Prop, let us consider GCR on CommonGen, where we must generate a set of sentences , such that each sentence contains all of the input concepts  as shown in <ref>a.
Given an LLM, we can design a prompt that contains a task-specific instruction and one or more examples containing the input concepts (denoted by [SRC] in <ref>) and the corresponding human-written sentences containing all given input concepts (denoted by [TGT]) to instruct the LLM to generate output sentences  (denoted by [OUTPUT]) for a given set of input concepts  (denoted by [INPUT]).
We refer to a prompt of this nature as a  prompt, and the corresponding set of generated sentences by _ def.



Note that the  prompt does not necessarily guarantee that the generated set of sentences will be diverse and an LLM could return sentences that are highly similar to each other.
To address this issue, we propose a  prompt as shown in <ref>b.
Specifically, the  prompt operates in two steps.
In Step 1, we require that the LLM generate N sentences that are different, in addition to being coherent and commonsense bearing.
Next, we use a suitable diversity metric to evaluate the level of diversity among the generated set of sentences.
If the diversity of the generated sentences is low, in Step 2, we show those sentences to the LLM and instruct it to generate sentences that are different to those.
As the criteria for triggering Step 2, we check whether the exact same sentence has been generated multiple times by the LLM during Step 1.
The final set of generated sentences is denoted by _ div.






 §.§ Diversity-based Sampling
 









Because of the limited availability of human-written reference sentences for evaluating GCR models, there exists a trade-off between quality vs. diversity when generating sentences for GCR tasks.[This trade-off is further empirically verified in <ref>.]
Simply maximising for diversity often leads to generations that do not cover the input concepts in a natural way.
For example, a randomly selected set of sentences would be highly diverse, yet unlikely to capture the input concept sets.
On the other hand, if we force an LLM to generate sentences that contain all of the input concepts, it might find difficult to generate semantically diverse sentences and resort to trivial lexical or syntactic diversity tricks such as morphological inflections or word-order permutations. 

To address this issue, we propose a diversity-based sampling method shown in <ref>.
Consider that the  prompt provides a set _ def of sentences that have not been optimised for diversity (likely to have a higher quality), while on the other hand the  prompt provides a set _ div of sentences that are further refined for diversity (likely to have a higher diversity).
We wish to find a set of sentences that simultaneously satisfies the following criteria:
(a) must contain exactly N sentences, as specified by the user, and
(b) must have a high diversity score, measured using a user-specified diversity metric f (∈_≥ 0).

We formalise this as a subset search problem, where we compute the union _ def∪_ div and search for the subset ^* that jointly satisfies those criteria following the procedure detailed in <ref>.
Although the total number of subsets of size N is |_ def∪_ div|N, it is sufficiently small for the values of N in our GCR tasks, which makes this subset search fast in practice.















































§ EXPERIMENTAL SETTINGS





 §.§ Tasks and Datasets

We evaluate Prop on three GCR tasks as follows.

Constrained Commonsense Reasoning:
In CommonGen <cit.> benchmark, a model is required to generate a sentence covering a given set of concepts such that background commonsense knowledge associated with the input concepts is reflected. 
This dataset contains 35K distinct concept sets (train = 32651, dev = 993, and test = 1497) with corresponding human written sentences (train = 67389, dev = 4018, and test = 6042). 
Each instance contains on average 3-5 input concepts.

Commonsense Explanation Reasoning: ComVE <cit.> is part of the SemEval 2020 commonsense validation task, where for a given counterfactual statement, a model is required to generate an explanation providing a reason describing why the statement is nonsensical. 
This dataset contains 10K (train = 8532, dev = 476, and test = 992) examples, where each example contains three reference outputs.

Diversified GCR: DimonGen <cit.> involves generating diverse sentences that describe the relationships between two given concepts.
It is a challenging task because it requires generating reasonable scenarios for a given pair of concepts without any context.
This dataset contains 17109 instances (train = 15263, dev = 665, test = 1181), where each instance has 3-5 references.





 §.§ Evaluation Metrics



We measure both the quality  and diversity  of the sentences generated by models using the metrics described next.



  §.§.§ Quality Metrics



We compare a generated sentence by a model against a set of human-written references to evaluate the quality of the generation using several metrics:
BLEU <cit.> measures n-gram precision against human reference texts, 
SPICE <cit.> measures the semantic propositional overlap between two sentences, and
BERTScore <cit.> uses contextualised word embeddings to measure the semantic similarity between tokens in two sentences.  
In alignment with prior works <cit.>, when multiple candidate sentences are generated for a test case, we select the highest-scoring candidate for evaluating quality.




  §.§.§ Diversity Metrics





  
Pairwise Diversity: 
We use self-BLEU <cit.> to measure n-gram overlap among sentences within each generated set. 
The metric computes the average sentence-level similarity between all pairwise combinations of the generations in the generation set. 
Note that unlike BLEU, self-BLEU does not require human generated references for measuring diversity.
We use self-BLEU3/4 (corresponding to n = 3 and 4) in our experiment. 
Lower self-BLEU scores indicate higher lexical diversity.



  
Corpus Diversity:
To measure the variety within our generated text corpus, we employ Distinct-k <cit.>, which calculates the ratio of unique k-grams to the total number of k-grams. 
This metric is particularly useful for adjusting the bias of LLMs toward generating longer sequences, ensuring that diversity is not artificially inflated by the sentence length. 
Additionally, we use Entropy-k to evaluate the distributional uniformity of k-gram occurrences, considering word frequencies for a more nuanced view of diversity.
Higher Distinct-k and Entropy-k scores indicate higher diversity.




  
Semantic Diversity: 
All previously described diversity metrics are limited to evaluating lexical diversity.
To measure diversity at a semantic level, we propose self-cosSim, which is the average pairwise cosine similarity between generated sentences, computed using sentence embeddings obtained from SimCSE <cit.>.
Likewise, we define the self-BERTScore as a diversity metric that averages the BERTScores for all generated sentence pairs.
Lower self-cosSim and self-BERTScore values indicate  higher semantic diversity.





  §.§.§ Combined Metrics
 



We would prefer GCR models that have both high quality and high diversity.
To incoporate both aspects into a single metric, we compute the Harmonic Mean between 
(a) the self-BLEU-4 as the diversity metric, and
(b) BERTScore as the quality metric.
As discussed in <ref>, there exists a trade-off between quality and diversity in GCR.
Therefore, the harmonic mean is suitable when averaging quality and diversity scores.[We use self-BLEU-4 for diversity and BERTScore for quality  in Harmonic Mean due to their reliability shown in preliminary evaluations. Other metric pairs are in <ref>. ]


<cit.> proposed FBD as a joint metric for simultaneously measuring both the quality and diversity of NLG.
FBD is inspired by the Fréchet Inception Distance (FID), proposed by <cit.>, for measuring the quality of image generation.
Specifically, FBD computes the pooler output[The last layer's hidden-state of the first token of the sequence is further processed by a Linear layer and a Tanh activation function.] of a sentence as its embedding <cit.> and represents a set of sentences using the mean vector and the covariance matrix computed from their sentence embeddings.
Next, Wasserstein-2 distance is computed between the set of reference sentences and the set of generated sentences, which captures both the distance between the means as well as variance in the distributions.
Lower FBD scores indicate high combined performance.




 §.§ Implementation Details


We use  and [<https://huggingface.co/lmsys/vicuna-13b-v1.5>] as LLMs with temperature set to 1.0 in our experiments.
By using two LLMs with significantly differing number of parameters and by including, Vicuna, an open source LLM, we plan to improve the reliability and reproducibility of our results.
Max response length is set to 25 tokens. 
The inference times for CommonGen, ComVE and DimonGen datasets are respectively 5-6, 2-3 and 1-2 hours.
The cost of running Prop with  are ca. $6, $4 and $4 respectively for CommonGen, ComVE and DimonGen datasets.
On the other hand, the costs of fine-tuning on  are much higher at $58.8 for CommonGen, $24.7 for ComVE and $32.0 for DimonGen.
Moreover, fine-tuning with LoRA <cit.>  with rank of 8 and alpha of 16 on Vicuna takes ca. 34 hours. 
We use [<https://huggingface.co/facebook/bart-large>] for MoE-based models. 
We use the  to generate sentences for the CommonGen train/dev/test sets using the ,  and for Prop.
For model training, we use the Adam optimiser <cit.> with a batch size of 64, a learning rate of 3e-5 and a beam size of 5. 
All of the MoE-based models are trained for 20 epochs and required to generate k = 3 sentences.
All experiments, except with GPT3.5-turbo, are conducted on a single RTX A6000 GPU. 




§ RESULTS AND DISCUSSION





 §.§ Commonsense Generation






We compare the commonsense generations made by Prop against those using the  and  prompts.
For this purpose, we use  as the LLM and use the same 10 few-shot examples in all prompts for ICL.
Further templates of the  and  prompts used for each task are given in <ref>.
 To assess the impact of ICL, we compare against fine-tune method, wherein  is fine-tuned on the entire training set in each dataset.
Specifically, we use multiple human-written sentences, available in the training data for the three datasets to separately fine-tune the models for each task.
It is noteworthy that the fine-tune method uses a substantially larger dataset for training (e.g., 67,389 sentences from CommonGen) compared to the 10 examples used by the ICL-based approaches.
We use self-BLEU-3 as the diversity metric f in <ref> for Prop in this evaluation.
The outcomes, presented in <ref>, highlight the diversity and quality metrics of these methods across the CommonGen, ConVE, and DimonGen datasets.
Additionally, a human baseline is introduced to evaluate the diversity of sentences written by humans, where we pair-wise compare the human-written sentences for each input in the instances in the benchmark datasets using diversity metrics.
Note that however, the human baseline must not be considered as an upper-bound for diversity because there are only a smaller number of human-written sentences per instance in the benchmark datasets.


From <ref>, we see that fine-tune generates sentences that have high semantic and corpus diversity, and outperforms the human baseline.
However, recall that fine-tune requires a much larger training set and is computationally costly compared to all ICL-based methods. 
Moreover, we see that Prop can strike a good balance between quality and diversity in the
sentences generated. 
Among the ICL-based methods, Prop achieves the best diversity scores on all diversity metrics in all three datasets. It also exhibits higher diversity compared against the human-written references. 
Moreover, Prop outperforms  and  according to the Combined metrics. Prop also achieves a Harmonic Mean comparable to that of the fine-tune baseline. 
Although  reports the best quality scores, it has low diversity, and is consistently outperformed by  and Prop on diversity metrics. 
On the other hand,  generally scores lower on the quality metrics. 
Compared to  and , Prop enhances generation diversity while maintaining a satisfactory level of quality. 
ICD is also more stable to the sampling method such as temperature than fine-tune, as shown in <ref>.
Note that fine-tune is not an ICL setting (the focus of this paper) and is included only as a baseline to demonstrate the level of performance that can be achieved by fine-tuning on a much larger dataset.

Despite this, Prop outperforms fine-tune on the Pairwise Diversity in all three datasets, and Combined metrics in the CommonGen dataset.





















As an open source alternative LLM to , we repeat this evaluation with  <cit.> in <ref>.
The same 10 few-shot examples as used with  are used in this experiment for the ICL-based methods.
Full table on three datasets are shown in <ref>.
<ref> reconfirms Prop's ability to balance both quality and diversity according to the Combined metrics (i.e. Harmonic Mean and FBD) on this dataset.
Interestingly, we see that methods that use Vicuna-13b to be more diverse compared to those that use , while the latter showing better generation quality.






In <ref>, we use different diversity metrics as f  in <ref> to study the effect on text generation of Prop.
We see that self-BLUE-3 and self-CosSim perform similarly across the quality metrics.
Self-BERTScore shows a slightly lower quality (BLEU-3 and SPICE), which indicates some level of over-fitting to the diversity metric being used.
According to the combined metrics, any of those diversity metrics can be used with Prop to obtain comparable performance.









 §.§ Downstream Evaluation



The experiments presented in <ref> show the ability of our proposed Prop to generate diverse and commonsense bearing sentences.
Therefore, an important question with practical implications is whether we can use the sentences generated by Prop as additional training data to improve both diversity and quality of previously proposed models on the GCR task, which could be seen as a downstream (extrinsic) evaluation.

For this purpose we select the MoE <cit.>, which diversifies the generation by selecting outputs from a mixture of experts. 
Each expert is assigned a randomly generated sequence of tokens, which is used as a prefix for all inputs sent to that expert.
For each input, an expert is selected according to the value of a latent variable, which is trained using the hard-EM algorithm.
We follow <cit.> and train three experts that retrieve sentences from the collection of sentences generated by Prop for concept sets in the CommonGen train split (210846 sentences in total).
We use  <cit.> as the base model, which has shown to produce high quality commonsense generations <cit.> as the generator for all experts (see <ref> for further details).
We denote this method by ICD+MoE.

As baselines for comparisons, we repeat the above process using the sentences generated by  and , which we denote respectively as +MoE and +MoE in <ref>.
Moreover, we compare the performance against two previously proposed MoE models: MoE  <cit.> and MoKGE <cit.>. 
MoE relies solely on the base model, whereas MoKGE requires each expert to use different sets of concepts from the ConceptNet <cit.> KG. 
Because <cit.> do not evaluate their MoKGE method on CommonGen, we ran their original implementation[<https://github.com/DM2-ND/MoKGE>] on CommonGen and report results in <ref>.

All previously proposed GCR methods are exclusively trained using human-created data (e.g. sentences written by human and/or manually compiled KGs such as ConceptNet), whereas the methods described thus far in this section are trained on sentences generated by an LLM ().
Therefore, to evaluate the feasibility of using LLM-generated sentences for training GCR models, we include the following previously proposed GCR models that are trained using a combination of corpora and KGs:
KG-BART <cit.>,EKI-BART <cit.> and KFCNet <cit.>.
For KFCNet, we present its two results – KFCNet w/o FC, which relies only on sentences including the input concepts, without further processing, and KFCNet, which additionally ranks candidates and adds contrastive modules for the encoder and the decoder <cit.>.
However, note that those methods do not consider diversification, and do not report performance using diversity metrics. 
Therefore, we report only their published results for generation quality in <ref>.

From <ref> we see that +MoE always outperforms the original MoE in all diversity metrics, which shows that sentences generated from LLMs can be used to diversify MoE-based GCR.
ICD+MoE closely matches the performance of +MoE on diversity metrics, while consistently outperforming both +MoE and +MoE on quality metrics.
In particular, the quality metrics reported by ICD+MoE (underlined in <ref>) are competitive against those obtained by the models that are trained on human-compiled resources (in the top block), except against KFCNet.
This finding hints at potential improvement gains for GCR by using hybrid training resources that combine both human-compiled and LLM-generated data, which we highlight as an interesting future research direction.





































 §.§ Diversity-Awareness of LLMs



Given that we use LLMs to produce diverse generations via ICL, it remains an open question whether an LLM would agree with humans on the diversity of a given set of sentences.
To answer this question, we use randomly selected 210 sentences (35 sets, each containing 6 sentences) generated by Prop (using self-BLEU-3 as the diversity metric) for the input concept sets in the CommonGen dataset.
We instruct  to rate the diversity of a given set of sentences according to five diversity ratings 1-5 with 1 being highly similar, while 5 being highly diverse.[Detailed prompt templates are shown in <ref>.]
We provide the same instruction as the annotation guidelines for eight human-annotators, who are graduate students in NLP.
To reduce the subjective variability in human judgements, we average and then normalise the ratings following the Likert scale.

In <ref>, we plot the GPT-assigned ratings against those by humans.
We further split the ratings into high vs. low diversity ratings depending on whether the rating is greater or lesser than 3.
The majority of the data points are distributed along the diagonal quadrants and a Cohen's Kappa of 0.409 indicating a moderate level of agreement between GPT and human ratings for diversity.

The generated sentences using the  prompt, Prop and the human references in CommonGen and ComVE datasets for a single test instance are shown in <ref>.
From <ref> we see that the sentences generated using the  prompt often results in significant token overlap, thereby lowering the diversity.
On the other hand, Prop generates both lexically and semantically diverse sentences, covering the diverse viewpoints in the human references.







§ CONCLUSION


We proposed, Prop, an ICL-based method for achieving the optimal balance between diversity and quality in text generation via LLMs. 
Our experiments, conducted on three GCR tasks, demonstrate that Prop significantly improves the diversity without substantially compromising the quality. 
Furthermore, we found that by training on the sentences generated by Prop, we can improve diversity in previously proposed GCR methods.





§ LIMITATIONS

This study primarily focuses on the generation of English sentences using pre-trained LLMs, a limitation shaped by the datasets we employed. Specifically, we used the ComVE <cit.>, CommonGen <cit.> and DimonGen <cit.> datasets, which are well-regarded for evaluating diversified commonsense reasoning in English. Therefore, our evaluation of the generation quality was limited to English, which is a morphologically limited language.  Future research could expand this scope to include multilingual pre-trained models, thereby encompassing a broader linguistic spectrum.

Our approach is primarily geared towards optimizing the trade-off between diversity and quality in text generation. Consequently, we maintained consistent default instructions across all experiments, adopting the standard commonsense generation prompts used in <cit.> as our default instructions.

We conducted our experiments using both a closed model (i.e. ) as well as an open-source one (i.e. ) to promote the reproducibility of our results, which are reported using multiple public available benchmarks. 
However, there exist many other LLMs with varying numbers of parameters and trained on different corpora.
Therefore, we consider it is important to evaluate our proposed method on a broad range of LLMs to verify the generalisability of our proposed method.
However, conducting such a broad analysis can be computationally costly and expensive.
For example, although GPT-4 is known to have superior text generation capabilities, it incurs substantially greater costs (being 30 times more expensive than  at the current  pricing).
Nevertheless, Prop is adaptable and could be extended to other LLMs.





§ ETHICAL CONSIDERATIONS









In this work, we did not create or release any manually annotated data. 
Our work is based on the publicly available datasets, CommonGen, ComVE, and DimonGen. 
To the best of our knowledge, no ethical issues have been reported for those datasets.
Therefore, we do not foresee any data-related ethical issues arising from our work. 

However, LLMs are known to generate responses that may reflect societal biases and potentially harmful content. 
We have not verified whether the  and  LLMs that we use in our experiments have similar problems. 

Therefore, it is important to test on existing benchmarks for social biases and harmful generations before the proposed method is deployed to diversify existing GCR methods used by human users.

To elicit human judgements of diversity for the sentences generated by Prop, we use annotators who are familiar with working with LLMs.
It is possible that their subjective (and possibly biased) viewpoints might have influenced the ratings provided. 
Therefore, it will be important to conduct the evaluation involving a group of annotators with different backgrounds to validate the findings reported in this analysis.






acl_natbib





§ SUPPLEMENTARY APPENDIX



§ MIXTURE OF EXPERTS


Given an input x, its corresponding LLM-generated sentences are divided into three random subsets. For each subset G_i = {g^i_1,...,g^i_k}, alongside the input x, we concatenate their token sequences with a separate latent variable z_i, resulting in the final input x_i^f. The z_i is a randomly initialised sequence of tokens.

    x_i^f = z_i[CLS] x [SEP] g^i_1 [SEP]...g^i_k

We train the model using the hard Expectation-Maximization (EM) approach, where during the E-step, for each x_i^f and its corresponding target y_i^target, we identify the input that yields the highest probability as the best training example, with θ representing the generator model parameters: 

The model is trained using hard-EM by assigning full responsibility to the expert with the largest joint probability. In the E-step, for each input x_i^fin and target y_i^tgt, choose the best input with the highest probability, to construct the training examples, where θ is the model's parameters.

    y_i^tgt = y_jargmax  p(y_j|x_i^f;θ)

Subsequently, in the M-step, we use these selected training examples to fine-tune the generator models. During inference, we input all diversified, context-aware inputs into the generator model to yield a range of diverse outputs.










§ IMPACT OF TEMPERATURE SAMPLING ON THE DIVERSITY AND QUALITY


In this section, we investigate the impact of temperature on various methods on the CommonGen dataset.  Although there are various sampling methods beyond temperature, such as top-k and top-p, we specifically focus on the general performance of Prop under different temperature settings. Our experiment in conducted on the GPT-3.5-turbo-0613. <ref> demonstrates that Prop consistently outperforms both  and  on the Combined metrics across all temperature settings, which aligns with our findings in <ref>. Moreover, Prop exhibits less sensitivity to temperature variations compared to the other baselines and performs better on Combined metrics with the increase of temperature, which can be considered as an additional advantage of our proposed method. 

Furthermore, we observe that the fine-tune method is also significantly influenced by temperature sampling on the GCR task. At T=1.5, the  baseline, which applies ICL on the same base model GPT-3.5-turbo, outperforms the fine-tune method. The fine-tuned model generates responses that are of very low quality, consisting mostly of nonsensical word combinations.










§ FULL RESULTS ON VICUNA-13B MODEL


<ref> shows the full result on the open source Vicuna-13b model across three datasets. It reconfirms Prop's ability to balance both quality and diversity according to the combined metrics. Furthermore, we find that methods using the Vicuna model show lower quality than those using GPT-3.5-turbo while generating more diverse sentences.








§ CANDIDATE METRICS FOR CALCULATING HARMONIC MEANS


In the paper, we use self-BLEU-4 and BERTScore to calculate the harmonic means as one of the combined metrics because self-BLEU <cit.> evaluates the n-gram overlap among sentences within the generated set, providing a measure of lexical diversity. On the other hand, BERTScore <cit.> assesses the semantic similarity between the generated sentences and the golden sentences from the dataset, capturing the quality aspect from a semantic perspective.

We also use other diversity and quality metrics' combination as the candidate metrics to calculate the harmonic means. In the <ref>, we could see that in each metrics' pair, the ICD method has the highest score among the ICL-based approaches. It also has comparable performance with the fine-tune method.










§ LLM PROMPT TEMPLATES


 <ref> shows the templates that are used for the two GCR tasks: CommonGen and ConVE. 
The  prompt is adapted from <cit.> and are task-specific.
On the other hand, the  prompt modifies the  prompt by appending a task-independent instruction that first checks whether the diversity of the sentences generated in Step 1 is low, and if presents the generated sentences to the LLM and re-prompts it to generate more diverse set of sentences.

We use  to predict the diversity of a given set of sentences using the prompt shown in <ref>.
This prompt uses five diversity categories (i.e. very similar, somewhat similar, neutral, somewhat diverse, and highly diverse) with increasing diversity with their definitions.
Next, the set of sentences to be evaluated for their diversity is presented.
Finally, the expected output format of the predictions is described at the end of the prompt.
 As recommended by <cit.>, we do not require the LLM to provide  reasons for its predictions because it sometimes forces the model to focus on the reason generation than the prediction. 
 
 After the LLM's evaluation, the classification are mapped to values from 1 to 5 where 1 being highly similar to 5 being highly diverse. For each sentences set, we request the LLM to perform the evaluation three times and take the average score.







§ HUMAN EVALUATION

As human-annotators, we recruited eight graduate students from the department of computer science who specialise in NLP and are fluent speakers in English.
We provided the human annotators with the same set of instructions as we provided to the LLMs.

The annotators were allowed to deviate from the provided instructions when handling edge cases.
Apart from the instruction in <ref>, we also instruct the human annotators to focus on the diversity, and not so much on their commonsense quality, which we evaluate separately using semi-automatic metrics by comparing against human-written reference sentences in the evaluation benchmarks.
Moreover, we informed the annotators that their evaluations would be used in a comparative analysis with the scores generated by an LLM.





































§ ADDIONAL GENERATION EXAMPLES

We show additional sentences generated by our proposed methods for the CommonGen, ComVE and DimonGen datasets in <ref>.






























