Hello,
I tried to use reinforcement learning (trlx library) to train a FLAN-T5. I have seen that as examples there are sentiment analysis or summarization tasks, but in my case it is a generic generation task. Trying to use my dataset, I tried the config: ppo_config_cnn_daily.yml.

The result I got is this, and it seems to be a summarization task:

Prompt: "[some text] ... from 1978, excluding finishing years incoming 62"
Output: "years incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming 62 incoming"
And I'm using this code with my personal prompts:

trlx.train(
    reward_fn=reward_fn,
    prompts=train_prompts,
    eval_prompts=valid_prompts,
    config=config
)
I have two problems, maybe caused by summarization task:

First the repetition at the end
Second, the expected output is very different
Should I change the code or are there generic examples?

My configuration is:
gen_kwargs:
    max_new_tokens: 100
  gen_experience_kwargs:
    max_new_tokens: 100
    do_sample: True
    temperature: 1.0
    top_k: 50
    top_p: 0.95

That works well for summarization. Any suggestion about how to change these parameters for generic text 2 text tasks and not summarization?
spam lang_mismatch pii not_appropriate hate_speech sexual_content quality toxicity humor creativity violence