Feature suggestion: Alignment Method

#881
by Bodor - opened

Instead of having a model of type chat models (RLHF, DPO, IFT, ...) it will be very informative to have an extra field of the used alignment method if any (DPO, KTO,CPO..etc)

Open LLM Leaderboard org

Hi @Bodor ,

It's an interesting suggestion, let us think about this

Open LLM Leaderboard org

We've researched and discussed this feature internally, and I'm back with my answer!

While adding an alignment method field could be informative, it may complicate the submission process, because currently, model types are user-submitted. Sometimes people faces challenges in accurately categorising models and additional fields could increase such errors.

However, if this information becomes easily extractable from model cards in the future, we'll certainly consider implementing more detailed categorisation.

For now, I'll close this discussion. Please feel free to open a new one if you have any other questions or suggestions!

alozowski changed discussion status to closed

Sign up or log in to comment