answer. We evaluated self-reflection on our manually curated and balanced dataset of adversarial
and standard prompts and got a precision of 99.4% for a recall of 95.6% (considering acceptable
prompts as positives).
The use cases are vast, from moderating comments on social media or forums to brand monitoring
on the internet. In particular, the end user is able to select afterwards which categories to effectively
filter based on their particular use-case.
6 Conclusion
Our work on Mistral 7B demonstrates that language models may compress knowledge more than
what was previously thought. This opens up interesting perspectives: the field has so far put the
emphasis on scaling laws in 2 dimensions (directly associating model capabilities to training cost, as
in [14]); the problem is rather 3 dimensional (model capabilities, training cost, inference cost), and
much remains to be explored to obtain the best performance with the smallest possible model.
Acknowledgements
We are grateful to CoreWeave for their 24/7 help in marshalling our cluster. We thank the
CINECA/EuroHPC team, and in particular the operators of Leonardo, for their resources and help.
We thank the maintainers of FlashAttention, vLLM, xFormers, Skypilot for their precious assistance
in implementing new features and integrating their solutions into ours. A huge thanks to Tri Dao
and Daniel Haziza for helping include Mistral related changes to FlashAttention and xFormers on