Llama 2: オープンファウンデーションとファインチューンされたチャットモデル
arXivプリント arXiv:2109.07958 , 2021.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke
Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint
arXiv:1907.11692 , 2019.
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret
Zoph, Jason Wei, et al. The flan collection: デザインデータとメソッドフォーエフェクティブインストラクションチューニング.
arXivプリント arXiv:2301.13688 , 2023.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 ,
2017.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreﬀe, Uri Alon, Nouha
Llama 2: Open Foundation and Fine-Tuned Chat Models
Llama 2: Open Foundation and Fine-Tuned Chat Models
arXiv preprint arXiv:2109.07958 , 2021.
Yinhan Liu,Myle Ott, NamanGoyal, Jingfei Du,Mandar Joshi, DanqiChen, Omer Levy,Mike Lewis, Luke
Zettlemoyer,andVeselinStoyanov. Roberta: Arobustlyoptimizedbertpretrainingapproach. arXivpreprint
arXiv:1907.11692 , 2019.
ShayneLongpre,LeHou,TuVu,AlbertWebson,HyungWonChung,YiTay,DennyZhou,QuocVLe,Barret
Zoph,JasonWei,etal. Theﬂancollection: Designingdataandmethodsforeﬀectiveinstructiontuning.
arXiv preprint arXiv:2301.13688 , 2023.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 ,
2017.
AmanMadaan,NiketTandon,PrakharGupta, SkylerHallinan, LuyuGao,SarahWiegreﬀe,UriAlon,Nouha