Data. In Proceedings of the 58th Annual Meet-
ing of the Association for Computational Lin-
guistics, pages 8413–8426, Online. Association
for Computational Linguistics.
Dani Yogatama, Cyprien de Masson d’Autume,
Jerome Connor, Tomas Kocisky, Mike
Chrzanowski, Lingpeng Kong, Angeliki Lazari-
dou, Wang Ling, Lei Yu, Chris Dyer, and Phil
Blunsom. 2019. Learning and Evaluating Gen-
eral Linguistic Intelligence. arXiv:1901.11373
[cs, stat].
Yang You, Jing Li, Sashank Reddi, Jonathan Hseu,
Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan
Song, James Demmel, and Cho-Jui Hsieh. 2019.
Large Batch Optimization for Deep Learning:
Training BERT in 76 Minutes. arXiv preprint
arXiv:1904.00962, 1(5).
Ali Hadi Zadeh and Andreas Moshovos. 2020.
GOBO: Quantizing Attention-Based NLP Mod-
els for Low Latency and Energy Efﬁcient Infer-
ence. arXiv:2005.03842 [cs, stat].
Oﬁr Zafrir, Guy Boudoukh, Peter Izsak, and Moshe
Wasserblat. 2019. Q8BERT: Quantized 8bit
BERT. arXiv preprint arXiv:1910.06188.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali
Farhadi, and Yejin Choi. 2019. HellaSwag: Can
a Machine Really Finish Your Sentence? In Pro-
ceedings of the 57th Annual Meeting of the As-
sociation for Computational Linguistics, pages
4791–4800.
Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang,
Maosong Sun, and Qun Liu. 2019. ERNIE: En-
hanced Language Representation with Informa-
tive Entities. In Proceedings of the 57th Annual
Meeting of the Association for Computational
Linguistics, pages 1441–1451, Florence, Italy.
Association for Computational Linguistics.
Zhuosheng Zhang, Yuwei Wu, Hai Zhao, Zuchao
Li, Shuailiang Zhang, Xi Zhou, and Xiang Zhou.
2020. Semantics-aware BERT for Language
Understanding. In AAAI 2020.
Sanqiang Zhao, Raghav Gupta, Yang Song,
and Denny Zhou. 2019. Extreme Lan-
guage Model Compression with Optimal Sub-
words and Shared Projections. arXiv preprint
arXiv:1909.11687.
Yiyun Zhao and Steven Bethard. 2020. How does
BERT’s attention change when you ﬁne-tune?
An analysis methodology and a case study in
negation scope. In Proceedings of the 58th An-
nual Meeting of the Association for Computa-
tional Linguistics, pages 4729–4747, Online. As-
sociation for Computational Linguistics.
Wenxuan Zhou, Junyi Du, and Xiang Ren.
2019. Improving BERT Fine-tuning with
Embedding Normalization. arXiv preprint
arXiv:1911.03918.
Xuhui Zhou, Yue Zhang, Leyang Cui, and Dandan
Huang. 2020. Evaluating Commonsense in Pre-
Trained Language Models. In AAAI 2020.
Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom
Goldstein, and Jingjing Liu. 2019. FreeLB: En-
hanced Adversarial Training for Language Un-
derstanding. arXiv:1909.11764 [cs].