RoBERTa¶
The RoBERTa model was proposed in RoBERTa: A Robustly Optimized BERT Pretraining Approach by Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, Veselin Stoyanov. It is based on Google’s BERT model released in 2018.
Note
This class is nearly identical to the PyTorch implementation of RoBERTa in Huggingface Transformers. For more information, visit the corresponding section in their documentation.