Just for performing some experiments. Do not use.

This needed to be restarted at 100k. I am getting memory errors at the end of the epoch. Not really sure why. 

Step 2 is therefore on train_2__4. Static learning rate for a while. The first 100k ended at 0.59. This is decent so early. No point in running more epochs here though. Changing the corpus and continue training.



{}
transformers jax tensorboard roberta fill-mask autotrain_compatible endpoints_compatible region:us
TAGS
#transformers #jax #tensorboard #roberta #fill-mask #autotrain_compatible #endpoints_compatible #region-us 

Just for performing some experiments. Do not use.

This needed to be restarted at 100k. I am getting memory errors at the end of the epoch. Not really sure why. 

Step 2 is therefore on train_2__4. Static learning rate for a while. The first 100k ended at 0.59. This is decent so early. No point in running more epochs here though. Changing the corpus and continue training.



TAGS
#transformers #jax #tensorboard #roberta #fill-mask #autotrain_compatible #endpoints_compatible #region-us 

passage: TAGS
#transformers #jax #tensorboard #roberta #fill-mask #autotrain_compatible #endpoints_compatible #region-us 
