3 epoch på norsk oscar corpus.

warmup_steps = 1000
learning_rate = 5e-3
block_size =512
per_device_train_batch_size = 64

cirka 1,5 time på TPU v3-8 per epoch
{}
transformers pytorch jax tensorboard gpt2 text-generation autotrain_compatible endpoints_compatible text-generation-inference region:us
TAGS
#transformers #pytorch #jax #tensorboard #gpt2 #text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

3 epoch på norsk oscar corpus.

warmup_steps = 1000
learning_rate = 5e-3
block_size =512
per_device_train_batch_size = 64

cirka 1,5 time på TPU v3-8 per epoch
TAGS
#transformers #pytorch #jax #tensorboard #gpt2 #text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 

passage: TAGS
#transformers #pytorch #jax #tensorboard #gpt2 #text-generation #autotrain_compatible #endpoints_compatible #text-generation-inference #region-us 
