pytorch roberta feature-extraction transformers
Pre Träna CodeBert med JavaFx + Java FXML + JavaFx relaterat logik kod (dvs. Model, Controller för olika JavaFx kod). 

Blev ungefär 130 k kod exemplar




````
***** train metrics *****
  epoch                    =        3.0
  train_loss               =     0.4556
  train_runtime            = 5:57:43.71
  train_samples            =     131945
  train_samples_per_second =     18.442
  train_steps_per_second   =      2.305
  
  
  ***** eval metrics *****
  epoch                   =        3.0
  eval_loss               =     0.2984
  eval_runtime            = 0:01:59.72
  eval_samples            =       6944
  eval_samples_per_second =     57.999
  eval_steps_per_second   =       7.25
  perplexity              =     1.3477
  
  ````