update model card README.md

README.md

@@ -13,9 +13,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [neuralmagic/oBERT-12-upstream-pruned-unstructured-97](https://huggingface.co/neuralmagic/oBERT-12-upstream-pruned-unstructured-97) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.1301
-- Micro f1: 0.5159
-- Macro f1: 0.0352
+- Loss: 0.2718
+- Micro f1: 0.3779
+- Macro f1: 0.0172
 
 ## Model description
 
@@ -40,28 +40,16 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 15
+- num_epochs: 30
 - mixed_precision_training: Native AMP
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss | Micro f1 | Macro f1 |
 |:-------------:|:-----:|:----:|:---------------:|:--------:|:--------:|
-| No log        | 1.0   | 25   | 0.3994          | 0.3783   | 0.0172   |
-| No log        | 2.0   | 50   | 0.2846          | 0.3791   | 0.0172   |
-| No log        | 3.0   | 75   | 0.2159          | 0.3791   | 0.0172   |
-| No log        | 4.0   | 100  | 0.1802          | 0.3791   | 0.0172   |
-| No log        | 5.0   | 125  | 0.1618          | 0.3791   | 0.0172   |
-| No log        | 6.0   | 150  | 0.1515          | 0.3791   | 0.0172   |
-| No log        | 7.0   | 175  | 0.1452          | 0.3791   | 0.0172   |
-| No log        | 8.0   | 200  | 0.1411          | 0.3931   | 0.0202   |
-| No log        | 9.0   | 225  | 0.1379          | 0.4413   | 0.0277   |
-| No log        | 10.0  | 250  | 0.1350          | 0.4694   | 0.0309   |
-| No log        | 11.0  | 275  | 0.1327          | 0.4993   | 0.0336   |
-| No log        | 12.0  | 300  | 0.1309          | 0.5084   | 0.0344   |
-| No log        | 13.0  | 325  | 0.1297          | 0.5147   | 0.0349   |
-| No log        | 14.0  | 350  | 0.1291          | 0.5060   | 0.0343   |
-| No log        | 15.0  | 375  | 0.1287          | 0.5107   | 0.0346   |
+| No log        | 1.0   | 49   | 0.2715          | 0.3791   | 0.0172   |
+| No log        | 2.0   | 98   | 0.1682          | 0.3791   | 0.0172   |
+| No log        | 3.0   | 147  | 0.1425          | 0.3791   | 0.0172   |
 
 
 ### Framework versions
@@ -70,67 +58,3 @@ The following hyperparameters were used during training:
 - Pytorch 1.13.0+cu116
 - Datasets 2.8.0
 - Tokenizers 0.13.2
-# Day 1 
-
-1.  Tried to use the Neural Magic Model "neuralmagic/oBERT-12-upstream-pruned-unstructured-97".  The macro and micro f1 scores were much smaller at the
-beginning of the model; the initial step did not increase much. However, it did outperform in the same epoch by .159 difference in the f1 score.
-2. Modification of the code was more significant was able to add errors in my program  to move to the CPU if there was an error in my program 
-``` Python
-import gc
-'''
-Try and Catch the block when training the model using more memory than the GPU, it will produce an error.
-1. Check the Amount of GPU memory used
-2. Move the model to the CPU
-3. Call the garbage collector 
-4. Free the GPU memory in the cache
-5. Check the amount of GPU memory used to see if it is freed
-'''
-def check_gpu_memory():
-    print(torch.cuda.memory_allocated()/1e9)
-    return torch.cuda.memory_allocated()/1e9
-try:
-    trainer.train()
-except RuntimeError as e:
-    if "CUDA out of memory" in str(e):
-        print("CUDA out of memory")
-        print("Let's free some GPU memory and re-allocate")
-        check_gpu_memory()
-        ## Move the model to CPU
-        model.to("cpu")
-        gc.collect()
-        ## Free the GPU memory
-        torch.cuda.empty_cache()
-        check_gpu_memory()
-    else:
-        raise e
-```
-4. Able to check if there was a number of support my model can support in my model 
-``` Python
-from transformers import Trainer, TrainingArguments
-def is_on_colab():
-    if 'google.colab' in sys.modules:
-        return True
-    return False
-
-training_args_fine_tune = TrainingArguments(
-    output_dir  = "./multi-label-class-classification-on-github-issues" , 
-    num_train_epochs = 15,
-    learning_rate = 3e-5,
-    per_device_train_batch_size = 64 , 
-    evaluation_strategy = "epoch" , 
-    save_strategy="epoch"  , 
-    load_best_model_at_end=True, 
-    metric_for_best_model='micro f1',
-    save_total_limit=1, 
-    log_level='error', 
-    push_to_hub = True  if is_on_colab else False ,
-    )
-if torch.cuda.is_available():
-    ## check if the Cuda GPU can bfloat16
-    if torch.cuda.is_bf16_supported():
-        print("Cuda GPU can support bfloat16")
-        training_args_fine_tune.fp16 = True
-    else:
-        print("Cuda GPU cannot support bfloat16 so instead we will use float16 ")
-        training_args_fine_tune.fp16 = True
-```