add pytorch profilier and allow_tf_32

README.md

@@ -12,14 +12,19 @@ license: mit
 
 # The Unsolved MNIST 🔢
 **M**odified **N**ational **I**nstitute for **S**tandards and **T**echnology Dataset
+###### TODO: Implementation
 
 # Description
+###### TODO: Implementation
 
 # Setup
+###### TODO: Implementation
 
 # Objective
+###### TODO: Implementation
 
 # Logs
+###### TODO: Implementation
 
 ## Model Summary
 ```log
@@ -187,7 +192,7 @@ Estimated Total Size (MB): 18.53
 ```
 
 ## Training Logs
-
+###### TODO: Implementation
 ```sh
 cd /usr/home/:USER:/UnsolvedMNIST
 tensorboard --logdir=logs
@@ -195,6 +200,7 @@ tensorboard --logdir=logs
 ```
 
 ## Performance Profiling
+###### TODO: Implementation
 ```log
 -------------------------------------------  ------------  ------------  ------------  ------------  ------------  ------------  
                                        Name    Self CPU %      Self CPU   CPU total %     CPU total  CPU time avg    # of Calls  
@@ -235,8 +241,6 @@ Self CPU time total: 636.439ms
 ```
 
 
-# Credits:
-- [pytorch_performance_profiling.md](https://gist.github.com/mingfeima/e08310d7e7bb9ae2a693adecf2d8a916)
-- [FLOPs calculation](https://medium.com/@dzmitrybahdanau/the-flops-calculus-of-language-model-training-3b19c1f025e4)
-- [software 2.0](https://karpathy.medium.com/software-2-0-a64152b37c35)
-- [weight init](https://towardsdatascience.com/weight-initialization-in-neural-networks-a-journey-from-the-basics-to-kaiming-954fb9b47c79)
+
+# Contribution
+###### TODO: Implementation

train.py

@@ -23,6 +23,7 @@ from utils import TRAIN_TRANSFORMS, TEST_TRANSFORMS
 
 
 # Auxilary utils
+torch.backends.cuda.matmul.allow_tf32=True
 torch.set_float32_matmul_precision('high')
 torch.cuda.amp.autocast(enabled=True,dtype=torch.float32)
 device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')
@@ -122,7 +123,7 @@ summary(
 trainer  = pl.Trainer(
                 max_epochs=CONFIG['training'].get('num_epochs',15),
                 logger=logger,
-                profiler=perf_profiler,#'advanced',
+                profiler='pytorch',#perf_profiler,#'advanced',
                 callbacks=call_backs,
                 precision=32,
                 enable_model_summary=False,