ith batch size 8
	



red circle with blue background



cyan circle with brown floral background



After 6000 steps with batch size 8:
	



red circle with blue background



cyan circle with brown floral background



Training on a 16 GB GPU
	
Enable the following optimizations to train on a 16GB GPU:
Gradient checkpointing
bitsandbyte’s 8-bit optimizer (take a look at the [installation]((https://github.com/TimDettmers/bitsandbytes#requireme