What are the difference between .bin and .pt pytorch saved model types?
Sometimes I see .bin files for pretrained pytorch, like the one here

https://github.com/allenai/scibert#pytorch-models

However, the files are usually saved as .pt files. 

What's the difference between these two parameter weights file formats? Why are there two?

There is no difference as it's just an extension. When it comes to UNIX-like OSes one can open the file no matter the extension (see here), Windows on the other hand is built with them in mind (here).
torch can read either .bin or .pt or .anything so it's probably convention employed by the creators of that repository.
Standard approach is to use .pt or .pth, though the second extension collides with Python's text file readable by interpreter, so .pt seems the best idea for now (see this github issue).
