How to remove a linked library in blender?
Okay, I figured it out!

In the outliner, I went to "Blender File" and beneath the "Current File" data, all the library files were listed. From there I could simply right-click and delete the unused library files.