Purpose of needing 12 swords in Camelot to trigger end of game with no Traitor
Remember the catapults 

If there are 12 of them at any time you lose the game. So even with no traitor and 8 white swords, if you are too tight and lose the invasion quests (or simply draw the wrong black cards), you may lose because of catapults before completing the 12 swords on the table.