However , what would happen if the attacker was a noble who had a castle , too strong for the townspeople ? This was often the case and certainly the town commune could not muster the forces to attack a castle . However , if the attacker was a noble , safely ensconced in a castle ( as was often the case ) , the town commune could not muster the forces to attack him directly .