as a consequence of the doctrine of perpetual virginity , which does not allow that mary had children after jesus , jerome considered that the term `` brother '' of the lord should be read `` cousin '' , and concluded that james `` the brother of the lord , '' is therefore james , son of alphaeus , one of the twelve apostles of jesus , as well as james , the son of mary cleophas .