 He also had an affair with a married woman named Elizabeth Toyne.However, his lack of steady employment meant that he could not form a meaningful relationship, and contributed to his decision to migrate to Jamaica.He had a life-long sexual relationship with a slave named Phibbah, a Coromantee who essentially became his "wife" (as she was called in the will) and had his only son, Mulatto John.