Oliver was the son of Major General Oliver De Lancey (1718--1785) and nephew of James De Lancey, a celebrated New York lawyer, who was chief-justice of that colony from 1733 to 1760, and lieutenant-governor from 1753 to 1760.
Oliver was the son of Major General Oliver De Lancey (1718--1785) and Phila Franks, the oldest daughter of an early New York Jewish family. He was the nephew of James De Lancey, a celebrated New York lawyer, who was chief-justice of that colony from 1733 to 1760, and lieutenant-governor from 1753 to 1760.