Quick Answer. Although George Washington selected the site of the White House in 1791, he never lived there. As of 2014, he is the only president who did not reside in the White House. Continue Reading.