First Lady Dolley Madison referred to it as The President's Castle. The first evide…nce of the public referring to it as The White House was recorded in 1811. In 1901, President Theodore Roosevelt made The White House it's official name.