Former White House adviser Lawrence Summers said Wednesday that student debt is slowing the housing recovery and the broader economic recovery, adding a prominent voice to the debate in Washington over whether Congress should reduce Americans’ student-loan burdens.

Mr. Summers, a Harvard University economist who previously led President Barack Obama’s National Economic Council, said high student debt is preventing households from spending money to fuel economic growth.