The report, a summary of which was released by the Homeland Security Department, said the agency needed more plainclothes and uniformed staff.
It added that the White House needed a new, taller fence.
The review came after an intruder with a knife entered the building.
Omar Gonzalez, a former US soldier, was apprehended deep inside the presidential residence in September after he had scaled a fence around the building and evaded several guards - including one with an attack dog.
Julia Pierson, the Secret Service director at the time, resigned partly as a result of this breach.
The Secret Service is tasked with guarding the US president, as well as several senior government officials.
According to an executive summary of the independent review, the agency's next director "will have to make difficult choices, identifying clear priorities for the organisation and holding management accountable for any failure to achieve those priorities".
"Only a director from outside the [Secret] Service, removed from organisational traditions and personal relationships, will be able to do the honest top-to-bottom reassessment this will require."
The review also said a better fence should be built "as soon as possible", recommending one that was several feet higher and curved outwards, making it harder to scale.
However, the review noted that the problems in the agency "go deeper than a new fence can fix". The panel said the agency's staff were over-stretched, working "an unsustainable number of hours".
It called for boosting the service's ranks by 85 special agents and 200 uniformed officers, to reduce overtime and allow staff to receive regular training.
The four-member panel based its findings on interviews with 50 employees of the Secret Service. The panel admitted that many of its recommendations had been made before - but not implemented.
Many details have already emerged of the security lapses that allowed Mr Gonzalez to enter the White House, pointing to a lack of training, bad staffing decisions and failures in communication.
Mr Obama and his family were not at the White House when the intrusion happened, having departed about 10 minutes earlier by helicopter.
Mr Gonzalez has been indicted on charges including unlawfully entering a restricted building or grounds while carrying a deadly or dangerous weapon and unlawful possession of ammunition.
The intrusion came after a series of embarrassing lapses by the Secret Service - including reports that agents had used prostitutes while on duty in Colombia.
The US Secret Service, which guards the US president, is too insular and must recruit its next head externally, a review prompted by a White House security breach says.