South Wales Police has confirmed that the matter is closed and that the three complainants have been told.
The force said it has informed Operation Hydrant, which oversees allegations of historical child sex abuse against high-profile people.
Lord Tonypandy, who was born in Port Talbot, died of cancer in 1997 aged 88.
Then known as George Thomas, he was a Labour MP in Cardiff from 1945 to 1983.
The methodist preacher was also Secretary of State for Wales from 1968 to 1970 and was Commons Speaker between 1976 and 1983.
Allegations of historical child sexual abuse were made against him four years ago.
A man, now in his fifties, living in Australia, alleged that Lord Tonypandy had sexually assaulted him in Cardiff when he was nine years old while he was staying at the boy's parents home in the 1960s and 1970s.
Other allegations against him included that he sexually assaulted a 16 or 17-year-old boy on a train between Newport and London in the 1960s.
There were also claims he inappropriately touched someone on a train from London to Aberystwyth in 1959.
On Wednesday, South Wales Police said its investigation has come to an end and that no further action would be taken.
"These allegations have been investigated proportionately by South Wales Police in line with national guidance," it said in a statement.
"The force takes all allegations of sexual abuse seriously no matter when the alleged offence took place and encourages any victims of sexual abuse to come forward, in the knowledge that they will be listened to and their allegations taken seriously."
Friends and people who knew Lord Tonypandy have previously said they found the allegations difficult to believe.
An investigation into sex abuse claims made against the former Speaker of the House of Commons Viscount Tonypandy has ended.