Media playback is not supported on this device
Shavers lost a 15-round decision to Ali in 1977, although he was a friend before the fight and has remained so in the intervening 35 years.
"The Klitschkos are big guys but they're not in the same league as Ali," Shavers told BBC Sport.
"Ali would have picked them apart. All us guys in the '70s would have."
Shavers, 66, first befriended Ali in 1973, when 'The Greatest' invited him to his Deer Lake training camp in Pennsylvania to prepare for Shavers's impending fight against Jimmy Ellis.
"When I was getting ready to fight Jimmy Ellis, Ali gave me the use of his camp for free, never charged me a dime, and gave me some advice [Ali beat Ellis in 1971]," said Shavers.
"Ali was a very nice man who treated me well. He was an easy-going guy, an easy guy to talk to. When the cameras came he put on a show, but away from the cameras he was a very quiet guy."
When Shavers fought Ali four years later, he rocked 'The Greatest' with a huge right hand in the second round - a blow Ali said "shook my kinfolk in Africa" - but it was Ali who was awarded a unanimous decision.
However, Ali's subdued performance, and the amount of punishment he absorbed from Shavers, led many in the media to call for his retirement.
"As time goes on you're going to lose some of your sharpness, which is why I thought I had a good chance of beating him," said Shavers, who until recently lived and worked in England.
Ali doesn't want you to feel sorry for him, but you can't help it because you love the guy. It's sad to see him in the shape he's in now
"I had mixed feelings [before the fight]. I didn't want to beat Ali - he was a great guy - but I was fighting for the championship. I had to give my best.
"I thought he was faking it [in the second round], trying to suck me in, but he was hurt. If I'd have gone in I might have stopped him."
While Shavers also fought Joe Bugner, Larry Holmes, Ron Lyle and Ken Norton - all of whom called him the hardest puncher they faced - in a stellar career, he says Ali is the one opponent everyone wants to talk about.
Shavers, who now makes his living signing autographs in Las Vegas, said: "I get asked every day about the Ali fight. Ali won the fight but it worked out well for both of us.
"Anyone who fought Ali and did well, they were basically made. I think I did quite well; I went 15 rounds with him. He won the fight but I kept my name clean.
Media playback is not supported on this device
"When Joe Frazier fought Ali, he made Joe Frazier. Joe Frazier was a good fighter but Ali made the whole thing."
Shavers remains close to Ali, meeting up once a year at a charity fund-raiser in Arizona, but admits it is difficult seeing his old friend diminished by Parkinson's syndrome.
"We're still very close," said Shavers. "But because Ali was such a great fighter and such a great man, it's sad to see him in the shape he's in now.
"Ali doesn't want you to feel sorry for him, but you can't help it because you love the guy. If you get to know Ali you'd think he never even had Parkinson's, the way he carries himself.
"But it's a shame, when you love a guy, to see him as he is now, when you remember him as he used to be. It's hard to be around him.
"Without a shadow of a doubt he was the greatest. People love him because he gave back. He helped so many people besides himself."
Earnie Shavers said modern heavyweights "were not fit to carry Muhammad Ali's gym bag", on the occasion of his old rival's 70th birthday.