Phil Smith, chief executive of IT company Cisco, likened the situation to not having access to water or a road and said businesses would not be able to survive without it.
He said a fast internet connection was vital to a country like Wales with a "complex landscape".
The Welsh government insisted Wales was leading the way with broadband.
Ministers and BT are currently delivering the scheme Superfast Cymru, which will give people access to fibre broadband.
Earlier this year a Superfast Cymru spokesman said more than 80% of properties in Wales had access to the superfast broadband.
But, speaking to BBC Wales at technology conference Digital 2016 in Newport, Mr Smith said: "I'm very surprised that broadband hasn't got to the level of penetration it should.
"To be honest it's like saying you don't have a road now, or you don't have water.
"Companies, countries and individuals can't survive without broadband, it's not some optional nice thing to have, this is the way business is done.
"A country like Wales, it's got a wide population and complex landscape, it needs people to be on video, to engage anywhere in the world at any time.
"I think it's really important that the government and industry push hard to get broadband to the highest level everywhere."
A Welsh Government spokesman said an Ofcom report in August showed Wales was leading the way among the UK's devolved nations with the availability of superfast broadband.
"The Superfast Cymru programme has already made superfast broadband speeds in excess of 24Mbps available to more than 581,000 Welsh homes and businesses, including in some very rural areas, and more premises will have access as the rollout continues," he added.
"We are committed to making Wales a truly digital nation and continue to support businesses to take advantage of broadband technology which will help them to grow, to boost Wales' economic prosperity and create jobs."
A high profile figure in the world of technology has criticised poor broadband in parts of Wales.