Mr Cairns said it would be "completely wrong" to remain in the EU's single market as both major UK parties told voters they wanted to leave.
But he played down differences with the Welsh Government over Brexit.
The UK government is under pressure to change its approach to leaving the EU, which critics call a "hard Brexit".
Mr Cairns's predecessor, Stephen Crabb, claimed the election had changed the terms of the Brexit debate.
The Preseli Pembrokeshire MP is backing calls from Scottish Tory leader Ruth Davidson for Theresa May to adopt a more flexible approach that achieves a consensus across parties.
Mr Crabb told BBC Wales: "The majority of business opinion in Wales and the UK supports Britain staying in the single market and staying in the customs union but still coming out of the European Union legally.
"And I think that argument needs to get looked at far more closely and if we can't achieve a satisfactory deal based on membership of the single market we need something that is pretty close to it if it's going to achieve the consensus that it needs to across the parties."
Wales Office Minister Guto Bebb said the process of listening to devolved governments had begun.
"It is seven times more important now because of the result of the election that the devolved countries have a real voice in the process of leaving the European Union," he said.
Mr Cairns, newly re-appointed to his role, said he wanted Brexit negotiations, due to start next week, to focus on the right outcome with businesses trading freely.
"We've talked about frictionless trade, the Welsh Government have talked about unfettered access. These are one and the same things," he said.
"Even the Labour Party don't want to remain part of the single market so it would be completely wrong to stay in the single market when the two largest parties are opposed to that."
But Carmarthen West and South Pembrokeshire MP Simon Hart echoed Mr Crabb's call for cross-party consensus.
He said he did not buy the idea that because Theresa May has had "an unquestionably difficult time for which she takes some of the blame" it means "it's game over".
Welsh Secretary Alun Cairns has rejected calls for a "softer" Brexit after the general election result led to a hung parliament.