Ex-Labour Northern Ireland Secretary Lord Hain said the peace process could "so easily unravel".
He said the Brexit bill should be changed to support maintaining the open border between Northern Ireland and the Republic of Ireland.
A government minister said the government was committed to building on the peace process.
They were speaking as peers debated the government's draft legislation that will authorise the UK's departure from the EU.
Among several amendments that were proposed was Lord Hain's, to require the prime minister to support the maintenance of the open border between Northern Ireland and the Republic of Ireland as set out in the Good Friday Agreement.
He said: "If the referendum means Brexit at any price it may well mean a dangerously high cost for the Northern Ireland peace process."
"The settlement in Northern Ireland is built on a delicate balance of the three strands of the Good Friday Agreement - relationships within Northern Ireland, between Belfast and Dublin, and Dublin and London," Lord Hain told peers.
"Brexit will test each of these relationships, and if the government pursues a hard Brexit it could do profound damage to all three."
Any physical border will create ill-feeling and could destabilise the region and "unravel" the peace process, he warned.
Northern Ireland Minister Lord Dunlop said the government was absolutely committed to maintaining and building upon the peace process.
He said: "Nobody wants to see a return to the borders of the past."
But he said the two-clause bill will simply trigger the Brexit process and did not need to be amended to enshrine a commitment to maintaining an open border, adding: "No such undertaking is necessary in this bill.
"Particularly in light of the strong assurances I have given, and our desire to keep this bill clean and simple."
Lord Hain, who withdrew his amendment in light of the minister's assurances, later attempted to amend the Brexit bill to keep the UK in the single market.
After a debate, this was rejected by peers by 299 to 136.
Other proposed amendments which will be considered by peers focus on the rights of EU nationals living in the UK to remain after Brexit, and calls by Labour for a "meaningful vote" on the deal reached after the negotiations.
MPs have already voted in favour of the government's Brexit bill, which was drawn up after the Supreme Court ruled that Theresa May could not invoke Article 50 of the Lisbon Treaty without consulting Parliament.
If the Lords back any amendments to the legislation, it will return to the Commons where MPs will decide whether to accept peers' proposals.
Brexit could mean a "dangerously high cost" for peace in Northern Ireland, a former minister has warned.