Colombia said it would not allow any further temporary openings of the border, which Venezuela shut last year.
More than 100,000 people went to Colombia last weekend, the second time the border was open in a year.
They crossed to buy basic goods that are in short supply in their country because of a severe economic crisis.
Venezuelan President Nicolas Maduro ordered the closure to fight cross-border crime. He said the area had been infiltrated by Colombian paramilitaries and gangs.
The UN human rights office called on Venezuela to consider accepting "humanitarian aid" to ensure food and medicine supplies were distributed.
Colombia's Foreign Minister, Maria Angela Holguin, said a decision had been made to not allow any further temporary reopening of the border.
"Let's work so that the opening, the next opening, is definitive," she said after meeting officials from the region neighbouring Venezuela.
She added that work needed to be done to make the frontier safer and to prevent criminal activities in the area.
Many in Venezuela say they have struggled to feed their families as the country has suffered severe shortages for months.
This is a result of the falling price of oil which is Venezuela's prime source of income.
Supermarkets have empty shelves and people spend days in queues to buy basic goods.
The Venezuelan opposition blames the government for the economic crisis, saying its policies have left businesses unable to import raw materials and essential parts.
President Maduro argues that his leftist government is the victim of an economic war.
Expressing concern about "severe shortages" of basic goods in Venezuela, the UN human rights office also warned about deteriorating human rights and growing violence in the country.
The body urged the authorities to ensure the right to freedom of expression and peaceful assembly.
Venezuela: Economy on the brink?
The government in Colombia has called for the permanent opening of its border with Venezuela after crisis-hit Venezuelans flocked to buy basic items.