( CNN ) -- Israel began releasing more than two dozen Palestinian prisoners late Tuesday , a goodwill gesture on the eve of new peace talks that have been complicated by Israeli plans for new housing in East Jerusalem . The vans began pulling away from the Israeli prison in Ramla shortly before 10 p.m. ( 3 p.m . ET ) , carrying the 26 freed inmates to flag-waving receptions in the West Bank and Gaza . One group was handed over the Palestinian officials at the Israeli prison at Ofer and they were driven to meet with Palestinian Authority President Mahmoud Abbas in nearby Ramallah . As they were driven away , Palestinian youths outside started throwing stones at the prison , where guards responded by firing stun grenades and tear gas . Some of the Palestinians had been held for more than 20 years . But as the vans rolled away , a group of demonstrators waved signs condemning the release , complaining that the prisoners had Israeli blood on their hands . In approving the release , a committee of Israeli Cabinet ministers stressed `` that if any of the released prisoners return to hostile activity against the state of Israel , they will be returned to continue serving their sentences . '' Abbas announced the release of the prisoners was just the beginning . `` We say to them and the rest who will follow , this is the beginning and there are other brothers who will be returning and be amongst you soon , '' he said . The prisoner release came after Israel said it would forge ahead with a plan to build 900 housing units in East Jerusalem , which the Palestinians hope will be the capital of their future state . The issue of Israeli settlements derailed the last round of direct talks , in 2010 , and critics of Israeli Prime Minister Benjamin Netanyahu say building on disputed territory could derail the new talks , slated to start Wednesday in Jerusalem . `` Netanyahu has to decide which government he is heading : a government that is trying to reach a peace agreement , or a government that is trying to undermine all possibilities of this agreement , '' said Shelly Yachimovich , head of the Labor Party and the Israeli opposition . In January , the United Nations Human Rights Council said Israeli settlements amount to `` creeping annexation '' of Palestinian territories by Israel and have taken a `` heavy toll '' on the rights and sovereignty of Palestinians . And in Washington , where American leaders have pushed both sides to return to the table , the State Department said it had `` serious concerns '' with the new announcement . `` We 've encouraged both sides broadly speaking to refrain from taking steps that could undermine trust . We said that from the beginning , '' State Department spokeswoman Marie Harf said . But , she added , `` We 've also been clear that these are complicated issues , and there will be bumps in the road . '' Daoud Kuttab , a columnist for the online Middle Eastern news site Al-Monitor , said the released prisoners are a bargaining chip to make sure the Palestinians do n't back out of the talks despite their anger over new settlements . `` They 're trapped , and Mahmoud Abbas has egg on his face today because he ca n't get out of the talks , '' Kuttab told CNN . But Rami Khoury , a veteran Middle East analyst at the American University of Beirut , said both sides agreed to resume talks for nine months under American mediation . `` This is a significant gesture , so therefore you 're going to hear a lot of noise from both sides -- complaints , grievances , '' Khoury said . `` But you can pretty much discount most of what you hear in public , because they 're both clearly committed to doing this for nine months . And we really need to give it time to see what 's going to happen . '' Both Abbas and Netanyahu are `` hemmed in by domestic constraints , '' leaving neither side likely to make significant changes on its own , Khoury said . U.S. Secretary of State John Kerry said last week that `` many difficult choices '' lie ahead for both sides , but held out hope that `` reasonable compromises '' could be reached . Khoury called that a `` weak statement '' that downplayed expectations for the talks . As a result , most observers have little hope that the talks will produce a breakthrough in the decades-old Israeli-Palestinian conflict . But he said the U.S. role in the talks is a `` wild card '' that could defy those expectations . `` What is the United States going to do ? '' he asked . `` Will it push both sides ? Will it pressure them ? Will it cajole them ? Will it entice them , and will it make serious bridging and endgame proposals ? We have no idea about any of that . '' CNN 's Vladimir Duthiers , Saad Abedine , Michael Schwartz , Kareem Khadder and Tom Cohen contributed to this report .
NEW : Palestinian Authority President Mahmoud Abbas welcomes prisoners `` home '' East Jerusalem releases Palestinians prisoners ahead of new peace talks . But Israeli are angered by Palestinians plans for new construction in Israel . West Bank handover followed by rock-throwing , tear gas .