Despite this good start , the wall was abandoned after only twenty years . The garrisons were sent back to Hadrian 's Wall . Despite this auspicious start the wall was abandoned after only twenty years , and the garrisons relocated back to Hadrian 's Wall .