How can an altimetry system error during cruise (500 ft at 35000 ft high) cause an insufficient fuel situation?
If the aircraft is cruising at FL350 it is operating in Reduced Vertical Separation Minima (RVSM) airspace (between FL290 and FL410). 

With an altimetry discrepancy between the two altimeters of more than 245ft, the aircraft is no longer RVSM approved. It shall therefore descent or climb out of the RVSM altitude block.

For the Airbus A321 the operating ceiling is below FL410, so the only option is to descent to FL280 or below. Since the fuel consumption in those altitudes is significantly higher the aircraft is unlikely to complete its 6 hour flight with the amount of fuel on board. Therefore the decision to return is likely the right one.