Why is chess still a benchmark for Artificial Intelligence
Chess isn't really a benchmark per say. 

The method developed in AlphaGo to play Go should in principle generalize quite nicely to other games of this sort, such as chess. Since Stockfish is quite dominantly the strongest Chess AI, the natural question would be to see how well AlphaGo's method compares to Stockfish.

Being one of the most well developed AI agents of all time, the situation concerning the defeat of Stockfish (AlphaZero was trained for only 4 hours entirely via self-play, without access to historical data) signifies the complete dominance of modern neural-network methods over classic methods (hard coded evaluation functions).

Also as @DukeZhou♦ mentioned in the comments, while Chess bots can regularly beat human players, it's still a useful metric to evaluate bots against each other via "games" of this sort.