Why can't you just set the altimeter to field elevation?
Good question. The reason is things fail.
If you set the altimeter to show field elevation, but the altimeter is faulty, you could end up flying differently from the rest, leading to loss of separation or worse.
But if you set the QNH, and then the reading is within 75 feet of the field elevation, you're good to go.

3. Altimeter Setting Procedures
3.1.3 If the difference from the known field elevation and the altitude read from the altimeter is plus or minus 75 feet or greater, the accuracy of the altimeter is questionable and the problem should be referred to an appropriately rated repair station for evaluation and possible correction. (USA AIP ENR 1.7)