Hair rotation doesn't understand global Z
The issue here is that while while the particle system is indeed pointing the objects towards the global z, it is pointing the front of the objects (in the +y direction i believe) towards the global z. In this case, you want the object to stay standing up, so if you rotate the original mesh by 90 degrees, the correct direction will be aligned to global z in the particle system.