apparent and absolute magnitude of meteors and space debris
The formula appears to compare the squares of distances, not taking into account the difference in extinction (which is small for small zenith distances, and for different altitudes provided they're high enough), this means $M - m = 5log(\frac{h}{cos(Z)})-5log(h_1)=-5log(cos z) - 5log(\frac{h_1}{100})$. Also, I believe the formula should say -5 cos z ... the meteor from the picture provided is farther away from the one at zenith and at the same altitude and should have higher (numerical value) magnitude.