Battery voltage quickly drops when measured
I think you are right in believing that the battery "is shot."
To understand the "mechanism," think of the battery as being made up of 10 sections, each capable of 9V 1ma (10ma total).  When there is no load, if 9 of them are "shot" and one still good, the voltage would still be 9V but it can only provide 1ma.  If your load is 1ma or more, the voltage will drop very quickly. I hope this analogy is helpful.