How awful is it to have asbestos in the drywall?
To give some insight, when the asbestos is in a spot that cannot be removed, the asbestos is "encapsulated" as it is called. It can be a coat of paint, or plastic sheeting taped up tight. I have seen both used. Anything that keeps the surface from abrading.

In your case it is trapped between two layers of paper with numerous coats of paint over the living space side. The occasional nail you refer to is the only thing that would violate that, short of vandalism.

In my 42 years in the residential building trade, I have never heard of asbestos in drywall, then again, I only heard for the first time last year about asbestos in vermiculite house insulation.