Where is it documented an airport is military?
AFB, MCAS, NAS, CGAS/CGS, and AAF would always be military, yes. BUT the lack of that in the name doesn't mean it's not military... Outside the US especially there's often no such thing to indicate a military base from a civilian field (though AB (for AirBase) is at times used).
How do you know? You're supposed to know where you're going when you're flying, and that includes what's off limits to you. 
http://www.post8.org/images/secchart.jpg shows the legend of a sectional (can't inline the image, SO refuses to), which says clearly "All military airports are indicated by abbreviations AFB, NAS, AAF, etc. For complete airport information consult DOD FLIP". So how do you know? The chart tells you by the name of the facility. It also shows a double ring instead of runway icons as indicating military, but on some charts I looked at that appears to not be consistently applied.