Are non-military aircraft allowed to fly with no registration numbers (tail numbers)?
14 CFR 45 subpart C governs the display of "nationality and registration marks".

14 CFR 45.21 says:


  Except as provided in §45.22, no person may operate a U.S.-registered aircraft unless that aircraft displays nationality and registration marks in accordance with the requirements of this section and §§45.23 through 45.33. 


Section 45.22 gives some exceptions for "exhibition and antique" aircraft, and for aircraft of unusual design that may need an alternative marking scheme.

I presume there is another regulation, or international agreement, requiring that foreign-registered aircraft must also display their registration marks, but I haven't found it yet.

As to which aircraft have to be US-registered in the first place, see 14 CFR 47.  It's a bit complicated.  

The US military is generally not bound by FAA regulations (though they may choose to follow some of them), see How much jurisdiction does the FAA have over military aircraft?.