How can I best physically box in my internet router
You also need to make sure the device(s) get convection air for cooling.  

I have one of those Ikea bookshelves that has a lot of squares and open backs.  The router is on one shelf, all the cables behind, boxed in by self-contained trays (so they don't lean in and block its airflow).  It's not trivial but not hard to pull the router out to get to the cables.  No one would play there by accident. 

Wood, paper and plastic attenuate the signal only a little.