How do I remove an exact rectangle from the rounded face of a hollow cylinder object?
You can use the boolean / difference no problem. 

It is such a simple shape that it will still be good topology after you apply the modifier. The reason that you get the hollow cube thingey, is that the two shape have exactly the same height. Weird stuff comes from 3D programs that will have to calculate difference between 2 items witch are the same location. If you scale the box on positive Z just a bit, it will work perfectly. 

Usually I don't like boolean because it trashes the topology, but in this case it fits the part!