What part(s) can I use to connect modules at a 90 degree angle?
Here are all possibilities I can think of (most of them already listed in your question):


Use part 10197 Technic, Axle and Pin Connector Hub with 2 Axles together with 2x 4265c Technic Bush 1/2 Smooth, one on each axle end. This will keep it snug and prevents it from moving, but the axles are, as you observed, only halfway inserted into the bricks.





Similar solution, but use 22961 Technic, Axle and Pin Connector Hub with 1 Axle combined with 4265c Technic Bush 1/2 Smooth, and place 18651 Technic, Axle Pin 3L with Friction Ridges Lengthwise and 2L Axle in the hole, together with another 4265c Technic Bush 1/2 Smooth. This combination will have one end solidly inserted (actually sticking out 1/2 a brick) and the other end is halfway in.





Exchange the "standard" Technic 1x2 brick for a 6541 Technic, Brick 1 x 1 with Hole, or even better, a 32000 Technic, Brick 1 x 2 with Holes. Then use a 44809 Technic, Pin Connector Perpendicular 2 x 2 Bent and two normal friction pins to make the connection. This should be rock solid.




The solution would look like this: 





BEGIN EDIT 2016/03/04:


I recently came across this piece that would be IDEAL for your original purpose: Part 6553 Technic Pole Reverser Handle. 




This piece puts the axle it can accept in the opening at a 1/2 stud distance from the brick it connects to, which would work perfectly in your original design. Combine this with any axle and you get the following solution:



This solution has the advantage that you can take it apart much easier than solution 3, which is more of a "permanent attachment" option. The axle connected to 6653 is 1.5 long, so it does stick 1/2 stud out from the back of the +-shaped Technic brick. You will need to account for this in your design.

END EDIT 2016/03/04