Realistic Rigging with real skeleton?
I will share my technique for muscle deformation: 

You can use shape keys. However, there is a difference in the approach between a hinge joint (elbow, knee) and a ball and socket joint (shoulder, hip). In the first case you only need one shape key (if you ignore forearm pronation / supination). 
When dealing with shoulder however, it is necessary to have a shape key for every extreme pose and then blend between them. 

In the image are some of the poses that need to have shape keys, but there can be more (arm raised in the back + rotation)


The way I’m applying the shape keys is to reduce one to 0 and to raise the other one to 1 at the same time. Blender will calculate the transition. Then it’s up to the user to estimate the “arm’s”
position relative to the extreme and to blend the shape keys.


I posted this because I don’t think a more complex rig or muscles system will make the difference. I hope it helps.