How to zoom into a video inside of a mask?
You can do this by selecting every layer except the solid (your letterbox) and precomping it. Then, you can switch into your precomp and do as much zooming as you like. If you then render the composition with your precomp plus the solid layer on top, your solid will be unaffected by the keyframing going on inside the precomp.