For my window, what's the best way to seal and insulate?
Theres alot to be left desired by this installation. First of all it looks like your in the city and installed into a masonry opening. By the way your window is resting hard on the masonry sill it looks like the installation is what we refer to as brick to brick. Basically your window fits snug around the brick with maybe a quarter inch perimeter gap. Typically when you have this type of installation you would waterproof the back up masonry jambs which is usually brick or a concrete masonry unit with a perm a barrier applied on a primer. S100 sealant would be applied to the masonry sill and a sill pan would go on top of that. The perm a barrier would then drape into the sill with the vertical sections of perm a barrier being terminated with s100 sealant to prevent the perm a barrier from peeling away from the masonry. The idea is the perm a barrier diverts water that gets passed your caulk on the exterior into your sill pan and keep it from getting inside your house. If you cant strap to the masonry then a buck could be built in the pocket and the window could be anchored to the buck providing the buck was properly anchored. Either way a backer rod would be applied to the exterior perimeter about a quarter inch depth and sealant like a sika flex or np1 around the perimeter. Sometimes they leave a couple of 1 inch gaps at the sill to let water caught by the sill pan leak out. On the interior they could insulate around the window with bat insulation or low expansion spray foam although brick to brick openings usually have to much of a gap to spray foam. If I am wrong and this is within frame then just seal the exterior of the new window to the exterior of the old and spray foam in between the two windows on the interior underneath whatever finish trim you will install. But before I would lift a finger I'd call the installer and ask why is my window inadequately anchored and not sealed on the exterior. Sounds like the job is unfinished.