What material should I use to insulate floor over a bumpout?
I had this exact problem.  It also got much worse when we refinished our basement.  Not sure if you have a basement, and if it is refinished.  Here is the summary of my issue.

The floor joists from the basement extend out to support the bump out.  So standing in the basement, you can reach out below the bump out.  The space was insulated, with the insulation sitting on the "floor" of the bumpout, meaning it was resting on MDF that was attached to the bottom of the overhanging floor joists.  The siding was then attached to the bottom of the mdf.  We do not have the insulated siding that Mike Powell mentioned, but that would be a good idea.  So above the insulation, it was open to the basement.  Air could flow into that space freely from the basement, which was not heated (the vents were closed), but stayed pretty warm in the winter, probably the 60's.    The floor upstairs was cool, but not cold.

We then refinished our basement, put in a dropped ceiling, and insulated all the floor joists in the house for noise.  This insulation prevented any air circulation from getting into the old space between the bumpout insulation and the floor.  And the drop ceiling kept most of the heat out of the basement ceiling.  So all this, you would think it would help, but it made it much worse and unbearable.  We had a guy look at it, and because the space in the bumpout is not getting warm air circulation anymore, that is why it is cooler than before.  To fix it, we need to actually remove some if the insulation, to allow as much warm air circulation to enter this space.  Still keep the insulation at the end cap of the bumpout, and on the floor, but keep it open between that insulation and the floor upstairs.

I put together a quick pic to illustrate.  This is across section on the bumpout: