How do I fill a hole under a sidewalk?
Fill the hole with concrete.  Concrete doesn't expand in the cold -- you're probably thinking of frost heave, which is where moisture in the soil freezes and expands, disturbing whatever's above it.  If your sidewalk hasn't heaved by now, then the addition of new concrete below it isn't going to change anything.

Filling the hole with dirt or sand may in fact cause the heaving you're concerned about though -- I can imagine a situation where you're unable to properly compact the dirt underneath the sidewalk, creating a nice loosely-filled pocket for water to sit in and freeze.