Magnetic transformers on the same circuit as electrical
Most common cheap light dimmers use phase controlled dimming, which isn't really compatible with inductive loads such as transformers.  It switches the AC on for the load at full power at some point during the sine wave, depending on the dimming level.  This works well for resistive loads, but inductive loads don't like to be slammed with voltage suddenly.

You might be successful, but it's also likely that you'll destroy the light dimmer or overheat the transformer, depending on which one is the weaker link.

What you really want is a variac (also known as an autotransformer), then a transformer.  For instance,

http://www.amazon.com/Variac-Variable-Transformer-300va-Output/dp/B006NGI8VS

This will allow you to vary the AC voltage without removing parts of the waveform.  It'll handle 300VA, so as long as your end load uses under 300 watts that one will work fine, otherwise you'll need a larger one.

There are other methods, however, of controlling your heating wire.  Replacing your dimmer and transformer with a real bench power supply will allow you to control your voltage and current very finely. They aren't as cheap as a dimmer and transformer, or a variac and transformer, but once you've done your testing you can make a cheap supply for your final product and keep the bench power supply around for other products.  Just make sure the bench power supply can handle the load you need at the voltage and current.