Help with connecting electromagnet to dimmer
Yes, you can control the amount of power going to your electromagnet by hooking a rheostat in series with the battery and electromagnet.

Of course you could buy a rheostat, but if your primary goal is to learn you can make one. Here is a nice tutorial explaining how make a rheostat. Just substitute your electromagnet for the light bulb shown.