What are the differences between SDRAM and SO-DIMM RAM, and which should I use for my iMac?
You're getting your terminologies mixed up.

SDRAM (Synchronous Dynamic Random Access Memory) is just the type of RAM on a DIMM. In your case as you've stated it's DDR3 (3rd generation of Double Data Rate)

A DIMM is a standard format Dual In-line Memory Module. These have different pins for the different types of memory specification, so a DDR3 module will have a different edge connector to a DDR2 module and so on.
A SO-DIMM is a Small Outline version of a DIMM, hence the "SO". These were designed for laptops where full size DIMMs wouldn't fit.

Apple use SO-DIMMs in their iMacs now, so DDR3 PC3-10600 (10600 is the peak transfer rate, so going lower than this will be a bad thing for performance) SO-DIMMs are what you want to get.