What's the difference between an AOSP ROM and a stock ROM? I'd like to know what's the difference between these two.

What are the Advantages or Disadvantages of using a AOSP ROM vs a Stock ROM?

Is it just the Theme/Skin/UI and Bloatware that's different or is there something else?
A stock ROM is the ROM that comes with a device; the device is "stocked" with that ROM by the manufacturer.  Android is generally customized by the manufacturer to some degree; at minimum there needs to be device-specific drivers and so on for Android to work on a particular device.  As Flow notes, customizations may include a custom theme, launcher, and default apps like HTC Sense does.

An AOSP ROM is a ROM based on the Android Open Source Project.  In the purest sense, AOSP refers to unmodified ROMs or code from Google.  The name is often co-opted for custom ROMs that are very close to the original AOSP, since these ROMs still need to be customized; for example, I can't download/compile the Android source code and run it on my Samsung Vibrant without doing a whole lot of customizations.  The name is often abused to refer to ROMs that don't have a custom launcher or many of the other modifications carriers make with Sense, TouchWiz, Motoblur et al., although they may still be themed.  A more accurate name for plain, mostly-unaltered ROMs for non-Google devices is probably "vanilla ROMs".  (This distinction is part of our tagging policy here; see this Meta discussion for more info.)

Techncially, stock ROMs are all AOSP ROMs apart from the versions of Android that haven't been released.  Honeycomb ROMs aren't AOSP, but (for example) Gingerbread ROMs are AOSP since the 2.3 source code is available at http://source.android.com/.  (That assumes Google hasn't hidden parts of the code and given it to manufacturers secretly.)

To further add to the confusion, a "custom ROM" does not refer to customized ROMs in general.  That term specifically refers to ROMs that have been customized by third parties (i.e., not the manufacturers or carriers).  For example, I'm running a custom ROM that is just a re-themed and tweaked stock ROM.  CyanogenMod is another example, and it has many features built from the ground up rather than mere tweaks.  Most AOSP ROMs you'll find for a specific device are stock ROMs that have been customized to remove some of the manufacturer/carrier tweaks and make them closer to the pure AOSP experience.