Install OS X in VirtualBox on iMac?
These instructions are for VirtualBox on Windows and Linux but they work well on OS X as well. Taken from here -- abridged and modified for OS X guest OS:


Rip your original retail DVD of Snow Leopard in ISO format.
In VirtualBox, create a new virtual machine. Name it as you want, but select "Mac OS X" as the operating system, and "Mac OS X Server" as the version. Click next. Assign a good amount of RAM to the VM. I chose 1024 MB. Click next. Create a new virtual hard disk, I chose 20 GB of dynamically expanding storage. Click finish.
Open the settings for this new virtual machine, and in the Storage section, load the Snow Leopard ISO in the now empty CD/DVD device.
Now close VirtualBox (important!). Find the XML file that defines the virtual machine that you just created. This will be in ~/Library/VirtualBox/Machines/&lt;name of the VM&gt;/&lt;name of the VM&gt;.xml. Search this file in a text edit for the ExtraDataItem tags. After the last instance of ExtraDataItem tags add the following two, new, ExtraDataItem tags:

&lt;ExtraDataItem name="VBoxInternal2/EfiBootArgs" value=" "/>

&lt;ExtraDataItem name="VBoxInternal2/SmcDeviceKey" value="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc"/>
Save and close the file.
Open VirtualBox and power on the virtual machine. The installation will start after some minutes. Choose your language. Now open Disk Utility under Utilities menu. Select the virtual disk and click Erase. Close it and you can now install OSX. Before that, you can choose to customize the installation. It can be interesting to disable extra language translations or printer support (you save about 2 GB).