Is there a Microcontroller with 16MB of Ram?
Even though the Motorola 68000 and the Sega Genesis are quite old (early 1980's), you are not going to find a low-end (i.e. 8-bit) AVR that can emulate the entire game machine.

The Sega Genesis ran at 7.61 MHz and had 72KB of RAM (plus an additional 64KB of video RAM).  However the game programs resided in ROM, so you will need additional RAM to hold them (unless you plan on being able to plug in the original cartridges somehow).  Most game cartridges were under 4 MB, but there is at least one game (Pier Solar, released in 2010) that has a 8 MB ROM.

In addition, the system certainly must have included a system ROM which acted as some sort of executive and also would have provided a common I/O library for the cartridges (I can't find any reference to how large this was).  You are going to have to find the ROM's (or ROM images) for those and copy them into your RAM also (or add a section of ROM to your system).

IMO you are going to want to use a 32-bit microcontroller. If you are going to use the original cartridges plus a program ROM, and don't need MB's of RAM, than you can use most any 32-bit microcontroller that has enough space for your emulator.  If you are going to download the cartridges and the system ROM image into RAM, then in order to get 8 MB or more of RAM, you are going to need a microcontroller that has an external memory bus (you can't get 8 MB on the same chip as the microcontroller).

Given that you want to stick with AVR, I suggest a processor like the AT32UC3A3256, which has 256 KB of Flash, 128 KB of RAM, and runs at 84 MHz.  The gotcha is that it is a 144 pin surface mount device, which is going to be difficult to solder.

However, there is an evaluation kit for this processor from Element 14 for only $31.25.  So you don't have to worry about soldering.  Plus, the board has 8 MB of external RAM, so you can load a cartridge into RAM.



Just for the record, I still think you should consider the Raspberry Pi, running at 700 MHz with 512 MB of RAM for slightly less than the cost of development board above.  Running at that speed, you would have no issues with emulating the 68000 code and performing I/O at the correct speed.

Whether you go the AVR or Raspberry Pi route, in addition to the 68000 the Sega Genesis also included a Zilog Z80 and several special purpose chips, including the Yamaha YM2612 and Texas Instruments SN76489A.  The Z80 was used to control the sound and also provide backwards compatibility with the earlier Sega Master System.  The Yamaha chip was an FM sound synthesizer and the TI chip was a Programmable Sound Generator (this machine had a lot of sound options).  There was also a Virtual Display Processor (VDP).  You can probably skip the sound (which means you don't need to worry about the Z80 or the Yamaha or TI chips) but you will have to emulate the graphics hardware.

A couple of resources:

EASy68K -- Editor/Assembler/Simulator for the 68000.  Open source so you should be able to dig out the 68K simulation code

Cyclone 68000 -- emulator for the 68000 microprocessor, written in ARM 32-bit assembly.  Only useful if you decide to use the Raspberry Pi

Finally, if you aren't already familiar with the 68000 instruction set, plan on spending weeks (or more) to become an expert.  A lot of your debugging will be down at the emulator level, trying to figure out why a section of 68000 code in the game cartridge isn't executing properly.  (Which means you probably will want to be able to set up a virtual breakpoint facility in the cartridge code.)   You'll also need a disassembler, so you won't have to deal with machine code; here's the source for one.